2024-02-26T10:18:29.245 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-26T10:18:29.245 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --verbose --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} --archive /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557 --name vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/orig.config.yaml 2024-02-26T10:18:29.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T10:18:29.370 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557 branch: wip-vshankar-testing-20240226.064629 description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} email: vshankar@redhat.com first_in_suite: false job_id: '7573557' kernel: client: branch: testing flavor: default kdb: true sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240226.064629 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240226.064629 sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 1879 sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 sleep_before_teardown: 0 subset: 788/1024 suite: fs suite_branch: wip-vshankar-testing-20240226.064629 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNZXEQayW+z0FSWCIGyNxq4hGDs3AtBWyqL+83QzXIFLyaMXmnNDH7mSH/q9RIINXLSmXCuPUhqNV+2Xkv6VY3s= smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzIN0+PSmnpktoeKuKPW5cX+J0qDvyfnUFf3H5ZNN5S4jHoYDQc8Ab/xnkEM2vKIs4KnCVib96B9xtHEbIUR54= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_multifs_auth teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-26_10:07:12 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-26T10:18:29.370 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa; will attempt to use it 2024-02-26T10:18:29.371 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks 2024-02-26T10:18:29.371 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-26T10:18:29.373 INFO:teuthology.task.internal:Checking packages... 2024-02-26T10:18:29.396 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6085b91fefeb74bf4310344a87d81d3217ef7893' 2024-02-26T10:18:29.396 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-26T10:18:29.396 INFO:teuthology.packaging:ref: None 2024-02-26T10:18:29.396 INFO:teuthology.packaging:tag: None 2024-02-26T10:18:29.396 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240226.064629 2024-02-26T10:18:29.396 INFO:teuthology.packaging:sha1: 6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:18:29.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240226.064629 2024-02-26T10:18:30.995 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1686.g6085b91f 2024-02-26T10:18:30.997 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-26T10:18:31.010 INFO:teuthology.task.internal:no buildpackages task found 2024-02-26T10:18:31.010 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-26T10:18:31.017 INFO:teuthology.task.internal:Saving configuration 2024-02-26T10:18:31.033 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-26T10:18:31.040 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:18:31.065 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:04.852051', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNZXEQayW+z0FSWCIGyNxq4hGDs3AtBWyqL+83QzXIFLyaMXmnNDH7mSH/q9RIINXLSmXCuPUhqNV+2Xkv6VY3s='} 2024-02-26T10:18:31.089 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:04.850833', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzIN0+PSmnpktoeKuKPW5cX+J0qDvyfnUFf3H5ZNN5S4jHoYDQc8Ab/xnkEM2vKIs4KnCVib96B9xtHEbIUR54='} 2024-02-26T10:18:31.089 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-26T10:18:31.098 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-02-26T10:18:31.098 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-02-26T10:18:31.098 INFO:teuthology.run_tasks:Running task console_log... 2024-02-26T10:18:31.174 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7112f1e160>, signals=[15]) 2024-02-26T10:18:31.174 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-26T10:18:31.185 INFO:teuthology.task.internal:Opening connections... 2024-02-26T10:18:31.185 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:18:31.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:18:31.292 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:18:31.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:18:31.380 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-26T10:18:31.391 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-02-26T10:18:31.409 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-02-26T10:18:31.410 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-02-26T10:18:31.466 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-02-26T10:18:31.467 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T10:18:31.467 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-02-26T10:18:31.467 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T10:18:31.467 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T10:18:31.467 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T10:18:31.467 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-02-26T10:18:31.488 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-02-26T10:18:31.505 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-02-26T10:18:31.506 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-02-26T10:18:31.562 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-02-26T10:18:31.563 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-26T10:18:31.563 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-02-26T10:18:31.563 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-26T10:18:31.563 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-26T10:18:31.563 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-26T10:18:31.563 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-02-26T10:18:31.587 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-26T10:18:31.599 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-26T10:18:31.608 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-26T10:18:31.608 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-02-26T10:18:31.611 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-02-26T10:18:31.627 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-26T10:18:31.644 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-26T10:18:31.644 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-02-26T10:18:31.668 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-02-26T10:18:31.703 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-26T10:18:31.758 INFO:teuthology.run_tasks:Running task kernel... 2024-02-26T10:18:31.773 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '7c0fbba6c508351b48a34af7de7b074eb29ad0f9'} 2024-02-26T10:18:31.774 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}} 2024-02-26T10:18:31.774 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-02-26T10:18:31.774 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T10:18:31.774 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-26T10:18:31.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:18:31.791 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-26T10:18:31.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:18:31.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-02-26T10:18:32.252 DEBUG:teuthology.task.kernel:sha1 for client.1 is 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:18:32.252 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "7c0fbba6c508351b48a34af7de7b074eb29ad0f9"... 2024-02-26T10:18:32.252 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-02-26T10:18:32.266 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:18:32.266 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7c0fbba6c508351b48a34af7de7b074eb29ad0f9"... 2024-02-26T10:18:32.267 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-02-26T10:18:32.270 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-361.el9.x86_64 2024-02-26T10:18:32.270 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:18:32.270 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-26T10:18:32.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum upgrade -y linux-firmware 2024-02-26T10:18:32.284 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-361.el9.x86_64 2024-02-26T10:18:32.284 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:18:32.285 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-26T10:18:32.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum upgrade -y linux-firmware 2024-02-26T10:18:35.189 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 6.3 MB/s | 8.1 MB 00:01 2024-02-26T10:18:35.216 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 6.5 MB/s | 8.1 MB 00:01 2024-02-26T10:18:37.181 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.5 MB 00:00 2024-02-26T10:18:37.232 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.5 MB 00:00 2024-02-26T10:18:39.732 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-02-26T10:18:42.464 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-02-26T10:18:51.745 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:11 2024-02-26T10:18:52.506 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:08 2024-02-26T10:18:56.312 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.1 MB 00:00 2024-02-26T10:18:57.829 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2024-02-26T10:18:59.924 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-26T10:19:04.773 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 726 kB/s | 6.1 MB 00:08 2024-02-26T10:19:04.879 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-02-26T10:19:06.532 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr:Modular dependency problems: 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 1: conflicting requests 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 2: conflicting requests 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.131 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 3: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 4: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 5: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 6: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 7: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 8: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 9: conflicting requests 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.132 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 10: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 11: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 12: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 13: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 14: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 15: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 16: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 17: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 18: conflicting requests 2024-02-26T10:19:07.133 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 19: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 20: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 21: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 22: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 23: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 24: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 25: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 26: conflicting requests 2024-02-26T10:19:07.134 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 27: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 28: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 29: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 30: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 31: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 32: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 33: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 34: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 35: conflicting requests 2024-02-26T10:19:07.135 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 36: conflicting requests 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 38: conflicting requests 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.136 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.137 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.138 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:19:07.789 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-02-26T10:19:07.797 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-02-26T10:19:07.798 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 1 Package 2024-02-26T10:19:07.798 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:19:07.798 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 354 M 2024-02-26T10:19:07.798 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-02-26T10:19:08.016 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2024-02-26T10:19:13.375 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr:Modular dependency problems: 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 1: conflicting requests 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 2: conflicting requests 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.739 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 3: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 4: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 5: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 6: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 7: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 8: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 9: conflicting requests 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.740 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 10: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 11: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 12: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 13: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 14: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 15: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 16: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 17: conflicting requests 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.741 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 18: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 19: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 20: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 21: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 22: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 23: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 24: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 25: conflicting requests 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.742 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 26: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 27: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 28: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 29: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 30: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 31: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 32: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 33: conflicting requests 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.743 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 34: conflicting requests 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 35: conflicting requests 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 36: conflicting requests 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 38: conflicting requests 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.744 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.745 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:19:15.746 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:19:16.296 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-02-26T10:19:16.304 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-26T10:19:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 1 Package 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 354 M 2024-02-26T10:19:16.305 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-02-26T10:19:42.016 INFO:teuthology.orchestra.run.smithi067.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 14 MB/s | 354 MB 00:25 2024-02-26T10:19:42.017 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:19:42.017 INFO:teuthology.orchestra.run.smithi067.stdout:Total 14 MB/s | 354 MB 00:25 2024-02-26T10:19:42.018 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-02-26T10:19:42.120 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-02-26T10:19:42.120 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-02-26T10:19:43.856 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-02-26T10:19:43.857 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-02-26T10:19:56.338 INFO:teuthology.orchestra.run.smithi082.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 7.3 MB/s | 354 MB 00:48 2024-02-26T10:19:56.339 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:19:56.340 INFO:teuthology.orchestra.run.smithi082.stdout:Total 7.3 MB/s | 354 MB 00:48 2024-02-26T10:19:56.340 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-02-26T10:19:56.425 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-02-26T10:19:56.425 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-02-26T10:19:57.950 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-02-26T10:19:57.951 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-02-26T10:20:06.378 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-02-26T10:20:06.654 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T10:20:06.763 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:20:13.927 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:20:13.928 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T10:20:14.390 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:20:14.390 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:20:14.390 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-02-26T10:20:14.390 INFO:teuthology.orchestra.run.smithi067.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-02-26T10:20:14.390 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:20:14.391 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-02-26T10:20:14.681 INFO:teuthology.task.kernel:Downloading kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T10:20:14.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:20:15.091 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ 2024-02-26T10:20:15.091 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-26T10:20:15.094 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi067.front.sepia.ceph.com to complete... 2024-02-26T10:20:19.344 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-02-26T10:20:19.626 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T10:20:19.732 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:20:27.339 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:20:27.339 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-02-26T10:20:27.733 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-02-26T10:20:27.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:20:27.734 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-02-26T10:20:27.734 INFO:teuthology.orchestra.run.smithi082.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-02-26T10:20:27.734 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:20:27.734 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-02-26T10:20:28.047 INFO:teuthology.task.kernel:Downloading kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.1... 2024-02-26T10:20:28.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c0fbba6c508351b48a34af7de7b074eb29ad0f9 2024-02-26T10:20:29.924 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ 2024-02-26T10:20:29.924 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/ --input-file=- 2024-02-26T10:20:29.927 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi082.front.sepia.ceph.com to complete... 2024-02-26T10:22:04.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26 10:22:04 URL:https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm [335918853/335918853] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-26T10:22:04.247 INFO:teuthology.orchestra.run.smithi082.stderr:FINISHED --2024-02-26 10:22:04-- 2024-02-26T10:22:04.247 INFO:teuthology.orchestra.run.smithi082.stderr:Total wall clock time: 1m 34s 2024-02-26T10:22:04.247 INFO:teuthology.orchestra.run.smithi082.stderr:Downloaded: 1 files, 320M in 1m 34s (3.41 MB/s) 2024-02-26T10:22:04.250 INFO:teuthology.task.kernel:Installing kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.1... 2024-02-26T10:22:04.250 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-26T10:22:06.188 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying... ######################################## 2024-02-26T10:22:06.348 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing... ######################################## 2024-02-26T10:22:06.428 INFO:teuthology.orchestra.run.smithi082.stdout:Updating / installing... 2024-02-26T10:22:13.952 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-6.8.0_rc4_g7c0fbba6c508-1 ######################################## 2024-02-26T10:22:27.726 INFO:teuthology.orchestra.run.smithi082.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-26T10:22:50.067 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-26T10:22:50.068 DEBUG:teuthology.orchestra.run.smithi082:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-26T10:22:50.152 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:50.152 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/System.map 2024-02-26T10:22:50.152 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/config 2024-02-26T10:22:50.152 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-26T10:22:50.153 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel 2024-02-26T10:22:50.154 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ansi_cprng.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/anubis.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/arc4.ko 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx 2024-02-26T10:22:50.155 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_memcpy.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_pq.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_tx.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_xor.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/raid6test.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authenc.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authencesn.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blake2b_generic.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_common.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_generic.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/camellia_generic.ko 2024-02-26T10:22:50.156 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast5_generic.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast6_generic.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast_common.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ccm.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cmac.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crc32_generic.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_engine.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_user.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cts.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/des_generic.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/dh_generic.ko 2024-02-26T10:22:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/echainiv.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/essiv.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/fcrypt.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/khazad.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4hc.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/md4.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/michael_mic.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcbc.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcrypt.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/rmd160.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/seed.ko 2024-02-26T10:22:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/serpent_generic.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tcrypt.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tea.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_common.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_generic.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/vmac.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/wp512.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xcbc.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xor.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xxhash_generic.ko 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi 2024-02-26T10:22:50.159 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_pad.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/custom_method.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/ec_sys.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/platform_profile.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbs.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbshc.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/video.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/acard-ahci.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ahci_platform.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ata_generic.ko 2024-02-26T10:22:50.160 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/libahci_platform.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_acpi.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ali.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_amd.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_artop.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atiixp.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atp867x.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd640.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd64x.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cypress.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_efar.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt366.ko 2024-02-26T10:22:50.161 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt37x.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it8213.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it821x.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_jmicron.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_marvell.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_mpiix.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_netcell.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ninja32.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87410.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87415.ko 2024-02-26T10:22:50.162 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_oldpiix.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_opti.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_optidma.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pcmcia.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_piccolo.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_rdc.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sch.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_serverworks.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sil680.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sis.ko 2024-02-26T10:22:50.163 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sl82c105.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_triflex.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_via.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pdc_adma.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_inic162x.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_mv.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_nv.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_promise.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_qstor.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil24.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sis.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_svw.ko 2024-02-26T10:22:50.164 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sx4.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_uli.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_via.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_vsc.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay/ks0108.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma/bcma.ko 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block 2024-02-26T10:22:50.165 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe/aoe.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/brd.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd/drbd.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/floppy.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/loop.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/nbd.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk/null_blk.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/pktcdvd.ko 2024-02-26T10:22:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/rbd.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/virtio_blk.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkfront.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram/zram.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hangcheck-timer.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-26T10:22:50.167 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/via-rng.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/lp.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave/mwave.ko 2024-02-26T10:22:50.168 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ppdev.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tlclk.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/uv_mmtimer.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/virtio_console.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-26T10:22:50.169 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-26T10:22:50.170 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-aes.ko 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-sha.ko 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca/dca.ko 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw 2024-02-26T10:22:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/amd64_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/e752x_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/edac_mce_amd.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3000_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3200_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5100_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5400_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7300_edac.ko 2024-02-26T10:22:50.172 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7core_edac.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i82975x_edac.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/ie31200_edac.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/sb_edac.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/x38_edac.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/edd.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-ich.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-appleir.ko 2024-02-26T10:22:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-aureal.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-axff.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-dr.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elecom.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elo.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-emsff.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gaff.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gt683r.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gyration.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtekff.ko 2024-02-26T10:22:50.174 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-hyperv.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-icade.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-keytouch.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-kye.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lcpower.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-led.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lenovo.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-multitouch.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-ortek.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-penmount.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-petalynx.ko 2024-02-26T10:22:50.175 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-picolcd.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-pl.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-plantronics.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-primax.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-rmi.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-common.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-26T10:22:50.176 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-saitek.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-samsung.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sjoy.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sony.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-speedlink.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-steelseries.ko 2024-02-26T10:22:50.177 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sunplus.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tivo.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tmff.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-topseed.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-twinhan.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-uclogic.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-waltop.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-wiimote.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-xinmo.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zpff.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zydacron.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/uhid.ko 2024-02-26T10:22:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/wacom.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_balloon.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_utils.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_vmbus.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru3.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7414.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7418.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adc128d818.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1025.ko 2024-02-26T10:22:50.179 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1026.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1029.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1031.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm9240.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ads7828.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7410.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7411.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7462.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7470.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7475.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7x10.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/amc6821.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/applesmc.ko 2024-02-26T10:22:50.180 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asb100.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asc7621.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/atxp1.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/coretemp.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dme1737.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds1621.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds620.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc1403.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc6w201.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71805f.ko 2024-02-26T10:22:50.181 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71882fg.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f75375s.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fam15h_power.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fschmd.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g760a.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g762.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl518sm.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl520sm.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5500_temp.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5k_amb.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmaem.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmpex.ko 2024-02-26T10:22:50.182 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina209.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina2xx.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/it87.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k10temp.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k8temp.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lineage-pem.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm63.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm73.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm75.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm77.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm78.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm80.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm83.ko 2024-02-26T10:22:50.183 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm85.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm87.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm90.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm92.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm93.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95234.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95241.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95245.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc2945.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4151.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4215.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4222.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4245.ko 2024-02-26T10:22:50.184 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4260.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4261.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max16065.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1619.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1668.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max197.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6639.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6650.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6697.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/mcp3021.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6683.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775-core.ko 2024-02-26T10:22:50.185 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct7802.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87360.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87427.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pcf8591.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-26T10:22:50.186 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/powr1220.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5627.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5636.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht15.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht21.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/shtc1.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sis5595.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47b397.ko 2024-02-26T10:22:50.187 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m1.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m192.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/thmc50.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp102.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp103.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp401.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp421.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via-cputemp.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via686a.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt1211.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt8231.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627ehf.ko 2024-02-26T10:22:50.188 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627hf.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83781d.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83791d.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83792d.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83793.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83795.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l785ts.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l786ng.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses 2024-02-26T10:22:50.189 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-26T10:22:50.190 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-dev.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-mux.ko 2024-02-26T10:22:50.191 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-smbus.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-stub.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_core.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-26T10:22:50.192 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser 2024-02-26T10:22:50.193 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/gameport.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/joydev.ko 2024-02-26T10:22:50.194 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/appletouch.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/bcm5974.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/cyapatp.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/sermouse.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-26T10:22:50.195 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/altera_ps2.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/arc_ps2.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/serio_raw.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/sparse-keymap.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-blinkm.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lm3530.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lp3944.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-ss4200.ko 2024-02-26T10:22:50.196 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-26T10:22:50.199 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache/bcache.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-bio-prison.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache-smq.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-crypt.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-delay.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-flakey.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-log-userspace.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-multipath.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-queue-length.ko 2024-02-26T10:22:50.200 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-raid.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-round-robin.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-service-time.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-switch.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-thin-pool.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-verity.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid0.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid1.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid10.ko 2024-02-26T10:22:50.201 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid456.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptbase.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptctl.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptfc.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptlan.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptsas.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptscsih.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptspi.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_ich.ko 2024-02-26T10:22:50.202 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_sch.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/mfd-core.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/sm501.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/viperboard.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/vx855.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/wl1273-core.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds9802als.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds990x.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/bh1770glc.ko 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710 2024-02-26T10:22:50.203 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710/cb710.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/at24.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/max6875.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/enclosure.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/hpilo.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29003.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29020.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-26T10:22:50.204 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-me.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-txe.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_7xx1.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_core.ko 2024-02-26T10:22:50.205 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tsl2550.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_balloon.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding/bonding.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-26T10:22:50.206 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dummy.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/eql.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-26T10:22:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-26T10:22:50.208 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-26T10:22:50.209 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-26T10:22:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-26T10:22:50.211 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-26T10:22:50.212 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dnet.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ethoc.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/fealnx.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000 2024-02-26T10:22:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-26T10:22:50.214 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/jme.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-26T10:22:50.215 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-26T10:22:50.216 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-26T10:22:50.217 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-26T10:22:50.218 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-26T10:22:50.219 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet 2024-02-26T10:22:50.220 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/geneve.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ifb.ko 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan 2024-02-26T10:22:50.221 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvlan.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvtap.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mii.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/net_failover.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/netconsole.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/nlmon.ko 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs 2024-02-26T10:22:50.222 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/amd.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm87xx.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/broadcom.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/cicada.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/davicom.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/dp83640.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/et1011c.ko 2024-02-26T10:22:50.223 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/icplus.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/lxt.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/marvell.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/micrel.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/national.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/phylink.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/qsemi.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/realtek.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/smsc.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/ste10Xp.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/vitesse.ko 2024-02-26T10:22:50.224 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_async.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppoe.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppox.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pptp.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slhc.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slip.ko 2024-02-26T10:22:50.225 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/sungem_phy.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tap.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_random.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tun.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/veth.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/virtio_net.ko 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3 2024-02-26T10:22:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan/vxlan.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netfront.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb/ntb.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme-core.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme.ko 2024-02-26T10:22:50.227 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_cs.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_pc.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_serial.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/xen-pcifront.ko 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia 2024-02-26T10:22:50.228 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/i82092.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/pd6729.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acerhdf.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-26T10:22:50.229 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/hdaps.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel_ips.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-26T10:22:50.230 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-26T10:22:50.231 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps_parport.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/pps_core.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp_kvm.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-26T10:22:50.232 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-em3027.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-26T10:22:50.233 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-max6900.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-26T10:22:50.234 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-x1205.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-9xxx.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-sas.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-xxxx.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/BusLogic.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/a100u2w.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/advansys.ko 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx 2024-02-26T10:22:50.235 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/am53c974.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/atp870u.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa/bfa.ko 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc 2024-02-26T10:22:50.236 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ch.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dc395x.ko 2024-02-26T10:22:50.237 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dmx3191d.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esp_scsi.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-26T10:22:50.238 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic/fnic.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hpsa.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hptiop.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hv_storvsc.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/initio.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ipr.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ips.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci/isci.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-26T10:22:50.239 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc/libfc.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas/libsas.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-26T10:22:50.240 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvumi.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pmcraid.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla1280.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx 2024-02-26T10:22:50.241 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/raid_class.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_debug.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ses.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/st.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/stex.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-26T10:22:50.242 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/virtio_scsi.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/wd719x.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb/ssb.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_file.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_iblock.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_mod.ko 2024-02-26T10:22:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_pscsi.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_user.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-26T10:22:50.244 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-26T10:22:50.245 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_gsm.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_hdlc.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/nozomi.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/arc_uart.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs 2024-02-26T10:22:50.246 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_aec.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_cif.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_pci_generic.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_sercos3.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-acm.ko 2024-02-26T10:22:50.247 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usblp.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usbtmc.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/mdc800.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/microtek.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/adutux.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/appledisplay.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi26.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi62.ko 2024-02-26T10:22:50.248 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ezusb.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/idmouse.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/iowarrior.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ldusb.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/legousbtower.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usb3503.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usblcd.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/uss720.ko 2024-02-26T10:22:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/yurex.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy/phy-generic.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/aircable.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ark3116.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ch341.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cp210x.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cyberjack.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/empeg.ko 2024-02-26T10:22:50.250 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_ti.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipaq.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipw.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ir-usb.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-26T10:22:50.251 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mct_u232.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7720.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7840.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/navman.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/omninet.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/opticon.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/option.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/oti6858.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/pl2303.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcaux.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcserial.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/quatech2.ko 2024-02-26T10:22:50.252 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/safe_serial.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/sierra.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ssu100.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/symbolserial.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_debug.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/visor.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/whiteheat.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage 2024-02-26T10:22:50.253 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/uas.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-karma.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-26T10:22:50.254 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/usb-storage.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-26T10:22:50.255 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_net.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_scsi.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/apple_bl.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/lcd.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio.ko 2024-02-26T10:22:50.256 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_balloon.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_mmio.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_ring.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2482.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2490.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-26T10:22:50.257 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/wire.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-26T10:22:50.258 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/hpwdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/i6300esb.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ib700wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ibmasr.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it87_wdt.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/machzwd.ko 2024-02-26T10:22:50.259 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/nv_tco.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/softdog.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/via_wdt.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/wdt_pci.ko 2024-02-26T10:22:50.260 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/xen_wdt.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-evtchn.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntalloc.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntdev.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-privcmd.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p 2024-02-26T10:22:50.261 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p/9p.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs/affs.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs/befs.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/binfmt_misc.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs/btrfs.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles/cachefiles.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph/ceph.ko 2024-02-26T10:22:50.262 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda/coda.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs/cramfs.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm/dlm.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs/f2fs.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/fat.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/msdos.ko 2024-02-26T10:22:50.263 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/vfat.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/cuse.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/fuse.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2/gfs2.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs/hfs.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus/hfsplus.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs/isofs.ko 2024-02-26T10:22:50.264 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs/jfs.ko 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd/lockd.ko 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix/minix.ko 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs/netfs.ko 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout 2024-02-26T10:22:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfs.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv3.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv4.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/grace.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/nfs_acl.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd/nfsd.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2/nilfs2.ko 2024-02-26T10:22:50.266 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-celtic.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-centeuro.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-croatian.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-cyrillic.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-gaelic.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-greek.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-iceland.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-inuit.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-roman.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-romanian.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-turkish.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1250.ko 2024-02-26T10:22:50.267 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1251.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1255.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp737.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp775.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp850.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp852.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp855.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp857.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp860.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp861.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp862.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp863.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp864.ko 2024-02-26T10:22:50.268 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp865.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp866.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp869.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp874.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp932.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp936.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp949.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp950.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_euc-jp.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-1.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-13.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-14.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-15.ko 2024-02-26T10:22:50.269 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-2.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-3.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-4.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-5.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-6.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-7.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-9.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-r.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-ru.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-u.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_utf8.ko 2024-02-26T10:22:50.270 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs/overlay.ko 2024-02-26T10:22:50.271 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore/ramoops.ko 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs/reiserfs.ko 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs/romfs.ko 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client/cifs.ko 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_arc4.ko 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_md4.ko 2024-02-26T10:22:50.272 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs/squashfs.ko 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv/sysv.ko 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf/udf.ko 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs/ufs.ko 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs/xfs.ko 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu/rcutorture.ko 2024-02-26T10:22:50.273 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/torture.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/asn1_encoder.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc-itu-t.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc8.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libarc4.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libdes.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/libcrc32c.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lru_cache.ko 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4 2024-02-26T10:22:50.274 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4_compress.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4hc_compress.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math/cordic.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6/raid6_pq.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_bm.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_fsm.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_kmp.ko 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd 2024-02-26T10:22:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd/zstd_compress.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm/hwpoison-inject.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/6lowpan.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_dest.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_fragment.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_hop.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_mobility.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_routing.ko 2024-02-26T10:22:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_udp.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/garp.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/mrp.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/p8022.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/psnap.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/stp.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q/8021q.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_fd.ko 2024-02-26T10:22:50.277 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_rdma.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_virtio.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv/batman-adv.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/br_netfilter.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/bridge.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-26T10:22:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-26T10:22:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtables.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph/libceph.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/failover.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/pktgen.ko 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp 2024-02-26T10:22:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_diag.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv4.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv6.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver/dns_resolver.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/dsa_core.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_brcm.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_dsa.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_none.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_trailer.ko 2024-02-26T10:22:50.281 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ah4.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/esp4.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/fou.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/gre.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/inet_diag.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_gre.ko 2024-02-26T10:22:50.282 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_tunnel.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_vti.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipcomp.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipip.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-26T10:22:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-26T10:22:50.284 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_bic.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_dctcp.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_diag.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_highspeed.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_htcp.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_hybla.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_illinois.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_lp.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_scalable.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_vegas.ko 2024-02-26T10:22:50.285 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_veno.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_westwood.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_yeah.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tunnel4.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_diag.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_tunnel.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ah6.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/esp6.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/fou6.ko 2024-02-26T10:22:50.286 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_tunnel.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_vti.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ipcomp6.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-26T10:22:50.287 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-26T10:22:50.288 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/sit.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/tunnel6.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key/af_key.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_core.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-26T10:22:50.289 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_eth.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip6.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_netlink.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ppp.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc/llc.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154/mac802154.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls/mpls_gso.ko 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset 2024-02-26T10:22:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-26T10:22:50.291 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-26T10:22:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conncount.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-26T10:22:50.293 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_log_syslog.ko 2024-02-26T10:22:50.294 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_irc.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_sip.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_tables.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_log.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-26T10:22:50.295 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_chain_nat.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_compat.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_ct.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fib.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_hash.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_limit.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_log.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_masq.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_nat.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_numgen.ko 2024-02-26T10:22:50.296 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_queue.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_quota.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_redir.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject_inet.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_socket.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_tunnel.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_AUDIT.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CT.ko 2024-02-26T10:22:50.297 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_DSCP.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HL.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HMARK.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LED.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LOG.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NETMAP.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFLOG.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_RATEEST.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_SECMARK.ko 2024-02-26T10:22:50.298 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TEE.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TPROXY.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TRACE.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_addrtype.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_bpf.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cgroup.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cluster.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_comment.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connbytes.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlabel.ko 2024-02-26T10:22:50.299 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlimit.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connmark.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_conntrack.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cpu.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dccp.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_devgroup.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dscp.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ecn.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_esp.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hashlimit.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_helper.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hl.ko 2024-02-26T10:22:50.300 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipcomp.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_iprange.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipvs.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_l2tp.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_length.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_limit.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mac.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mark.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_multiport.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nat.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nfacct.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_osf.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_owner.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_physdev.ko 2024-02-26T10:22:50.301 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_pkttype.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_policy.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_quota.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_rateest.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_realm.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_recent.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_sctp.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_set.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_socket.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_state.ko 2024-02-26T10:22:50.302 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_statistic.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_string.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_tcpmss.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_time.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_u32.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink/netlink_diag.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh/nsh.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/openvswitch.ko 2024-02-26T10:22:50.303 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-geneve.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-gre.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-vxlan.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet/af_packet_diag.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_rdma.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_tcp.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_csum.ko 2024-02-26T10:22:50.304 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_gact.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_mirred.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_nat.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_pedit.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_police.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_simple.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_skbedit.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_vlan.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_basic.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_bpf.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_flow.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_fw.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_route.ko 2024-02-26T10:22:50.305 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_u32.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_cmp.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_ipset.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_meta.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_nbyte.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_text.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_u32.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_choke.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_codel.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_drr.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_fq.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_gred.ko 2024-02-26T10:22:50.306 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hfsc.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hhf.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_htb.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_ingress.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio_lib.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_multiq.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_netem.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_pie.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_plug.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_prio.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_qfq.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_red.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfb.ko 2024-02-26T10:22:50.307 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfq.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_tbf.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_teql.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp_diag.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/sunrpc.ko 2024-02-26T10:22:50.308 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/diag.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/tipc.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix/unix_diag.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-26T10:22:50.309 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys/trusted.ko 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib/irqbypass.ko 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias.bin 2024-02-26T10:22:50.310 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.alias.bin 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.bin 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.modinfo 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep.bin 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.devname 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.order 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.softdep 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols.bin 2024-02-26T10:22:50.311 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T10:22:50.315 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:50.316 DEBUG:teuthology.orchestra.run.smithi082:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-26T10:22:50.338 INFO:teuthology.orchestra.run.smithi082.stdout:postinstall scriptlet (using /bin/sh): 2024-02-26T10:22:50.338 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout: /usr/bin/kernel-install add 6.8.0-rc4-g7c0fbba6c508 /lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:for file in vmlinuz System.map config; do 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508"; then 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout: cp "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508" 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout: fi 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:done 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:new-kernel-pkg --remove 6.8.0-rc4-g7c0fbba6c508 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc4-g7c0fbba6c508.img 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-install remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:50.339 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-02-26T10:22:50.340 INFO:teuthology.orchestra.run.smithi082.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-26T10:22:50.340 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-26T10:22:50.340 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/update-bootloader --remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:50.340 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-02-26T10:22:50.340 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.06 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 68.el9 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Group : Unspecified 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 8264254 2024-02-26T10:22:50.413 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : builder@centos.org 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : CentOS 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-26T10:22:50.414 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-26T10:22:50.415 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2024-02-26T10:22:50.415 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:22:50.415 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2024-02-26T10:22:50.416 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-26T10:22:50.416 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:50.416 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-26T10:22:51.019 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-02-26T10:22:52.449 INFO:teuthology.orchestra.run.smithi082.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-26T10:22:52.488 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-02-26T10:22:52.490 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-02-26T10:22:52.505 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.e7g90nc9OD 2024-02-26T10:22:52.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.e7g90nc9OD 2024-02-26T10:22:52.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.e7g90nc9OD 2024-02-26T10:22:52.714 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.e7g90nc9OD is 6KB 2024-02-26T10:22:52.764 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.e7g90nc9OD 2024-02-26T10:22:52.779 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2024-02-26T10:22:52.843 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-26T10:22:52.843 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-26T10:22:52.844 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-26T10:22:52.844 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508.conf 2024-02-26T10:22:52.845 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:22:52.995 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-02-26T10:23:22.998 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-02-26T10:23:22.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-26T10:23:23.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:24:23.002 DEBUG:teuthology.orchestra.remote:timed out 2024-02-26T10:24:32.004 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-26T10:24:32.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:24:39.895 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-02-26T10:24:40.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-26T10:24:40.597 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-rc4-g7c0fbba6c508"... 2024-02-26T10:24:40.597 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-02-26T10:24:40.636 INFO:teuthology.orchestra.run.smithi082.stdout:6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:24:40.637 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc4-g7c0fbba6c508 vs 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:24:40.637 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-26T10:24:40.637 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-26T10:24:41.640 DEBUG:teuthology.parallel:result is None 2024-02-26T10:24:54.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26 10:24:54 URL:https://1.chacra.ceph.com/r/kernel/testing/7c0fbba6c508351b48a34af7de7b074eb29ad0f9/centos/9/flavors/default/x86_64/kernel-6.8.0_rc4_g7c0fbba6c508-1.x86_64.rpm [335918853/335918853] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-26T10:24:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:FINISHED --2024-02-26 10:24:54-- 2024-02-26T10:24:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:Total wall clock time: 4m 40s 2024-02-26T10:24:54.892 INFO:teuthology.orchestra.run.smithi067.stderr:Downloaded: 1 files, 320M in 4m 39s (1.15 MB/s) 2024-02-26T10:24:54.895 INFO:teuthology.task.kernel:Installing kernel 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 on client.0... 2024-02-26T10:24:54.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-26T10:24:56.930 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying... ######################################## 2024-02-26T10:24:57.012 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing... ######################################## 2024-02-26T10:24:57.096 INFO:teuthology.orchestra.run.smithi067.stdout:Updating / installing... 2024-02-26T10:25:04.932 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-6.8.0_rc4_g7c0fbba6c508-1 ######################################## 2024-02-26T10:25:18.217 INFO:teuthology.orchestra.run.smithi067.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-26T10:25:41.370 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-26T10:25:41.370 DEBUG:teuthology.orchestra.run.smithi067:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-26T10:25:41.447 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:25:41.448 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/System.map 2024-02-26T10:25:41.448 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/config 2024-02-26T10:25:41.448 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel 2024-02-26T10:25:41.448 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch 2024-02-26T10:25:41.448 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-26T10:25:41.449 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm 2024-02-26T10:25:41.450 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/arch/x86/kvm/kvm.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ansi_cprng.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/anubis.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/arc4.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_memcpy.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_pq.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_tx.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/async_xor.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/async_tx/raid6test.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authenc.ko 2024-02-26T10:25:41.451 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/authencesn.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blake2b_generic.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_common.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/blowfish_generic.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/camellia_generic.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast5_generic.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast6_generic.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cast_common.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/ccm.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cmac.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crc32_generic.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_engine.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/crypto_user.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/cts.ko 2024-02-26T10:25:41.452 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/des_generic.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/dh_generic.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/echainiv.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/essiv.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/fcrypt.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/khazad.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/lz4hc.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/md4.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/michael_mic.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcbc.ko 2024-02-26T10:25:41.453 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/pcrypt.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/rmd160.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/seed.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/serpent_generic.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tcrypt.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/tea.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_common.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/twofish_generic.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/vmac.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/wp512.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xcbc.ko 2024-02-26T10:25:41.454 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xor.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/crypto/xxhash_generic.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/acpi_pad.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/custom_method.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/ec_sys.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/platform_profile.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbs.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/sbshc.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/acpi/video.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/acard-ahci.ko 2024-02-26T10:25:41.455 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ahci_platform.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/ata_generic.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/libahci_platform.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_acpi.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ali.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_amd.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_artop.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atiixp.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_atp867x.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd640.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cmd64x.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_cypress.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_efar.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt366.ko 2024-02-26T10:25:41.456 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt37x.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it8213.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_it821x.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_jmicron.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_marvell.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_mpiix.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_netcell.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ninja32.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87410.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_ns87415.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_oldpiix.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_opti.ko 2024-02-26T10:25:41.457 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_optidma.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pcmcia.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_piccolo.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_rdc.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sch.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_serverworks.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sil680.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sis.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_sl82c105.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_triflex.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pata_via.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/pdc_adma.ko 2024-02-26T10:25:41.458 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_inic162x.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_mv.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_nv.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_promise.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_qstor.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sil24.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sis.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_svw.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_sx4.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_uli.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_via.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ata/sata_vsc.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/auxdisplay/ks0108.ko 2024-02-26T10:25:41.459 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/bcma/bcma.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/aoe/aoe.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/brd.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/drbd/drbd.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/floppy.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/loop.ko 2024-02-26T10:25:41.460 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/nbd.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/null_blk/null_blk.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/pktcdvd.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/rbd.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/virtio_blk.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/xen-blkfront.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/block/zram/zram.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hangcheck-timer.ko 2024-02-26T10:25:41.461 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/via-rng.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/lp.ko 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave 2024-02-26T10:25:41.462 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/mwave/mwave.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/ppdev.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tlclk.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/uv_mmtimer.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/char/virtio_console.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-26T10:25:41.463 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/ccp/ccp.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-aes.ko 2024-02-26T10:25:41.464 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/padlock-sha.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dca/dca.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/amd64_edac.ko 2024-02-26T10:25:41.465 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/e752x_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/edac_mce_amd.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3000_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i3200_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5100_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i5400_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7300_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i7core_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/i82975x_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/ie31200_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/sb_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/edac/x38_edac.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/edd.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-26T10:25:41.466 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-ich.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-appleir.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-aureal.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-axff.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-dr.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elecom.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-elo.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-emsff.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gaff.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gt683r.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-gyration.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-26T10:25:41.467 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-holtekff.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-hyperv.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-icade.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-keytouch.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-kye.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lcpower.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-led.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-lenovo.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-multitouch.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-ortek.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-penmount.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-petalynx.ko 2024-02-26T10:25:41.468 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-picolcd.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-pl.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-plantronics.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-primax.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-rmi.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-common.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-26T10:25:41.469 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-roccat.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-saitek.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-samsung.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sjoy.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sony.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-speedlink.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-steelseries.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-sunplus.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tivo.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-tmff.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-topseed.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-twinhan.ko 2024-02-26T10:25:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-uclogic.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-waltop.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-wiimote.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-xinmo.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zpff.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/hid-zydacron.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/uhid.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hid/wacom.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_balloon.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_utils.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hv/hv_vmbus.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru.ko 2024-02-26T10:25:41.471 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/abituguru3.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7414.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ad7418.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adc128d818.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1025.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1026.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1029.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm1031.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adm9240.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ads7828.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7410.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7411.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7462.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7470.ko 2024-02-26T10:25:41.472 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7475.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/adt7x10.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/amc6821.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/applesmc.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asb100.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asc7621.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/atxp1.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/coretemp.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/dme1737.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds1621.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ds620.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc1403.ko 2024-02-26T10:25:41.473 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/emc6w201.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71805f.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f71882fg.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/f75375s.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fam15h_power.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/fschmd.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g760a.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/g762.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl518sm.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/gl520sm.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5500_temp.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/i5k_amb.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmaem.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ibmpex.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina209.ko 2024-02-26T10:25:41.474 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ina2xx.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/it87.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k10temp.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/k8temp.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lineage-pem.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm63.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm73.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm75.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm77.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm78.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm80.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm83.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm85.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm87.ko 2024-02-26T10:25:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm90.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm92.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm93.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95234.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95241.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/lm95245.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc2945.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4151.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4215.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4222.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4245.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4260.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/ltc4261.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max16065.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1619.ko 2024-02-26T10:25:41.476 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max1668.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max197.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6639.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6650.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/max6697.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/mcp3021.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6683.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775-core.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct6775.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/nct7802.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87360.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pc87427.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pcf8591.ko 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus 2024-02-26T10:25:41.477 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/powr1220.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5627.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch5636.ko 2024-02-26T10:25:41.478 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht15.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sht21.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/shtc1.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/sis5595.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47b397.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m1.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/smsc47m192.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/thmc50.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp102.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp103.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp401.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/tmp421.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via-cputemp.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/via686a.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt1211.ko 2024-02-26T10:25:41.479 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/vt8231.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627ehf.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83627hf.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83781d.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83791d.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83792d.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83793.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83795.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l785ts.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/hwmon/w83l786ng.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses 2024-02-26T10:25:41.480 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-26T10:25:41.481 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-dev.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-mux.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-smbus.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/i2c/i2c-stub.ko 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband 2024-02-26T10:25:41.482 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_core.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-26T10:25:41.483 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input 2024-02-26T10:25:41.484 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/gameport/gameport.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/joydev.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/appletouch.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/bcm5974.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/cyapatp.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/sermouse.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-26T10:25:41.485 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/altera_ps2.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/arc_ps2.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/serio/serio_raw.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/input/sparse-keymap.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-blinkm.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lm3530.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-lp3944.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/leds-ss4200.ko 2024-02-26T10:25:41.486 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/bcache/bcache.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-bio-prison.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache-smq.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-cache.ko 2024-02-26T10:25:41.487 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-crypt.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-delay.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-flakey.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-log-userspace.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-multipath.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-queue-length.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-raid.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-round-robin.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-service-time.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-switch.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-thin-pool.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/dm-verity.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-26T10:25:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid0.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid1.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid10.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/md/raid456.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptbase.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptctl.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptfc.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptlan.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptsas.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptscsih.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/message/fusion/mptspi.ko 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd 2024-02-26T10:25:41.489 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_ich.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/lpc_sch.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/mfd-core.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/sm501.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/viperboard.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/vx855.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/mfd/wl1273-core.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds9802als.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/apds990x.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/bh1770glc.ko 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710 2024-02-26T10:25:41.490 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/cb710/cb710.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/at24.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/eeprom/max6875.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/enclosure.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/hpilo.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29003.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/isl29020.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-me.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei-txe.ko 2024-02-26T10:25:41.491 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/mei/mei.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_7xx1.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tifm_core.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/tsl2550.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_balloon.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net 2024-02-26T10:25:41.492 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding 2024-02-26T10:25:41.493 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/bonding/bonding.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/dummy.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/eql.ko 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com 2024-02-26T10:25:41.495 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon 2024-02-26T10:25:41.496 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-26T10:25:41.497 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio 2024-02-26T10:25:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip 2024-02-26T10:25:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/dnet.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ethoc.ko 2024-02-26T10:25:41.500 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/fealnx.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-26T10:25:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/jme.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox 2024-02-26T10:25:41.502 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-26T10:25:41.503 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc 2024-02-26T10:25:41.504 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-26T10:25:41.505 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-26T10:25:41.506 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/geneve.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-26T10:25:41.507 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ifb.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvlan.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/macvtap.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/mii.ko 2024-02-26T10:25:41.508 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/net_failover.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/netconsole.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/nlmon.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/amd.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/bcm87xx.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/broadcom.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/cicada.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/davicom.ko 2024-02-26T10:25:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/dp83640.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/et1011c.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/icplus.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/lxt.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/marvell.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/micrel.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/national.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/phylink.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/qsemi.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/realtek.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/smsc.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/ste10Xp.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/phy/vitesse.ko 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp 2024-02-26T10:25:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_async.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppoe.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pppox.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/ppp/pptp.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slhc.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/slip/slip.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/sungem_phy.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tap.ko 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team 2024-02-26T10:25:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_random.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/tun.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/veth.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/virtio_net.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/vxlan/vxlan.ko 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback 2024-02-26T10:25:41.512 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/net/xen-netfront.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ntb/ntb.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme-core.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/nvme/host/nvme.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_cs.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_pc.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/parport/parport_serial.ko 2024-02-26T10:25:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pci/xen-pcifront.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/i82092.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/pd6729.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86 2024-02-26T10:25:41.514 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/acerhdf.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/hdaps.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-26T10:25:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/intel_ips.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/platform/x86/wmi.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps 2024-02-26T10:25:41.516 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/clients/pps_parport.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/pps/pps_core.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ptp/ptp_kvm.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-26T10:25:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-em3027.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-max6900.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-26T10:25:41.518 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/rtc/rtc-x1205.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-9xxx.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-sas.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/3w-xxxx.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/BusLogic.ko 2024-02-26T10:25:41.519 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/a100u2w.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/advansys.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/am53c974.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/atp870u.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bfa/bfa.ko 2024-02-26T10:25:41.520 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ch.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dc395x.ko 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler 2024-02-26T10:25:41.521 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/dmx3191d.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/esp_scsi.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/fnic/fnic.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hpsa.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hptiop.ko 2024-02-26T10:25:41.522 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/hv_storvsc.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/initio.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ipr.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ips.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/isci/isci.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libfc/libfc.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/libsas/libsas.ko 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc 2024-02-26T10:25:41.523 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/mvumi.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/pmcraid.ko 2024-02-26T10:25:41.524 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla1280.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/raid_class.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_debug.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/ses.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/st.ko 2024-02-26T10:25:41.525 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/stex.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/virtio_scsi.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/scsi/wd719x.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ssb/ssb.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_file.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_iblock.ko 2024-02-26T10:25:41.526 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_mod.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_pscsi.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/target_core_user.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-26T10:25:41.527 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_gsm.ko 2024-02-26T10:25:41.528 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/n_hdlc.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/nozomi.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/arc_uart.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio.ko 2024-02-26T10:25:41.529 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_aec.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_cif.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_pci_generic.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/uio/uio_sercos3.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-acm.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usblp.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/class/usbtmc.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/mdc800.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/image/microtek.ko 2024-02-26T10:25:41.530 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/adutux.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/appledisplay.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi26.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/emi62.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ezusb.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/idmouse.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/iowarrior.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/ldusb.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/legousbtower.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usb3503.ko 2024-02-26T10:25:41.531 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usblcd.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/uss720.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/misc/yurex.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/phy/phy-generic.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/aircable.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ark3116.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ch341.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cp210x.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cyberjack.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-26T10:25:41.532 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/empeg.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/io_ti.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipaq.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ipw.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ir-usb.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mct_u232.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7720.ko 2024-02-26T10:25:41.533 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/mos7840.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/navman.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/omninet.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/opticon.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/option.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/oti6858.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/pl2303.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcaux.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/qcserial.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/quatech2.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/safe_serial.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/sierra.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ssu100.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/symbolserial.ko 2024-02-26T10:25:41.534 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_debug.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/visor.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/whiteheat.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/uas.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-26T10:25:41.535 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-karma.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/storage/usb-storage.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci 2024-02-26T10:25:41.536 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_net.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/vhost/vhost_scsi.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/apple_bl.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/lcd.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-26T10:25:41.537 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_balloon.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_mmio.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/virtio/virtio_ring.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2482.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/masters/ds2490.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-26T10:25:41.538 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/w1/wire.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-26T10:25:41.539 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/hpwdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/i6300esb.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ib700wdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ibmasr.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/it87_wdt.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/machzwd.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/nv_tco.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-26T10:25:41.540 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/softdog.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/via_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/wdt_pci.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/watchdog/xen_wdt.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-evtchn.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntalloc.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-gntdev.ko 2024-02-26T10:25:41.541 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xen-privcmd.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/9p/9p.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/affs/affs.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/befs/befs.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/binfmt_misc.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/btrfs/btrfs.ko 2024-02-26T10:25:41.542 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cachefiles/cachefiles.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ceph/ceph.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/coda/coda.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/cramfs/cramfs.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/dlm/dlm.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/f2fs/f2fs.ko 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat 2024-02-26T10:25:41.543 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/fat.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/msdos.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fat/vfat.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/cuse.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/fuse/fuse.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/gfs2/gfs2.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfs/hfs.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/hfsplus/hfsplus.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/isofs/isofs.ko 2024-02-26T10:25:41.544 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/jfs/jfs.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/lockd/lockd.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/minix/minix.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/netfs/netfs.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-26T10:25:41.545 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfs.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv3.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs/nfsv4.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/grace.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfs_common/nfs_acl.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nfsd/nfsd.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nilfs2/nilfs2.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-celtic.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-centeuro.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-croatian.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-cyrillic.ko 2024-02-26T10:25:41.546 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-gaelic.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-greek.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-iceland.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-inuit.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-roman.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-romanian.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/mac-turkish.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1250.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1251.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp1255.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp737.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp775.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp850.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp852.ko 2024-02-26T10:25:41.547 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp855.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp857.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp860.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp861.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp862.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp863.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp864.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp865.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp866.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp869.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp874.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp932.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp936.ko 2024-02-26T10:25:41.548 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp949.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_cp950.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_euc-jp.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-1.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-13.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-14.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-15.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-2.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-3.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-4.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-5.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-6.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-7.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_iso8859-9.ko 2024-02-26T10:25:41.549 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-r.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-ru.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_koi8-u.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/nls/nls_utf8.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-26T10:25:41.550 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/overlayfs/overlay.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/pstore/ramoops.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/reiserfs/reiserfs.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/romfs/romfs.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/client/cifs.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_arc4.ko 2024-02-26T10:25:41.551 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/smb/common/cifs_md4.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/squashfs/squashfs.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/sysv/sysv.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/udf/udf.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/ufs/ufs.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/fs/xfs/xfs.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/rcu/rcutorture.ko 2024-02-26T10:25:41.552 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/torture.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/asn1_encoder.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc-itu-t.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crc8.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libarc4.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/crypto/libdes.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/libcrc32c.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lru_cache.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4_compress.ko 2024-02-26T10:25:41.553 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/lz4/lz4hc_compress.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/math/cordic.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/raid6/raid6_pq.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_bm.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_fsm.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/ts_kmp.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/lib/zstd/zstd_compress.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/mm/hwpoison-inject.ko 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net 2024-02-26T10:25:41.554 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/6lowpan.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_dest.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_fragment.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_hop.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_mobility.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_routing.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/6lowpan/nhc_udp.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/garp.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/mrp.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/p8022.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/psnap.ko 2024-02-26T10:25:41.555 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/802/stp.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/8021q/8021q.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_fd.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_rdma.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/9p/9pnet_virtio.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/batman-adv/batman-adv.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/br_netfilter.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/bridge.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-26T10:25:41.556 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-26T10:25:41.557 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/ebtables.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ceph/libceph.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/failover.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/core/pktgen.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp.ko 2024-02-26T10:25:41.558 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_diag.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv4.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dccp/dccp_ipv6.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dns_resolver/dns_resolver.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/dsa_core.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_brcm.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_dsa.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_none.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/dsa/tag_trailer.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154.ko 2024-02-26T10:25:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ah4.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/esp4.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/fou.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/gre.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/inet_diag.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_gre.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_tunnel.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ip_vti.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipcomp.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/ipip.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-26T10:25:41.560 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-26T10:25:41.561 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_bic.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_dctcp.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_diag.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_highspeed.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_htcp.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_hybla.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_illinois.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_lp.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_scalable.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_vegas.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_veno.ko 2024-02-26T10:25:41.562 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_westwood.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tcp_yeah.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/tunnel4.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_diag.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/udp_tunnel.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ah6.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/esp6.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/fou6.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_tunnel.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ip6_vti.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/ipcomp6.ko 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter 2024-02-26T10:25:41.563 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-26T10:25:41.564 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/sit.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/tunnel6.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/key/af_key.ko 2024-02-26T10:25:41.565 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_core.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_eth.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ip6.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_netlink.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/l2tp/l2tp_ppp.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/llc/llc.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mac802154/mac802154.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/mpls/mpls_gso.ko 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter 2024-02-26T10:25:41.566 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-26T10:25:41.567 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-26T10:25:41.568 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conncount.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-26T10:25:41.569 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_log_syslog.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_irc.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_sip.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-26T10:25:41.570 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nf_tables.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_log.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_chain_nat.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_compat.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_ct.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fib.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_hash.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_limit.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_log.ko 2024-02-26T10:25:41.571 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_masq.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_nat.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_numgen.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_queue.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_quota.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_redir.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_reject_inet.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_socket.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/nft_tunnel.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_AUDIT.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-26T10:25:41.572 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_CT.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_DSCP.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HL.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_HMARK.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LED.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_LOG.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NETMAP.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFLOG.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_RATEEST.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_SECMARK.ko 2024-02-26T10:25:41.573 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TEE.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TPROXY.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_TRACE.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_addrtype.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_bpf.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cgroup.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cluster.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_comment.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connbytes.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlabel.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connlimit.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_connmark.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_conntrack.ko 2024-02-26T10:25:41.574 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_cpu.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dccp.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_devgroup.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_dscp.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ecn.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_esp.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hashlimit.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_helper.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_hl.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipcomp.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_iprange.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_ipvs.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_l2tp.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_length.ko 2024-02-26T10:25:41.575 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_limit.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mac.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_mark.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_multiport.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nat.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_nfacct.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_osf.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_owner.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_physdev.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_pkttype.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_policy.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_quota.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_rateest.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_realm.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_recent.ko 2024-02-26T10:25:41.576 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_sctp.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_set.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_socket.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_state.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_statistic.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_string.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_tcpmss.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_time.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netfilter/xt_u32.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/netlink/netlink_diag.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/nsh/nsh.ko 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch 2024-02-26T10:25:41.577 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/openvswitch.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-geneve.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-gre.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/openvswitch/vport-vxlan.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/packet/af_packet_diag.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_rdma.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/rds/rds_tcp.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_csum.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_gact.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_mirred.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_nat.ko 2024-02-26T10:25:41.578 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_pedit.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_police.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_simple.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_skbedit.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/act_vlan.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_basic.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_bpf.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_flow.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_fw.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_route.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/cls_u32.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_cmp.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_ipset.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_meta.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_nbyte.ko 2024-02-26T10:25:41.579 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_text.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/em_u32.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_choke.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_codel.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_drr.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_fq.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_gred.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hfsc.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_hhf.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_htb.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_ingress.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_mqprio_lib.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_multiq.ko 2024-02-26T10:25:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_netem.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_pie.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_plug.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_prio.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_qfq.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_red.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfb.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_sfq.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_tbf.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sched/sch_teql.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sctp/sctp_diag.ko 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss 2024-02-26T10:25:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/sunrpc.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/diag.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/tipc/tipc.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/unix/unix_diag.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-26T10:25:41.582 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/security/keys/trusted-keys/trusted.ko 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/kernel/virt/lib/irqbypass.ko 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.alias.bin 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.alias.bin 2024-02-26T10:25:41.583 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.bin 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.builtin.modinfo 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.dep.bin 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.devname 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.order 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.softdep 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/modules.symbols.bin 2024-02-26T10:25:41.584 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T10:25:41.588 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:25:41.588 DEBUG:teuthology.orchestra.run.smithi067:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout:postinstall scriptlet (using /bin/sh): 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/bin/kernel-install add 6.8.0-rc4-g7c0fbba6c508 /lib/modules/6.8.0-rc4-g7c0fbba6c508/vmlinuz 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout:for file in vmlinuz System.map config; do 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508"; then 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout: cp "/lib/modules/6.8.0-rc4-g7c0fbba6c508/${file}" "/boot/${file}-6.8.0-rc4-g7c0fbba6c508" 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout: fi 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout:done 2024-02-26T10:25:41.613 INFO:teuthology.orchestra.run.smithi067.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:new-kernel-pkg --remove 6.8.0-rc4-g7c0fbba6c508 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc4-g7c0fbba6c508.img 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-install remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/update-bootloader --remove 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:25:41.614 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-02-26T10:25:41.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-02-26T10:25:41.699 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-26T10:25:41.700 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:25:41.701 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-02-26T10:25:41.701 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-26T10:25:41.702 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:25:41.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-26T10:25:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-02-26T10:25:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-26T10:25:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-02-26T10:25:43.843 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:25:43.858 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.M8CuAeFcNm 2024-02-26T10:25:43.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.M8CuAeFcNm 2024-02-26T10:25:43.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.M8CuAeFcNm 2024-02-26T10:25:44.081 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.M8CuAeFcNm is 6KB 2024-02-26T10:25:44.129 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.M8CuAeFcNm 2024-02-26T10:25:44.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-02-26T10:25:44.213 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-26T10:25:44.213 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-26T10:25:44.213 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-26T10:25:44.214 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508.conf 2024-02-26T10:25:44.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:25:44.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-02-26T10:26:14.374 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-26T10:26:14.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-26T10:26:14.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:26:49.020 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-02-26T10:26:58.021 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-26T10:26:58.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:27:16.412 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-02-26T10:27:28.413 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-26T10:27:28.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:27:31.084 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-02-26T10:27:31.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-26T10:27:31.768 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc4-g7c0fbba6c508"... 2024-02-26T10:27:31.769 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-02-26T10:27:31.808 INFO:teuthology.orchestra.run.smithi067.stdout:6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:27:31.808 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc4-g7c0fbba6c508 vs 6.8.0-rc4-g7c0fbba6c508 2024-02-26T10:27:31.809 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-26T10:27:31.809 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-26T10:27:32.810 DEBUG:teuthology.parallel:result is None 2024-02-26T10:27:32.810 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-26T10:27:32.821 INFO:teuthology.task.internal:Creating test directory... 2024-02-26T10:27:32.821 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T10:27:32.829 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-26T10:27:32.888 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-26T10:27:32.895 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-26T10:27:32.903 INFO:teuthology.task.internal:Creating archive directory... 2024-02-26T10:27:32.903 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T10:27:32.913 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-26T10:27:32.980 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-26T10:27:32.986 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-26T10:27:32.987 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T10:27:32.992 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-26T10:27:33.152 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:27:33.168 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:27:33.215 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:27:33.276 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-26T10:27:33.281 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-26T10:27:33.288 INFO:teuthology.task.internal:Configuring sudo... 2024-02-26T10:27:33.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T10:27:33.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-26T10:27:33.396 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-26T10:27:33.405 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-26T10:27:33.405 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T10:27:33.410 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-26T10:27:33.467 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:27:33.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:27:33.643 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:27:33.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:27:33.803 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:27:33.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T10:27:33.940 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:27:34.029 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-26T10:27:34.135 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:27:34.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-26T10:27:34.316 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:27:34.317 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-26T10:27:34.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-02-26T10:27:34.417 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-02-26T10:27:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:27:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:27:34.836 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-26T10:27:34.847 INFO:teuthology.task.internal:Starting timer... 2024-02-26T10:27:34.847 INFO:teuthology.run_tasks:Running task pcp... 2024-02-26T10:27:34.856 INFO:teuthology.run_tasks:Running task selinux... 2024-02-26T10:27:34.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-02-26T10:27:34.964 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-02-26T10:27:34.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-02-26T10:27:35.112 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-02-26T10:27:35.116 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-26T10:27:35.116 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-02-26T10:27:35.166 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-02-26T10:27:35.166 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-02-26T10:27:35.224 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-02-26T10:27:35.225 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi082.front.sepia.ceph.com': 'permissive'} 2024-02-26T10:27:35.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:27:35.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:27:35.302 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:27:35.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:27:35.377 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-26T10:27:35.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-02-26T10:27:35.514 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-02-26T10:27:35.604 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-26T10:27:35.615 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-26T10:27:35.635 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-26T10:27:35.637 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi082.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-26T10:34:06.710 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com')] 2024-02-26T10:34:06.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-26T10:34:06.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:34:06.865 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-02-26T10:34:07.022 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-26T10:34:07.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-26T10:34:07.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-26T10:34:07.168 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-02-26T10:34:07.321 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-02-26T10:34:07.322 INFO:teuthology.run_tasks:Running task clock... 2024-02-26T10:34:07.337 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-26T10:34:07.337 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T10:34:07.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:34:07.347 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-26T10:34:07.347 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:34:07.447 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T10:34:07.448 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-26T10:34:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T10:34:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-26T10:34:07.591 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-02-26T10:34:07.611 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-02-26T10:34:07.636 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-02-26T10:34:07.656 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-02-26T10:34:07.686 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T10:34:07.710 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-26T10:34:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T10:34:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-26T10:34:07.867 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:34:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:34:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-02-26T10:34:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.906 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:34:07.914 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:34:07.914 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-02-26T10:34:07.914 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.914 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.914 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.915 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:34:07.916 INFO:teuthology.run_tasks:Running task install... 2024-02-26T10:34:07.924 DEBUG:teuthology.task.install:project ceph 2024-02-26T10:34:07.925 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'}} 2024-02-26T10:34:07.925 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'} 2024-02-26T10:34:07.925 INFO:teuthology.task.install:Using flavor: default 2024-02-26T10:34:07.931 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-26T10:34:07.931 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-26T10:34:07.931 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'tag': None, 'wait_for_package': False} 2024-02-26T10:34:07.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:34:07.934 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'tag': None, 'wait_for_package': False} 2024-02-26T10:34:07.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:34:08.399 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/ 2024-02-26T10:34:08.399 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1686.g6085b91f 2024-02-26T10:34:08.933 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T10:34:08.934 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:34:08.934 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T10:34:09.033 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T10:34:09.033 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6085b91fefeb74bf4310344a87d81d3217ef7893/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T10:34:09.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T10:34:09.300 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T10:34:09.385 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-02-26T10:34:09.390 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-02-26T10:34:09.584 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/ 2024-02-26T10:34:09.584 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1686.g6085b91f 2024-02-26T10:34:09.754 INFO:teuthology.orchestra.run.smithi082.stdout:74 files removed 2024-02-26T10:34:09.793 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T10:34:10.454 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240226.064629/6085b91fefeb74bf4310344a87d81d3217ef7893/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-26T10:34:10.455 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:34:10.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-26T10:34:10.557 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-26T10:34:10.557 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6085b91fefeb74bf4310344a87d81d3217ef7893/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-26T10:34:10.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-26T10:34:10.844 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-26T10:34:10.938 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-02-26T10:34:10.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-02-26T10:34:11.315 INFO:teuthology.orchestra.run.smithi067.stdout:74 files removed 2024-02-26T10:34:11.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-26T10:34:13.436 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 52 kB/s | 82 kB 00:01 2024-02-26T10:34:14.560 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 19 kB/s | 82 kB 00:04 2024-02-26T10:34:16.468 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 3.4 kB/s | 10 kB 00:02 2024-02-26T10:34:18.602 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 2.5 kB/s | 10 kB 00:03 2024-02-26T10:34:19.886 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 581 B/s | 1.9 kB 00:03 2024-02-26T10:34:20.918 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 865 B/s | 1.9 kB 00:02 2024-02-26T10:34:21.031 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 7.6 MB/s | 8.1 MB 00:01 2024-02-26T10:34:21.613 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-02-26T10:34:23.340 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.5 MB 00:00 2024-02-26T10:34:23.639 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.5 MB 00:00 2024-02-26T10:34:24.577 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-02-26T10:34:24.893 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-02-26T10:34:27.127 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-02-26T10:34:29.025 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-02-26T10:34:34.652 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.1 MB 00:03 2024-02-26T10:34:36.703 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2024-02-26T10:34:41.470 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 744 kB/s | 6.1 MB 00:08 2024-02-26T10:34:43.479 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-02-26T10:34:45.522 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-26T10:34:51.656 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr:Modular dependency problems: 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 1: conflicting requests 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 2: conflicting requests 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.446 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 3: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 4: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 5: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 6: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 7: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 8: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 9: conflicting requests 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.447 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 10: conflicting requests 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 11: conflicting requests 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 12: conflicting requests 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 13: conflicting requests 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 14: conflicting requests 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 15: conflicting requests 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.448 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 16: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 17: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 18: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 19: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 20: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 21: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 22: conflicting requests 2024-02-26T10:34:54.449 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 23: conflicting requests 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 24: conflicting requests 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 25: conflicting requests 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 26: conflicting requests 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 27: conflicting requests 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 28: conflicting requests 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.450 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 29: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 30: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 31: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 32: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 33: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 34: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 35: conflicting requests 2024-02-26T10:34:54.451 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 36: conflicting requests 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 38: conflicting requests 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.452 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:34:54.453 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:34:54.454 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.455 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.455 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:34:54.455 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:34:55.042 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:34:55.044 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:34:55.065 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T10:34:55.067 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T10:34:55.068 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T10:34:55.070 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T10:34:55.072 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T10:34:55.076 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T10:34:55.077 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T10:34:55.179 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-02-26T10:34:55.203 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================= 2024-02-26T10:34:55.203 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-02-26T10:34:55.203 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================= 2024-02-26T10:34:55.203 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-02-26T10:34:55.203 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 6.6 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.3 M 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 888 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 145 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 1.5 M 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 138 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 1.8 M 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 7.4 M 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 49 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 9.8 M 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 47 M 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 264 k 2024-02-26T10:34:55.204 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 748 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 220 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 27 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 18 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 31 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 744 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 127 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T10:34:55.205 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 162 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 320 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 298 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 100 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 85 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 170 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T10:34:55.206 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.4 M 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T10:34:55.207 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 20 M 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 31 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 2.1 M 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 248 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 4.7 M 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 17 M 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 15 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 25 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T10:34:55.208 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 163 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 501 k 2024-02-26T10:34:55.209 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.1 M 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T10:34:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 45 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 131 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T10:34:55.211 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T10:34:55.212 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T10:34:55.213 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T10:34:55.214 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T10:34:55.215 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:34:55.216 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-02-26T10:34:55.216 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================= 2024-02-26T10:34:55.216 INFO:teuthology.orchestra.run.smithi067.stdout:Install 150 Packages 2024-02-26T10:34:55.216 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 16 Packages 2024-02-26T10:34:55.216 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:34:55.216 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 222 M 2024-02-26T10:34:55.217 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-02-26T10:34:57.408 INFO:teuthology.orchestra.run.smithi067.stdout:(1/166): ceph-19.0.0-1686.g6085b91f.el9.x86_64. 4.1 kB/s | 6.6 kB 00:01 2024-02-26T10:34:58.315 INFO:teuthology.orchestra.run.smithi067.stdout:(2/166): ceph-fuse-19.0.0-1686.g6085b91f.el9.x8 980 kB/s | 888 kB 00:00 2024-02-26T10:34:58.441 INFO:teuthology.orchestra.run.smithi067.stdout:(3/166): ceph-immutable-object-cache-19.0.0-168 1.1 MB/s | 145 kB 00:00 2024-02-26T10:34:59.533 INFO:teuthology.orchestra.run.smithi067.stdout:(4/166): ceph-mds-19.0.0-1686.g6085b91f.el9.x86 2.0 MB/s | 2.1 MB 00:01 2024-02-26T10:34:59.847 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 914 kB/s | 20 MB 00:22 2024-02-26T10:35:00.292 INFO:teuthology.orchestra.run.smithi067.stdout:(5/166): ceph-mgr-19.0.0-1686.g6085b91f.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-02-26T10:35:01.134 INFO:teuthology.orchestra.run.smithi067.stdout:(6/166): ceph-base-19.0.0-1686.g6085b91f.el9.x8 1.0 MB/s | 5.3 MB 00:05 2024-02-26T10:35:02.292 INFO:teuthology.orchestra.run.smithi067.stdout:(7/166): ceph-mon-19.0.0-1686.g6085b91f.el9.x86 2.3 MB/s | 4.7 MB 00:01 2024-02-26T10:35:05.591 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-02-26T10:35:06.675 INFO:teuthology.orchestra.run.smithi067.stdout:(8/166): ceph-radosgw-19.0.0-1686.g6085b91f.el9 2.2 MB/s | 9.8 MB 00:04 2024-02-26T10:35:06.843 INFO:teuthology.orchestra.run.smithi067.stdout:(9/166): ceph-selinux-19.0.0-1686.g6085b91f.el9 152 kB/s | 25 kB 00:00 2024-02-26T10:35:08.277 INFO:teuthology.orchestra.run.smithi082.stderr:Modular dependency problems: 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 1: conflicting requests 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240219154654:20125149.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 2: conflicting requests 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 3: conflicting requests 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 4: conflicting requests 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 5: conflicting requests 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 6: conflicting requests 2024-02-26T10:35:08.278 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 7: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 8: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 9: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 10: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 11: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 12: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 13: conflicting requests 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.279 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 14: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 15: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 16: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 17: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 18: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 19: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 20: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 21: conflicting requests 2024-02-26T10:35:08.280 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 22: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 23: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 24: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 25: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 26: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 27: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 28: conflicting requests 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.281 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 29: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 30: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 31: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 32: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 33: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 34: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 35: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 36: conflicting requests 2024-02-26T10:35:08.282 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 37: cannot install the best candidate for the job 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 38: conflicting requests 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.283 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 39: cannot install the best candidate for the job 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 40: cannot install the best candidate for the job 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 41: cannot install the best candidate for the job 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.284 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 42: cannot install the best candidate for the job 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 43: cannot install the best candidate for the job 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: Problem 44: cannot install the best candidate for the job 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.285 INFO:teuthology.orchestra.run.smithi082.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-26T10:35:08.832 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:35:08.834 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-26T10:35:08.854 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-26T10:35:08.855 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-26T10:35:08.856 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-26T10:35:08.858 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-26T10:35:08.860 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-26T10:35:08.863 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-26T10:35:08.864 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-26T10:35:08.957 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-02-26T10:35:08.998 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================= 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================= 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 6.6 k 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.3 M 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 888 k 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 145 k 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 1.5 M 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 138 k 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 1.8 M 2024-02-26T10:35:08.999 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 7.4 M 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 49 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 9.8 M 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 47 M 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 264 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 748 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 220 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 27 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 18 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 31 k 2024-02-26T10:35:09.000 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 744 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 127 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 162 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 320 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 298 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 100 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 85 k 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:35:09.001 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 170 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager x86_64 1:1.45.91-1.el9 baseos 2.3 M 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 baseos 1.8 M 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 baseos 42 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 baseos 247 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.4 M 2024-02-26T10:35:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 3.1 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 20 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 31 k 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 2.1 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 248 k 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 4.7 M 2024-02-26T10:35:09.003 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 17 M 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1686.g6085b91f.el9 ceph-noarch 15 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 25 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 163 k 2024-02-26T10:35:09.004 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 501 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 5.1 M 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-26T10:35:09.005 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 45 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-1686.g6085b91f.el9 ceph 131 k 2024-02-26T10:35:09.006 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-26T10:35:09.007 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-26T10:35:09.008 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-26T10:35:09.009 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-02-26T10:35:09.010 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-02-26T10:35:09.011 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================= 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout:Install 150 Packages 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 16 Packages 2024-02-26T10:35:09.012 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:35:09.013 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 222 M 2024-02-26T10:35:09.013 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-02-26T10:35:09.892 INFO:teuthology.orchestra.run.smithi082.stdout:(1/166): ceph-19.0.0-1686.g6085b91f.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-02-26T10:35:11.709 INFO:teuthology.orchestra.run.smithi082.stdout:(2/166): ceph-fuse-19.0.0-1686.g6085b91f.el9.x8 489 kB/s | 888 kB 00:01 2024-02-26T10:35:12.017 INFO:teuthology.orchestra.run.smithi082.stdout:(3/166): ceph-immutable-object-cache-19.0.0-168 472 kB/s | 145 kB 00:00 2024-02-26T10:35:13.743 INFO:teuthology.orchestra.run.smithi082.stdout:(4/166): ceph-base-19.0.0-1686.g6085b91f.el9.x8 1.2 MB/s | 5.3 MB 00:04 2024-02-26T10:35:15.026 INFO:teuthology.orchestra.run.smithi082.stdout:(5/166): ceph-mgr-19.0.0-1686.g6085b91f.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-02-26T10:35:16.492 INFO:teuthology.orchestra.run.smithi082.stdout:(6/166): ceph-mds-19.0.0-1686.g6085b91f.el9.x86 487 kB/s | 2.1 MB 00:04 2024-02-26T10:35:16.534 INFO:teuthology.orchestra.run.smithi067.stdout:(10/166): ceph-osd-19.0.0-1686.g6085b91f.el9.x8 1.1 MB/s | 17 MB 00:15 2024-02-26T10:35:16.809 INFO:teuthology.orchestra.run.smithi067.stdout:(11/166): cephfs-mirror-19.0.0-1686.g6085b91f.e 803 kB/s | 220 kB 00:00 2024-02-26T10:35:16.910 INFO:teuthology.orchestra.run.smithi067.stdout:(12/166): cephfs-shell-19.0.0-1686.g6085b91f.el 272 kB/s | 27 kB 00:00 2024-02-26T10:35:17.010 INFO:teuthology.orchestra.run.smithi067.stdout:(13/166): libcephfs-devel-19.0.0-1686.g6085b91f 313 kB/s | 31 kB 00:00 2024-02-26T10:35:17.710 INFO:teuthology.orchestra.run.smithi067.stdout:(14/166): libcephfs2-19.0.0-1686.g6085b91f.el9. 1.0 MB/s | 744 kB 00:00 2024-02-26T10:35:17.886 INFO:teuthology.orchestra.run.smithi067.stdout:(15/166): libcephsqlite-19.0.0-1686.g6085b91f.e 936 kB/s | 163 kB 00:00 2024-02-26T10:35:18.069 INFO:teuthology.orchestra.run.smithi067.stdout:(16/166): librados-devel-19.0.0-1686.g6085b91f. 694 kB/s | 127 kB 00:00 2024-02-26T10:35:18.826 INFO:teuthology.orchestra.run.smithi082.stdout:(7/166): ceph-mon-19.0.0-1686.g6085b91f.el9.x86 1.2 MB/s | 4.7 MB 00:03 2024-02-26T10:35:18.994 INFO:teuthology.orchestra.run.smithi067.stdout:(17/166): libradosstriper1-19.0.0-1686.g6085b91 542 kB/s | 501 kB 00:00 2024-02-26T10:35:23.761 INFO:teuthology.orchestra.run.smithi067.stdout:(18/166): librgw2-19.0.0-1686.g6085b91f.el9.x86 1.1 MB/s | 5.1 MB 00:04 2024-02-26T10:35:24.950 INFO:teuthology.orchestra.run.smithi082.stdout:(8/166): ceph-radosgw-19.0.0-1686.g6085b91f.el9 1.6 MB/s | 9.8 MB 00:06 2024-02-26T10:35:25.058 INFO:teuthology.orchestra.run.smithi082.stdout:(9/166): ceph-selinux-19.0.0-1686.g6085b91f.el9 234 kB/s | 25 kB 00:00 2024-02-26T10:35:25.163 INFO:teuthology.orchestra.run.smithi067.stdout:(19/166): python3-ceph-argparse-19.0.0-1686.g60 32 kB/s | 45 kB 00:01 2024-02-26T10:35:25.436 INFO:teuthology.orchestra.run.smithi067.stdout:(20/166): python3-ceph-common-19.0.0-1686.g6085 479 kB/s | 131 kB 00:00 2024-02-26T10:35:25.920 INFO:teuthology.orchestra.run.smithi067.stdout:(21/166): python3-cephfs-19.0.0-1686.g6085b91f. 336 kB/s | 162 kB 00:00 2024-02-26T10:35:26.204 INFO:teuthology.orchestra.run.smithi067.stdout:(22/166): python3-rados-19.0.0-1686.g6085b91f.e 1.1 MB/s | 320 kB 00:00 2024-02-26T10:35:28.988 INFO:teuthology.orchestra.run.smithi067.stdout:(23/166): python3-rbd-19.0.0-1686.g6085b91f.el9 107 kB/s | 298 kB 00:02 2024-02-26T10:35:29.087 INFO:teuthology.orchestra.run.smithi067.stdout:(24/166): python3-rgw-19.0.0-1686.g6085b91f.el9 1.0 MB/s | 100 kB 00:00 2024-02-26T10:35:29.358 INFO:teuthology.orchestra.run.smithi082.stdout:(10/166): ceph-common-19.0.0-1686.g6085b91f.el9 1.0 MB/s | 20 MB 00:20 2024-02-26T10:35:29.579 INFO:teuthology.orchestra.run.smithi067.stdout:(25/166): rbd-fuse-19.0.0-1686.g6085b91f.el9.x8 174 kB/s | 85 kB 00:00 2024-02-26T10:35:29.841 INFO:teuthology.orchestra.run.smithi082.stdout:(11/166): cephfs-mirror-19.0.0-1686.g6085b91f.e 456 kB/s | 220 kB 00:00 2024-02-26T10:35:30.250 INFO:teuthology.orchestra.run.smithi082.stdout:(12/166): cephfs-shell-19.0.0-1686.g6085b91f.el 66 kB/s | 27 kB 00:00 2024-02-26T10:35:30.666 INFO:teuthology.orchestra.run.smithi082.stdout:(13/166): libcephfs-devel-19.0.0-1686.g6085b91f 75 kB/s | 31 kB 00:00 2024-02-26T10:35:31.546 INFO:teuthology.orchestra.run.smithi067.stdout:(26/166): rbd-mirror-19.0.0-1686.g6085b91f.el9. 1.6 MB/s | 3.1 MB 00:01 2024-02-26T10:35:31.738 INFO:teuthology.orchestra.run.smithi067.stdout:(27/166): rbd-nbd-19.0.0-1686.g6085b91f.el9.x86 893 kB/s | 170 kB 00:00 2024-02-26T10:35:31.830 INFO:teuthology.orchestra.run.smithi067.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1686.g 345 kB/s | 31 kB 00:00 2024-02-26T10:35:31.988 INFO:teuthology.orchestra.run.smithi067.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1686.g6085b91 875 kB/s | 138 kB 00:00 2024-02-26T10:35:33.649 INFO:teuthology.orchestra.run.smithi082.stdout:(14/166): libcephfs2-19.0.0-1686.g6085b91f.el9. 250 kB/s | 744 kB 00:02 2024-02-26T10:35:34.208 INFO:teuthology.orchestra.run.smithi082.stdout:(15/166): libcephsqlite-19.0.0-1686.g6085b91f.e 293 kB/s | 163 kB 00:00 2024-02-26T10:35:34.357 INFO:teuthology.orchestra.run.smithi082.stdout:(16/166): librados-devel-19.0.0-1686.g6085b91f. 855 kB/s | 127 kB 00:00 2024-02-26T10:35:34.834 INFO:teuthology.orchestra.run.smithi082.stdout:(17/166): libradosstriper1-19.0.0-1686.g6085b91 1.0 MB/s | 501 kB 00:00 2024-02-26T10:35:34.855 INFO:teuthology.orchestra.run.smithi067.stdout:(30/166): ceph-mgr-dashboard-19.0.0-1686.g6085b 636 kB/s | 1.8 MB 00:02 2024-02-26T10:35:35.759 INFO:teuthology.orchestra.run.smithi082.stdout:(18/166): ceph-osd-19.0.0-1686.g6085b91f.el9.x8 893 kB/s | 17 MB 00:19 2024-02-26T10:35:35.892 INFO:teuthology.orchestra.run.smithi082.stdout:(19/166): python3-ceph-argparse-19.0.0-1686.g60 341 kB/s | 45 kB 00:00 2024-02-26T10:35:36.177 INFO:teuthology.orchestra.run.smithi082.stdout:(20/166): python3-ceph-common-19.0.0-1686.g6085 464 kB/s | 131 kB 00:00 2024-02-26T10:35:36.401 INFO:teuthology.orchestra.run.smithi082.stdout:(21/166): python3-cephfs-19.0.0-1686.g6085b91f. 723 kB/s | 162 kB 00:00 2024-02-26T10:35:36.735 INFO:teuthology.orchestra.run.smithi082.stdout:(22/166): python3-rados-19.0.0-1686.g6085b91f.e 962 kB/s | 320 kB 00:00 2024-02-26T10:35:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:(23/166): python3-rbd-19.0.0-1686.g6085b91f.el9 1.2 MB/s | 298 kB 00:00 2024-02-26T10:35:37.102 INFO:teuthology.orchestra.run.smithi082.stdout:(24/166): librgw2-19.0.0-1686.g6085b91f.el9.x86 2.3 MB/s | 5.1 MB 00:02 2024-02-26T10:35:37.127 INFO:teuthology.orchestra.run.smithi082.stdout:(25/166): python3-rgw-19.0.0-1686.g6085b91f.el9 666 kB/s | 100 kB 00:00 2024-02-26T10:35:37.202 INFO:teuthology.orchestra.run.smithi082.stdout:(26/166): rbd-fuse-19.0.0-1686.g6085b91f.el9.x8 859 kB/s | 85 kB 00:00 2024-02-26T10:35:37.348 INFO:teuthology.orchestra.run.smithi082.stdout:(27/166): rbd-nbd-19.0.0-1686.g6085b91f.el9.x86 1.2 MB/s | 170 kB 00:00 2024-02-26T10:35:37.778 INFO:teuthology.orchestra.run.smithi082.stdout:(28/166): ceph-grafana-dashboards-19.0.0-1686.g 73 kB/s | 31 kB 00:00 2024-02-26T10:35:37.970 INFO:teuthology.orchestra.run.smithi082.stdout:(29/166): ceph-mgr-cephadm-19.0.0-1686.g6085b91 723 kB/s | 138 kB 00:00 2024-02-26T10:35:38.830 INFO:teuthology.orchestra.run.smithi082.stdout:(30/166): ceph-mgr-dashboard-19.0.0-1686.g6085b 2.1 MB/s | 1.8 MB 00:00 2024-02-26T10:35:39.645 INFO:teuthology.orchestra.run.smithi082.stdout:(31/166): rbd-mirror-19.0.0-1686.g6085b91f.el9. 1.2 MB/s | 3.1 MB 00:02 2024-02-26T10:35:39.978 INFO:teuthology.orchestra.run.smithi082.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1686.g60 746 kB/s | 248 kB 00:00 2024-02-26T10:35:40.096 INFO:teuthology.orchestra.run.smithi082.stdout:(33/166): ceph-mgr-rook-19.0.0-1686.g6085b91f.e 425 kB/s | 49 kB 00:00 2024-02-26T10:35:40.229 INFO:teuthology.orchestra.run.smithi082.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1686.g6 116 kB/s | 15 kB 00:00 2024-02-26T10:35:40.579 INFO:teuthology.orchestra.run.smithi082.stdout:(35/166): ceph-volume-19.0.0-1686.g6085b91f.el9 754 kB/s | 264 kB 00:00 2024-02-26T10:35:41.446 INFO:teuthology.orchestra.run.smithi082.stdout:(36/166): cephadm-19.0.0-1686.g6085b91f.el9.noa 864 kB/s | 748 kB 00:00 2024-02-26T10:35:41.571 INFO:teuthology.orchestra.run.smithi082.stdout:(37/166): cephfs-top-19.0.0-1686.g6085b91f.el9. 147 kB/s | 18 kB 00:00 2024-02-26T10:35:41.763 INFO:teuthology.orchestra.run.smithi082.stdout:(38/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 92 kB/s | 18 kB 00:00 2024-02-26T10:35:41.955 INFO:teuthology.orchestra.run.smithi082.stdout:(39/166): fuse-2.9.9-15.el9.x86_64.rpm 421 kB/s | 80 kB 00:00 2024-02-26T10:35:42.272 INFO:teuthology.orchestra.run.smithi082.stdout:(40/166): jq-1.6-16.el9.x86_64.rpm 592 kB/s | 187 kB 00:00 2024-02-26T10:35:42.413 INFO:teuthology.orchestra.run.smithi067.stdout:(31/166): ceph-mgr-diskprediction-local-19.0.0- 1.0 MB/s | 7.4 MB 00:07 2024-02-26T10:35:42.664 INFO:teuthology.orchestra.run.smithi067.stdout:(32/166): ceph-mgr-modules-core-19.0.0-1686.g60 995 kB/s | 248 kB 00:00 2024-02-26T10:35:42.780 INFO:teuthology.orchestra.run.smithi067.stdout:(33/166): ceph-mgr-rook-19.0.0-1686.g6085b91f.e 425 kB/s | 49 kB 00:00 2024-02-26T10:35:42.822 INFO:teuthology.orchestra.run.smithi082.stdout:(41/166): ceph-mgr-diskprediction-local-19.0.0- 1.9 MB/s | 7.4 MB 00:03 2024-02-26T10:35:42.889 INFO:teuthology.orchestra.run.smithi067.stdout:(34/166): ceph-prometheus-alerts-19.0.0-1686.g6 142 kB/s | 15 kB 00:00 2024-02-26T10:35:43.148 INFO:teuthology.orchestra.run.smithi082.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 602 kB/s | 195 kB 00:00 2024-02-26T10:35:43.175 INFO:teuthology.orchestra.run.smithi067.stdout:(35/166): ceph-volume-19.0.0-1686.g6085b91f.el9 933 kB/s | 264 kB 00:00 2024-02-26T10:35:43.481 INFO:teuthology.orchestra.run.smithi082.stdout:(43/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 656 kB/s | 218 kB 00:00 2024-02-26T10:35:43.640 INFO:teuthology.orchestra.run.smithi082.stdout:(44/166): libgfortran-11.4.1-3.el9.x86_64.rpm 587 kB/s | 803 kB 00:01 2024-02-26T10:35:43.823 INFO:teuthology.orchestra.run.smithi067.stdout:(36/166): cephadm-19.0.0-1686.g6085b91f.el9.noa 1.1 MB/s | 748 kB 00:00 2024-02-26T10:35:43.873 INFO:teuthology.orchestra.run.smithi082.stdout:(45/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 647 kB/s | 253 kB 00:00 2024-02-26T10:35:43.915 INFO:teuthology.orchestra.run.smithi067.stdout:(37/166): cephfs-top-19.0.0-1686.g6085b91f.el9. 201 kB/s | 18 kB 00:00 2024-02-26T10:35:44.057 INFO:teuthology.orchestra.run.smithi067.stdout:(38/166): libaio-devel-0.3.112-1.el8.x86_64.rpm 125 kB/s | 18 kB 00:00 2024-02-26T10:35:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 535 kB/s | 106 kB 00:00 2024-02-26T10:35:44.175 INFO:teuthology.orchestra.run.smithi067.stdout:(39/166): fuse-2.9.9-15.el9.x86_64.rpm 693 kB/s | 80 kB 00:00 2024-02-26T10:35:44.241 INFO:teuthology.orchestra.run.smithi067.stdout:(40/166): jq-1.6-16.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-02-26T10:35:44.299 INFO:teuthology.orchestra.run.smithi082.stdout:(47/166): python3-pycparser-2.20-6.el9.noarch.r 603 kB/s | 135 kB 00:00 2024-02-26T10:35:44.341 INFO:teuthology.orchestra.run.smithi067.stdout:(41/166): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-02-26T10:35:44.408 INFO:teuthology.orchestra.run.smithi067.stdout:(42/166): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-02-26T10:35:44.459 INFO:teuthology.orchestra.run.smithi067.stdout:(43/166): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 218 kB 00:00 2024-02-26T10:35:44.508 INFO:teuthology.orchestra.run.smithi067.stdout:(44/166): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-02-26T10:35:44.516 INFO:teuthology.orchestra.run.smithi082.stdout:(48/166): python3-requests-2.25.1-8.el9.noarch. 578 kB/s | 125 kB 00:00 2024-02-26T10:35:44.592 INFO:teuthology.orchestra.run.smithi067.stdout:(45/166): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-02-26T10:35:44.643 INFO:teuthology.orchestra.run.smithi067.stdout:(46/166): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-02-26T10:35:44.693 INFO:teuthology.orchestra.run.smithi067.stdout:(47/166): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-02-26T10:35:44.743 INFO:teuthology.orchestra.run.smithi067.stdout:(48/166): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-02-26T10:35:44.793 INFO:teuthology.orchestra.run.smithi067.stdout:(49/166): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-02-26T10:35:44.958 INFO:teuthology.orchestra.run.smithi082.stdout:(49/166): python3-urllib3-1.26.5-5.el9.noarch.r 487 kB/s | 215 kB 00:00 2024-02-26T10:35:45.133 INFO:teuthology.orchestra.run.smithi082.stdout:(50/166): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-02-26T10:35:45.216 INFO:teuthology.orchestra.run.smithi082.stdout:(51/166): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-02-26T10:35:45.267 INFO:teuthology.orchestra.run.smithi082.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-02-26T10:35:45.309 INFO:teuthology.orchestra.run.smithi082.stdout:(53/166): elfutils-libelf-devel-0.190-2.el9.x86 519 kB/s | 21 kB 00:00 2024-02-26T10:35:45.367 INFO:teuthology.orchestra.run.smithi082.stdout:(54/166): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-02-26T10:35:45.409 INFO:teuthology.orchestra.run.smithi082.stdout:(55/166): flexiblas-3.0.4-8.el9.x86_64.rpm 796 kB/s | 33 kB 00:00 2024-02-26T10:35:45.502 INFO:teuthology.orchestra.run.smithi067.stdout:(50/166): automake-1.16.2-8.el9.noarch.rpm 974 kB/s | 689 kB 00:00 2024-02-26T10:35:45.560 INFO:teuthology.orchestra.run.smithi082.stdout:(56/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-02-26T10:35:45.601 INFO:teuthology.orchestra.run.smithi082.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 445 kB/s | 18 kB 00:00 2024-02-26T10:35:45.652 INFO:teuthology.orchestra.run.smithi082.stdout:(58/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-02-26T10:35:45.694 INFO:teuthology.orchestra.run.smithi082.stdout:(59/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-02-26T10:35:45.752 INFO:teuthology.orchestra.run.smithi082.stdout:(60/166): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-02-26T10:35:45.811 INFO:teuthology.orchestra.run.smithi082.stdout:(61/166): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-02-26T10:35:45.861 INFO:teuthology.orchestra.run.smithi082.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 516 kB/s | 26 kB 00:00 2024-02-26T10:35:45.903 INFO:teuthology.orchestra.run.smithi082.stdout:(63/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-02-26T10:35:45.953 INFO:teuthology.orchestra.run.smithi067.stdout:(51/166): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-02-26T10:35:45.953 INFO:teuthology.orchestra.run.smithi082.stdout:(64/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.8 MB/s | 292 kB 00:00 2024-02-26T10:35:45.996 INFO:teuthology.orchestra.run.smithi082.stdout:(65/166): lua-5.4.4-4.el9.x86_64.rpm 4.5 MB/s | 188 kB 00:00 2024-02-26T10:35:46.037 INFO:teuthology.orchestra.run.smithi082.stdout:(66/166): openblas-0.3.21-2.el9.x86_64.rpm 860 kB/s | 35 kB 00:00 2024-02-26T10:35:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:(52/166): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-02-26T10:35:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:(53/166): elfutils-libelf-devel-0.190-2.el9.x86 373 kB/s | 21 kB 00:00 2024-02-26T10:35:46.188 INFO:teuthology.orchestra.run.smithi082.stdout:(67/166): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-02-26T10:35:46.255 INFO:teuthology.orchestra.run.smithi067.stdout:(54/166): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-02-26T10:35:46.337 INFO:teuthology.orchestra.run.smithi067.stdout:(55/166): flexiblas-3.0.4-8.el9.x86_64.rpm 396 kB/s | 33 kB 00:00 2024-02-26T10:35:46.348 INFO:teuthology.orchestra.run.smithi082.stdout:(68/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 26 MB/s | 4.1 MB 00:00 2024-02-26T10:35:46.438 INFO:teuthology.orchestra.run.smithi082.stdout:(69/166): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2024-02-26T10:35:46.513 INFO:teuthology.orchestra.run.smithi082.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 905 kB 00:00 2024-02-26T10:35:46.555 INFO:teuthology.orchestra.run.smithi082.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 135 kB 00:00 2024-02-26T10:35:46.731 INFO:teuthology.orchestra.run.smithi082.stdout:(72/166): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2024-02-26T10:35:46.885 INFO:teuthology.orchestra.run.smithi082.stdout:(73/166): protobuf-3.14.0-13.el9.x86_64.rpm 6.6 MB/s | 1.0 MB 00:00 2024-02-26T10:35:46.931 INFO:teuthology.orchestra.run.smithi082.stdout:(74/166): python3-attrs-20.3.0-7.el9.noarch.rpm 1.9 MB/s | 87 kB 00:00 2024-02-26T10:35:47.004 INFO:teuthology.orchestra.run.smithi067.stdout:(56/166): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.5 MB/s | 3.0 MB 00:00 2024-02-26T10:35:47.071 INFO:teuthology.orchestra.run.smithi067.stdout:(57/166): flexiblas-openblas-openmp-3.0.4-8.el9 277 kB/s | 18 kB 00:00 2024-02-26T10:35:47.107 INFO:teuthology.orchestra.run.smithi082.stdout:(75/166): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-02-26T10:35:47.129 INFO:teuthology.orchestra.run.smithi067.stdout:(58/166): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-02-26T10:35:47.157 INFO:teuthology.orchestra.run.smithi082.stdout:(76/166): python3-devel-3.9.17-2.el9.x86_64.rpm 4.9 MB/s | 245 kB 00:00 2024-02-26T10:35:47.205 INFO:teuthology.orchestra.run.smithi067.stdout:(59/166): librabbitmq-0.11.0-7.el9.x86_64.rpm 608 kB/s | 45 kB 00:00 2024-02-26T10:35:47.207 INFO:teuthology.orchestra.run.smithi082.stdout:(77/166): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-02-26T10:35:47.249 INFO:teuthology.orchestra.run.smithi082.stdout:(78/166): python3-jsonpatch-1.21-16.el9.noarch. 627 kB/s | 26 kB 00:00 2024-02-26T10:35:47.291 INFO:teuthology.orchestra.run.smithi082.stdout:(79/166): python3-jsonpointer-2.0-4.el9.noarch. 469 kB/s | 19 kB 00:00 2024-02-26T10:35:47.342 INFO:teuthology.orchestra.run.smithi082.stdout:(80/166): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-02-26T10:35:47.346 INFO:teuthology.orchestra.run.smithi067.stdout:(60/166): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-02-26T10:35:47.416 INFO:teuthology.orchestra.run.smithi082.stdout:(81/166): python3-cryptography-36.0.1-4.el9.x86 332 kB/s | 1.2 MB 00:03 2024-02-26T10:35:47.442 INFO:teuthology.orchestra.run.smithi082.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 349 kB/s | 35 kB 00:00 2024-02-26T10:35:47.455 INFO:teuthology.orchestra.run.smithi067.stdout:(61/166): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-02-26T10:35:47.492 INFO:teuthology.orchestra.run.smithi082.stdout:(83/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-02-26T10:35:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:(62/166): libuuid-devel-2.37.4-15.el9.x86_64.rp 386 kB/s | 26 kB 00:00 2024-02-26T10:35:47.543 INFO:teuthology.orchestra.run.smithi082.stdout:(84/166): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-02-26T10:35:47.581 INFO:teuthology.orchestra.run.smithi067.stdout:(63/166): libzstd-devel-1.5.1-2.el9.x86_64.rpm 821 kB/s | 47 kB 00:00 2024-02-26T10:35:47.584 INFO:teuthology.orchestra.run.smithi082.stdout:(85/166): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-02-26T10:35:47.626 INFO:teuthology.orchestra.run.smithi082.stdout:(86/166): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-02-26T10:35:47.648 INFO:teuthology.orchestra.run.smithi067.stdout:(64/166): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-02-26T10:35:47.677 INFO:teuthology.orchestra.run.smithi082.stdout:(87/166): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-02-26T10:35:47.715 INFO:teuthology.orchestra.run.smithi067.stdout:(65/166): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-02-26T10:35:47.727 INFO:teuthology.orchestra.run.smithi082.stdout:(88/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 159 kB 00:00 2024-02-26T10:35:47.798 INFO:teuthology.orchestra.run.smithi067.stdout:(66/166): openblas-0.3.21-2.el9.x86_64.rpm 426 kB/s | 35 kB 00:00 2024-02-26T10:35:47.852 INFO:teuthology.orchestra.run.smithi082.stdout:(89/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-02-26T10:35:47.911 INFO:teuthology.orchestra.run.smithi082.stdout:(90/166): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-02-26T10:35:47.936 INFO:teuthology.orchestra.run.smithi082.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 613 kB/s | 51 kB 00:00 2024-02-26T10:35:47.961 INFO:teuthology.orchestra.run.smithi082.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-02-26T10:35:48.003 INFO:teuthology.orchestra.run.smithi082.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-02-26T10:35:48.045 INFO:teuthology.orchestra.run.smithi082.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-02-26T10:35:48.095 INFO:teuthology.orchestra.run.smithi082.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 4.3 MB/s | 216 kB 00:00 2024-02-26T10:35:48.137 INFO:teuthology.orchestra.run.smithi082.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 854 kB/s | 35 kB 00:00 2024-02-26T10:35:48.179 INFO:teuthology.orchestra.run.smithi082.stdout:(97/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-02-26T10:35:48.229 INFO:teuthology.orchestra.run.smithi082.stdout:(98/166): socat-1.7.4.1-5.el9.x86_64.rpm 6.1 MB/s | 305 kB 00:00 2024-02-26T10:35:48.272 INFO:teuthology.orchestra.run.smithi082.stdout:(99/166): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-02-26T10:35:48.313 INFO:teuthology.orchestra.run.smithi082.stdout:(100/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-02-26T10:35:48.355 INFO:teuthology.orchestra.run.smithi082.stdout:(101/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-02-26T10:35:48.415 INFO:teuthology.orchestra.run.smithi082.stdout:(102/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-02-26T10:35:48.424 INFO:teuthology.orchestra.run.smithi067.stdout:(67/166): openblas-openmp-0.3.21-2.el9.x86_64.r 7.6 MB/s | 4.7 MB 00:00 2024-02-26T10:35:48.656 INFO:teuthology.orchestra.run.smithi082.stdout:(103/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-02-26T10:35:48.757 INFO:teuthology.orchestra.run.smithi067.stdout:(68/166): openssl-devel-3.0.7-25.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2024-02-26T10:35:48.856 INFO:teuthology.orchestra.run.smithi082.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2024-02-26T10:35:48.933 INFO:teuthology.orchestra.run.smithi067.stdout:(69/166): postgresql-13.11-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-02-26T10:35:49.041 INFO:teuthology.orchestra.run.smithi067.stdout:(70/166): postgresql-contrib-13.11-1.el9.x86_64 8.3 MB/s | 905 kB 00:00 2024-02-26T10:35:49.108 INFO:teuthology.orchestra.run.smithi067.stdout:(71/166): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 135 kB 00:00 2024-02-26T10:35:49.486 INFO:teuthology.orchestra.run.smithi067.stdout:(72/166): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-02-26T10:35:49.784 INFO:teuthology.orchestra.run.smithi067.stdout:(73/166): ceph-test-19.0.0-1686.g6085b91f.el9.x 1.1 MB/s | 47 MB 00:42 2024-02-26T10:35:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:(74/166): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-02-26T10:35:50.009 INFO:teuthology.orchestra.run.smithi067.stdout:(75/166): python3-attrs-20.3.0-7.el9.noarch.rpm 390 kB/s | 87 kB 00:00 2024-02-26T10:35:50.265 INFO:teuthology.orchestra.run.smithi082.stdout:(105/166): ceph-test-19.0.0-1686.g6085b91f.el9. 1.9 MB/s | 47 MB 00:25 2024-02-26T10:35:50.293 INFO:teuthology.orchestra.run.smithi067.stdout:(76/166): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-02-26T10:35:50.315 INFO:teuthology.orchestra.run.smithi082.stdout:(106/166): protobuf-compiler-3.14.0-13.el9.x86_ 521 kB/s | 863 kB 00:01 2024-02-26T10:35:50.326 INFO:teuthology.orchestra.run.smithi067.stdout:(77/166): python3-devel-3.9.17-2.el9.x86_64.rpm 777 kB/s | 245 kB 00:00 2024-02-26T10:35:50.360 INFO:teuthology.orchestra.run.smithi067.stdout:(78/166): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-02-26T10:35:50.394 INFO:teuthology.orchestra.run.smithi067.stdout:(79/166): python3-jsonpatch-1.21-16.el9.noarch. 389 kB/s | 26 kB 00:00 2024-02-26T10:35:50.427 INFO:teuthology.orchestra.run.smithi067.stdout:(80/166): python3-jsonpointer-2.0-4.el9.noarch. 290 kB/s | 19 kB 00:00 2024-02-26T10:35:50.603 INFO:teuthology.orchestra.run.smithi067.stdout:(81/166): ceph-common-19.0.0-1686.g6085b91f.el9 374 kB/s | 20 MB 00:54 2024-02-26T10:35:50.651 INFO:teuthology.orchestra.run.smithi067.stdout:(82/166): python3-markupsafe-1.1.1-12.el9.x86_6 156 kB/s | 35 kB 00:00 2024-02-26T10:35:50.687 INFO:teuthology.orchestra.run.smithi067.stdout:(83/166): python3-mako-1.1.4-6.el9.noarch.rpm 590 kB/s | 172 kB 00:00 2024-02-26T10:35:50.745 INFO:teuthology.orchestra.run.smithi067.stdout:(84/166): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.6 MB/s | 250 kB 00:00 2024-02-26T10:35:50.829 INFO:teuthology.orchestra.run.smithi067.stdout:(85/166): python3-packaging-20.9-5.el9.noarch.r 936 kB/s | 77 kB 00:00 2024-02-26T10:35:50.862 INFO:teuthology.orchestra.run.smithi067.stdout:(86/166): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-02-26T10:35:50.904 INFO:teuthology.orchestra.run.smithi067.stdout:(87/166): python3-prettytable-0.7.2-27.el9.noar 569 kB/s | 42 kB 00:00 2024-02-26T10:35:50.988 INFO:teuthology.orchestra.run.smithi067.stdout:(88/166): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-02-26T10:35:51.063 INFO:teuthology.orchestra.run.smithi067.stdout:(89/166): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-02-26T10:35:51.097 INFO:teuthology.orchestra.run.smithi067.stdout:(90/166): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-02-26T10:35:51.131 INFO:teuthology.orchestra.run.smithi067.stdout:(91/166): python3-pytz-2021.1-5.el9.noarch.rpm 766 kB/s | 51 kB 00:00 2024-02-26T10:35:51.156 INFO:teuthology.orchestra.run.smithi067.stdout:(92/166): python3-requests-oauthlib-1.3.0-12.el 924 kB/s | 54 kB 00:00 2024-02-26T10:35:51.231 INFO:teuthology.orchestra.run.smithi067.stdout:(93/166): python3-toml-0.10.2-6.el9.noarch.rpm 561 kB/s | 42 kB 00:00 2024-02-26T10:35:51.298 INFO:teuthology.orchestra.run.smithi067.stdout:(94/166): python3-wcwidth-0.2.5-8.el9.noarch.rp 671 kB/s | 44 kB 00:00 2024-02-26T10:35:51.390 INFO:teuthology.orchestra.run.smithi067.stdout:(95/166): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-02-26T10:35:51.457 INFO:teuthology.orchestra.run.smithi067.stdout:(96/166): qatlib-service-23.11.0-1.el9.x86_64.r 531 kB/s | 35 kB 00:00 2024-02-26T10:35:51.524 INFO:teuthology.orchestra.run.smithi067.stdout:(97/166): qatzip-libs-1.1.2-2.el9.x86_64.rpm 677 kB/s | 44 kB 00:00 2024-02-26T10:35:51.565 INFO:teuthology.orchestra.run.smithi082.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 16 kB/s | 19 kB 00:01 2024-02-26T10:35:51.607 INFO:teuthology.orchestra.run.smithi067.stdout:(98/166): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-02-26T10:35:51.733 INFO:teuthology.orchestra.run.smithi067.stdout:(99/166): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-02-26T10:35:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:(100/166): uuid-1.6.2-55.el9.x86_64.rpm 409 kB/s | 58 kB 00:00 2024-02-26T10:35:51.817 INFO:teuthology.orchestra.run.smithi067.stdout:(101/166): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-02-26T10:35:51.842 INFO:teuthology.orchestra.run.smithi067.stdout:(102/166): xmlstarlet-1.6.1-20.el9.x86_64.rpm 699 kB/s | 64 kB 00:00 2024-02-26T10:35:51.884 INFO:teuthology.orchestra.run.smithi067.stdout:(103/166): zlib-devel-1.2.11-40.el9.x86_64.rpm 684 kB/s | 45 kB 00:00 2024-02-26T10:35:52.609 INFO:teuthology.orchestra.run.smithi067.stdout:(104/166): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-02-26T10:35:53.010 INFO:teuthology.orchestra.run.smithi067.stdout:(105/166): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-02-26T10:35:53.076 INFO:teuthology.orchestra.run.smithi067.stdout:(106/166): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-02-26T10:35:53.127 INFO:teuthology.orchestra.run.smithi067.stdout:(107/166): grpc-data-1.46.7-10.el9.noarch.rpm 397 kB/s | 19 kB 00:00 2024-02-26T10:35:53.140 INFO:teuthology.orchestra.run.smithi082.stdout:(108/166): abseil-cpp-20211102.0-3.el9.x86_64.r 128 kB/s | 548 kB 00:04 2024-02-26T10:35:53.391 INFO:teuthology.orchestra.run.smithi082.stdout:(109/166): gperftools-libs-2.9.1-2.el9.x86_64.r 99 kB/s | 309 kB 00:03 2024-02-26T10:35:53.477 INFO:teuthology.orchestra.run.smithi067.stdout:(108/166): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-02-26T10:35:53.578 INFO:teuthology.orchestra.run.smithi067.stdout:(109/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-02-26T10:35:53.591 INFO:teuthology.orchestra.run.smithi082.stdout:(110/166): libarrow-doc-9.0.0-10.el9.noarch.rpm 58 kB/s | 26 kB 00:00 2024-02-26T10:35:53.619 INFO:teuthology.orchestra.run.smithi067.stdout:(110/166): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-02-26T10:35:53.653 INFO:teuthology.orchestra.run.smithi067.stdout:(111/166): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-02-26T10:35:53.695 INFO:teuthology.orchestra.run.smithi067.stdout:(112/166): luarocks-3.9.2-1.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-02-26T10:35:53.724 INFO:teuthology.orchestra.run.smithi082.stdout:(111/166): liboath-2.6.7-2.el9.x86_64.rpm 146 kB/s | 49 kB 00:00 2024-02-26T10:35:53.762 INFO:teuthology.orchestra.run.smithi067.stdout:(113/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-02-26T10:35:53.821 INFO:teuthology.orchestra.run.smithi067.stdout:(114/166): python3-asyncssh-2.13.2-4.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-02-26T10:35:53.855 INFO:teuthology.orchestra.run.smithi067.stdout:(115/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-02-26T10:35:53.896 INFO:teuthology.orchestra.run.smithi067.stdout:(116/166): python3-cachetools-4.2.4-1.el9.noarc 788 kB/s | 32 kB 00:00 2024-02-26T10:35:53.930 INFO:teuthology.orchestra.run.smithi067.stdout:(117/166): python3-certifi-2021.10.8-2.el9.noar 461 kB/s | 15 kB 00:00 2024-02-26T10:35:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:(118/166): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-02-26T10:35:54.022 INFO:teuthology.orchestra.run.smithi067.stdout:(119/166): python3-cherrypy-18.6.1-2.el9.noarch 7.2 MB/s | 358 kB 00:00 2024-02-26T10:35:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:(120/166): protobuf-compiler-3.14.0-13.el9.x86_ 395 kB/s | 863 kB 00:02 2024-02-26T10:35:54.131 INFO:teuthology.orchestra.run.smithi067.stdout:(121/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-02-26T10:35:54.174 INFO:teuthology.orchestra.run.smithi067.stdout:(122/166): python3-google-auth-2.28.0-1.el9.noa 4.9 MB/s | 203 kB 00:00 2024-02-26T10:35:54.203 INFO:teuthology.orchestra.run.smithi067.stdout:(123/166): python3-colorama-0.4.4-12.el9.noarch 274 kB/s | 35 kB 00:00 2024-02-26T10:35:54.295 INFO:teuthology.orchestra.run.smithi067.stdout:(124/166): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-02-26T10:35:54.320 INFO:teuthology.orchestra.run.smithi067.stdout:(125/166): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-02-26T10:35:54.346 INFO:teuthology.orchestra.run.smithi067.stdout:(126/166): python3-jaraco-8.2.1-3.el9.noarch.rp 215 kB/s | 11 kB 00:00 2024-02-26T10:35:54.372 INFO:teuthology.orchestra.run.smithi067.stdout:(127/166): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-02-26T10:35:54.375 INFO:teuthology.orchestra.run.smithi082.stdout:(112/166): libunwind-1.6.2-1.el9.x86_64.rpm 86 kB/s | 67 kB 00:00 2024-02-26T10:35:54.397 INFO:teuthology.orchestra.run.smithi067.stdout:(128/166): python3-jaraco-collections-3.0.0-8.e 469 kB/s | 23 kB 00:00 2024-02-26T10:35:54.422 INFO:teuthology.orchestra.run.smithi067.stdout:(129/166): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2024-02-26T10:35:54.448 INFO:teuthology.orchestra.run.smithi067.stdout:(130/166): python3-jaraco-text-3.2.0-6.el9.noar 399 kB/s | 20 kB 00:00 2024-02-26T10:35:54.498 INFO:teuthology.orchestra.run.smithi067.stdout:(131/166): python3-logutils-0.3.5-21.el9.noarch 943 kB/s | 46 kB 00:00 2024-02-26T10:35:54.540 INFO:teuthology.orchestra.run.smithi067.stdout:(132/166): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-02-26T10:35:54.565 INFO:teuthology.orchestra.run.smithi067.stdout:(133/166): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-02-26T10:35:54.590 INFO:teuthology.orchestra.run.smithi067.stdout:(134/166): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-02-26T10:35:54.632 INFO:teuthology.orchestra.run.smithi067.stdout:(135/166): python3-portend-3.1.0-2.el9.noarch.r 405 kB/s | 16 kB 00:00 2024-02-26T10:35:54.691 INFO:teuthology.orchestra.run.smithi067.stdout:(136/166): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-02-26T10:35:54.725 INFO:teuthology.orchestra.run.smithi067.stdout:(137/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 988 kB/s | 90 kB 00:00 2024-02-26T10:35:54.941 INFO:teuthology.orchestra.run.smithi067.stdout:(138/166): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.3 MB/s | 16 MB 00:03 2024-02-26T10:35:54.967 INFO:teuthology.orchestra.run.smithi067.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 103 kB/s | 28 kB 00:00 2024-02-26T10:35:54.992 INFO:teuthology.orchestra.run.smithi067.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-02-26T10:35:55.018 INFO:teuthology.orchestra.run.smithi067.stdout:(141/166): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-02-26T10:35:55.043 INFO:teuthology.orchestra.run.smithi067.stdout:(142/166): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-02-26T10:35:55.068 INFO:teuthology.orchestra.run.smithi067.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-02-26T10:35:55.094 INFO:teuthology.orchestra.run.smithi067.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-02-26T10:35:55.111 INFO:teuthology.orchestra.run.smithi067.stdout:(145/166): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-02-26T10:35:55.145 INFO:teuthology.orchestra.run.smithi067.stdout:(146/166): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-02-26T10:35:55.169 INFO:teuthology.orchestra.run.smithi067.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-02-26T10:35:55.203 INFO:teuthology.orchestra.run.smithi067.stdout:(148/166): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-02-26T10:35:55.270 INFO:teuthology.orchestra.run.smithi067.stdout:(149/166): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-02-26T10:35:55.345 INFO:teuthology.orchestra.run.smithi067.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-02-26T10:35:55.554 INFO:teuthology.orchestra.run.smithi067.stdout:(151/166): NetworkManager-1.45.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-02-26T10:35:55.617 INFO:teuthology.orchestra.run.smithi082.stdout:(113/166): luarocks-3.9.2-1.el9.noarch.rpm 80 kB/s | 151 kB 00:01 2024-02-26T10:35:55.654 INFO:teuthology.orchestra.run.smithi067.stdout:(152/166): NetworkManager-libnm-1.45.91-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-02-26T10:35:55.705 INFO:teuthology.orchestra.run.smithi067.stdout:(153/166): NetworkManager-team-1.45.91-1.el9.x8 865 kB/s | 42 kB 00:00 2024-02-26T10:35:55.755 INFO:teuthology.orchestra.run.smithi067.stdout:(154/166): NetworkManager-tui-1.45.91-1.el9.x86 4.9 MB/s | 247 kB 00:00 2024-02-26T10:35:55.797 INFO:teuthology.orchestra.run.smithi067.stdout:(155/166): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-02-26T10:35:55.847 INFO:teuthology.orchestra.run.smithi067.stdout:(156/166): elfutils-debuginfod-client-0.190-2.e 741 kB/s | 36 kB 00:00 2024-02-26T10:35:55.897 INFO:teuthology.orchestra.run.smithi067.stdout:(157/166): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-02-26T10:35:55.948 INFO:teuthology.orchestra.run.smithi067.stdout:(158/166): elfutils-libs-0.190-2.el9.x86_64.rpm 5.1 MB/s | 256 kB 00:00 2024-02-26T10:35:55.990 INFO:teuthology.orchestra.run.smithi067.stdout:(159/166): libacl-2.3.1-4.el9.x86_64.rpm 559 kB/s | 23 kB 00:00 2024-02-26T10:35:56.073 INFO:teuthology.orchestra.run.smithi067.stdout:(160/166): openssl-3.0.7-25.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-02-26T10:35:56.182 INFO:teuthology.orchestra.run.smithi067.stdout:(161/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 20 MB/s | 2.2 MB 00:00 2024-02-26T10:35:56.232 INFO:teuthology.orchestra.run.smithi067.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 6.5 MB/s | 323 kB 00:00 2024-02-26T10:35:56.308 INFO:teuthology.orchestra.run.smithi067.stdout:(163/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-02-26T10:35:56.450 INFO:teuthology.orchestra.run.smithi067.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 528 kB/s | 74 kB 00:00 2024-02-26T10:35:57.458 INFO:teuthology.orchestra.run.smithi067.stdout:(165/166): librbd1-19.0.0-1686.g6085b91f.el9.x8 1.4 MB/s | 3.1 MB 00:02 2024-02-26T10:35:57.743 INFO:teuthology.orchestra.run.smithi067.stdout:(166/166): librados2-19.0.0-1686.g6085b91f.el9. 1.3 MB/s | 3.4 MB 00:02 2024-02-26T10:35:57.748 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:35:57.749 INFO:teuthology.orchestra.run.smithi067.stdout:Total 3.5 MB/s | 222 MB 01:02 2024-02-26T10:35:58.433 INFO:teuthology.orchestra.run.smithi082.stdout:(114/166): python3-asyncssh-2.13.2-4.el9.noarch 195 kB/s | 548 kB 00:02 2024-02-26T10:35:58.884 INFO:teuthology.orchestra.run.smithi082.stdout:(115/166): python3-bcrypt-3.2.2-1.el9.x86_64.rp 97 kB/s | 43 kB 00:00 2024-02-26T10:35:59.134 INFO:teuthology.orchestra.run.smithi082.stdout:(116/166): parquet-libs-9.0.0-10.el9.x86_64.rpm 176 kB/s | 838 kB 00:04 2024-02-26T10:35:59.201 INFO:teuthology.orchestra.run.smithi082.stdout:(117/166): python3-cachetools-4.2.4-1.el9.noarc 102 kB/s | 32 kB 00:00 2024-02-26T10:35:59.343 INFO:teuthology.orchestra.run.smithi082.stdout:(118/166): python3-certifi-2021.10.8-2.el9.noar 72 kB/s | 15 kB 00:00 2024-02-26T10:35:59.452 INFO:teuthology.orchestra.run.smithi082.stdout:(119/166): python3-cheroot-8.6.0-4.el9.noarch.r 689 kB/s | 172 kB 00:00 2024-02-26T10:35:59.735 INFO:teuthology.orchestra.run.smithi082.stdout:(120/166): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-02-26T10:35:59.769 INFO:teuthology.orchestra.run.smithi082.stdout:(121/166): python3-cherrypy-18.6.1-2.el9.noarch 845 kB/s | 358 kB 00:00 2024-02-26T10:35:59.852 INFO:teuthology.orchestra.run.smithi082.stdout:(122/166): python3-colorama-0.4.4-12.el9.noarch 304 kB/s | 35 kB 00:00 2024-02-26T10:35:59.969 INFO:teuthology.orchestra.run.smithi082.stdout:(123/166): python3-google-auth-2.28.0-1.el9.noa 1.0 MB/s | 203 kB 00:00 2024-02-26T10:36:00.245 INFO:teuthology.orchestra.run.smithi082.stdout:(124/166): python3-grpcio-tools-1.46.7-10.el9.x 527 kB/s | 144 kB 00:00 2024-02-26T10:36:00.478 INFO:teuthology.orchestra.run.smithi082.stdout:(125/166): python3-jaraco-8.2.1-3.el9.noarch.rp 46 kB/s | 11 kB 00:00 2024-02-26T10:36:00.679 INFO:teuthology.orchestra.run.smithi082.stdout:(126/166): python3-jaraco-classes-3.2.1-5.el9.n 89 kB/s | 18 kB 00:00 2024-02-26T10:36:01.162 INFO:teuthology.orchestra.run.smithi082.stdout:(127/166): python3-jaraco-collections-3.0.0-8.e 48 kB/s | 23 kB 00:00 2024-02-26T10:36:01.333 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-02-26T10:36:01.463 INFO:teuthology.orchestra.run.smithi082.stdout:(128/166): python3-jaraco-functools-3.5.0-2.el9 65 kB/s | 19 kB 00:00 2024-02-26T10:36:01.711 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-02-26T10:36:01.711 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-02-26T10:36:01.863 INFO:teuthology.orchestra.run.smithi082.stdout:(129/166): libarrow-9.0.0-10.el9.x86_64.rpm 439 kB/s | 4.4 MB 00:10 2024-02-26T10:36:02.088 INFO:teuthology.orchestra.run.smithi082.stdout:(130/166): python3-jaraco-text-3.2.0-6.el9.noar 31 kB/s | 20 kB 00:00 2024-02-26T10:36:02.355 INFO:teuthology.orchestra.run.smithi082.stdout:(131/166): python3-grpcio-1.46.7-10.el9.x86_64. 834 kB/s | 2.0 MB 00:02 2024-02-26T10:36:02.372 INFO:teuthology.orchestra.run.smithi082.stdout:(132/166): python3-logutils-0.3.5-21.el9.noarch 164 kB/s | 46 kB 00:00 2024-02-26T10:36:02.914 INFO:teuthology.orchestra.run.smithi082.stdout:(133/166): python3-natsort-7.1.1-5.el9.noarch.r 107 kB/s | 58 kB 00:00 2024-02-26T10:36:02.989 INFO:teuthology.orchestra.run.smithi082.stdout:(134/166): python3-more-itertools-8.12.0-2.el9. 125 kB/s | 79 kB 00:00 2024-02-26T10:36:03.256 INFO:teuthology.orchestra.run.smithi082.stdout:(135/166): python3-portend-3.1.0-2.el9.noarch.r 62 kB/s | 16 kB 00:00 2024-02-26T10:36:03.656 INFO:teuthology.orchestra.run.smithi082.stdout:(136/166): python3-kubernetes-26.1.0-2.el9.noar 584 kB/s | 1.0 MB 00:01 2024-02-26T10:36:03.865 INFO:teuthology.orchestra.run.smithi082.stdout:(137/166): python3-pecan-1.4.2-3.el9.noarch.rpm 287 kB/s | 272 kB 00:00 2024-02-26T10:36:03.898 INFO:teuthology.orchestra.run.smithi082.stdout:(138/166): python3-pyOpenSSL-21.0.0-1.el9.noarc 140 kB/s | 90 kB 00:00 2024-02-26T10:36:03.948 INFO:teuthology.orchestra.run.smithi082.stdout:(139/166): python3-pyperclip-1.8.2-2.el9.noarch 97 kB/s | 28 kB 00:00 2024-02-26T10:36:04.032 INFO:teuthology.orchestra.run.smithi082.stdout:(140/166): python3-repoze-lru-0.7-16.el9.noarch 185 kB/s | 31 kB 00:00 2024-02-26T10:36:04.267 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-02-26T10:36:04.269 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-02-26T10:36:04.349 INFO:teuthology.orchestra.run.smithi082.stdout:(141/166): python3-routes-2.5.1-5.el9.noarch.rp 419 kB/s | 188 kB 00:00 2024-02-26T10:36:04.441 INFO:teuthology.orchestra.run.smithi082.stdout:(142/166): python3-rsa-4.9-2.el9.noarch.rpm 120 kB/s | 59 kB 00:00 2024-02-26T10:36:04.491 INFO:teuthology.orchestra.run.smithi082.stdout:(143/166): python3-tempora-5.0.0-2.el9.noarch.r 78 kB/s | 36 kB 00:00 2024-02-26T10:36:04.566 INFO:teuthology.orchestra.run.smithi082.stdout:(144/166): python3-typing-extensions-4.4.0-2.el 235 kB/s | 51 kB 00:00 2024-02-26T10:36:04.841 INFO:teuthology.orchestra.run.smithi082.stdout:(145/166): python3-webob-1.8.7-6.el9.noarch.rpm 574 kB/s | 230 kB 00:00 2024-02-26T10:36:04.934 INFO:teuthology.orchestra.run.smithi082.stdout:(146/166): python3-websocket-client-1.2.3-2.el9 203 kB/s | 90 kB 00:00 2024-02-26T10:36:05.176 INFO:teuthology.orchestra.run.smithi082.stdout:(147/166): python3-zc-lockfile-2.0-10.el9.noarc 60 kB/s | 20 kB 00:00 2024-02-26T10:36:05.351 INFO:teuthology.orchestra.run.smithi082.stdout:(148/166): python3-werkzeug-2.0.3-3.el9.1.noarc 545 kB/s | 427 kB 00:00 2024-02-26T10:36:05.376 INFO:teuthology.orchestra.run.smithi082.stdout:(149/166): re2-20211101-3.el9.x86_64.rpm 434 kB/s | 192 kB 00:00 2024-02-26T10:36:06.509 INFO:teuthology.orchestra.run.smithi082.stdout:(150/166): thrift-0.15.0-2.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-02-26T10:36:06.935 INFO:teuthology.orchestra.run.smithi082.stdout:(151/166): librbd1-19.0.0-1686.g6085b91f.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-02-26T10:36:10.017 INFO:teuthology.orchestra.run.smithi082.stdout:(152/166): librados2-19.0.0-1686.g6085b91f.el9. 743 kB/s | 3.4 MB 00:04 2024-02-26T10:36:10.151 INFO:teuthology.orchestra.run.smithi082.stdout:(153/166): NetworkManager-libnm-1.45.91-1.el9.x 577 kB/s | 1.8 MB 00:03 2024-02-26T10:36:10.176 INFO:teuthology.orchestra.run.smithi082.stdout:(154/166): NetworkManager-team-1.45.91-1.el9.x8 269 kB/s | 42 kB 00:00 2024-02-26T10:36:10.293 INFO:teuthology.orchestra.run.smithi082.stdout:(155/166): NetworkManager-1.45.91-1.el9.x86_64. 615 kB/s | 2.3 MB 00:03 2024-02-26T10:36:10.318 INFO:teuthology.orchestra.run.smithi082.stdout:(156/166): acl-2.3.1-4.el9.x86_64.rpm 505 kB/s | 71 kB 00:00 2024-02-26T10:36:10.394 INFO:teuthology.orchestra.run.smithi082.stdout:(157/166): elfutils-debuginfod-client-0.190-2.e 364 kB/s | 36 kB 00:00 2024-02-26T10:36:10.636 INFO:teuthology.orchestra.run.smithi082.stdout:(158/166): NetworkManager-tui-1.45.91-1.el9.x86 511 kB/s | 247 kB 00:00 2024-02-26T10:36:10.663 INFO:teuthology.orchestra.run.smithi082.stdout:(159/166): elfutils-libelf-0.190-2.el9.x86_64.r 563 kB/s | 192 kB 00:00 2024-02-26T10:36:10.720 INFO:teuthology.orchestra.run.smithi082.stdout:(160/166): libacl-2.3.1-4.el9.x86_64.rpm 280 kB/s | 23 kB 00:00 2024-02-26T10:36:10.903 INFO:teuthology.orchestra.run.smithi082.stdout:(161/166): elfutils-libs-0.190-2.el9.x86_64.rpm 503 kB/s | 256 kB 00:00 2024-02-26T10:36:11.414 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-02-26T10:36:11.453 INFO:teuthology.orchestra.run.smithi082.stdout:(162/166): xfsdump-3.1.12-4.el9.x86_64.rpm 589 kB/s | 323 kB 00:00 2024-02-26T10:36:11.560 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T10:36:11.635 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T10:36:11.745 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T10:36:11.806 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T10:36:11.842 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T10:36:11.945 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T10:36:12.187 INFO:teuthology.orchestra.run.smithi082.stdout:(163/166): openssl-3.0.7-25.el9.x86_64.rpm 814 kB/s | 1.2 MB 00:01 2024-02-26T10:36:12.270 INFO:teuthology.orchestra.run.smithi082.stdout:(164/166): libacl-devel-2.3.1-4.el9.x86_64.rpm 900 kB/s | 74 kB 00:00 2024-02-26T10:36:12.369 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T10:36:12.407 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:36:12.592 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:36:12.624 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:36:12.662 INFO:teuthology.orchestra.run.smithi082.stdout:(165/166): xfsprogs-6.3.0-1.el9.x86_64.rpm 910 kB/s | 1.1 MB 00:01 2024-02-26T10:36:12.748 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:36:12.790 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:36:12.937 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T10:36:13.032 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T10:36:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:(166/166): openssl-libs-3.0.7-25.el9.x86_64.rpm 945 kB/s | 2.2 MB 00:02 2024-02-26T10:36:13.062 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-02-26T10:36:13.063 INFO:teuthology.orchestra.run.smithi082.stdout:Total 3.5 MB/s | 222 MB 01:04 2024-02-26T10:36:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T10:36:13.409 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T10:36:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:36:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:36:13.641 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:36:13.728 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:36:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:36:13.835 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:36:14.130 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:36:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:36:14.496 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T10:36:14.673 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:36:14.746 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:36:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:36:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:36:15.334 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:36:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T10:36:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T10:36:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T10:36:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T10:36:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T10:36:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T10:36:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T10:36:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T10:36:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T10:36:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T10:36:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T10:36:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T10:36:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T10:36:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T10:36:16.494 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-02-26T10:36:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 36/182 2024-02-26T10:36:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 37/182 2024-02-26T10:36:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 38/182 2024-02-26T10:36:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T10:36:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T10:36:16.857 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-02-26T10:36:16.857 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-02-26T10:36:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T10:36:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T10:36:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T10:36:17.871 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T10:36:17.968 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T10:36:18.007 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T10:36:18.063 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T10:36:18.135 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T10:36:18.190 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 49/182 2024-02-26T10:36:18.273 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T10:36:18.515 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T10:36:18.562 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T10:36:18.644 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T10:36:18.744 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T10:36:18.795 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T10:36:18.836 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T10:36:18.941 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T10:36:18.988 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T10:36:19.056 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T10:36:19.127 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T10:36:19.246 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-02-26T10:36:19.246 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-02-26T10:36:19.330 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T10:36:19.382 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T10:36:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T10:36:19.486 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T10:36:19.630 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T10:36:19.669 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T10:36:19.716 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T10:36:19.750 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T10:36:19.784 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T10:36:19.955 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T10:36:20.165 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T10:36:20.214 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T10:36:20.252 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T10:36:20.298 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T10:36:20.358 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T10:36:20.411 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T10:36:20.461 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T10:36:20.515 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T10:36:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T10:36:20.670 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T10:36:20.716 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T10:36:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T10:36:20.950 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T10:36:20.994 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T10:36:21.067 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T10:36:21.230 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T10:36:21.292 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T10:36:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T10:36:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T10:36:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T10:36:22.653 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T10:36:22.719 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T10:36:22.830 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T10:36:22.874 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T10:36:23.116 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T10:36:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T10:36:23.297 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T10:36:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T10:36:24.410 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T10:36:26.047 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T10:36:26.141 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T10:36:26.178 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T10:36:26.221 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T10:36:26.333 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-02-26T10:36:26.477 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/182 2024-02-26T10:36:26.568 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/182 2024-02-26T10:36:26.608 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T10:36:26.645 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:36:26.697 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/182 2024-02-26T10:36:26.751 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:36:26.772 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/182 2024-02-26T10:36:26.785 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 106/182 2024-02-26T10:36:26.810 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/182 2024-02-26T10:36:26.840 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T10:36:26.902 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/182 2024-02-26T10:36:27.315 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/182 2024-02-26T10:36:27.350 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:36:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T10:36:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:36:27.506 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:36:27.515 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:36:27.533 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 110/182 2024-02-26T10:36:27.542 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:36:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:36:27.681 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:36:27.843 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/182 2024-02-26T10:36:27.991 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/182 2024-02-26T10:36:28.293 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/182 2024-02-26T10:36:28.351 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/182 2024-02-26T10:36:28.384 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:36:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 15/182 2024-02-26T10:36:28.561 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:36:28.646 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 16/182 2024-02-26T10:36:28.678 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:36:28.743 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 17/182 2024-02-26T10:36:29.032 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:36:29.139 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 18/182 2024-02-26T10:36:29.476 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 19/182 2024-02-26T10:36:29.669 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:36:29.766 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 20/182 2024-02-26T10:36:29.957 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:36:30.030 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:36:30.176 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:36:30.213 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:36:30.372 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 21/182 2024-02-26T10:36:30.563 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 22/182 2024-02-26T10:36:30.691 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 23/182 2024-02-26T10:36:30.746 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 24/182 2024-02-26T10:36:30.795 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 25/182 2024-02-26T10:36:30.894 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:36:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/182 2024-02-26T10:36:30.933 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:36:30.940 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/182 2024-02-26T10:36:31.002 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/182 2024-02-26T10:36:31.044 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/182 2024-02-26T10:36:31.084 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/182 2024-02-26T10:36:31.103 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:36:31.103 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T10:36:31.103 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T10:36:31.103 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:31.142 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:36:31.210 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/182 2024-02-26T10:36:31.290 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 32/182 2024-02-26T10:36:31.341 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/182 2024-02-26T10:36:31.815 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/182 2024-02-26T10:36:31.950 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/182 2024-02-26T10:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 36/182 2024-02-26T10:36:32.055 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 37/182 2024-02-26T10:36:32.091 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 38/182 2024-02-26T10:36:32.210 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/182 2024-02-26T10:36:32.301 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-1.el9.noarch 40/182 2024-02-26T10:36:32.410 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/182 2024-02-26T10:36:32.586 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/182 2024-02-26T10:36:32.669 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/182 2024-02-26T10:36:33.353 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/182 2024-02-26T10:36:33.449 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/182 2024-02-26T10:36:33.527 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 46/182 2024-02-26T10:36:33.579 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 47/182 2024-02-26T10:36:33.669 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 48/182 2024-02-26T10:36:33.728 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 49/182 2024-02-26T10:36:33.808 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/182 2024-02-26T10:36:34.017 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 51/182 2024-02-26T10:36:34.100 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 52/182 2024-02-26T10:36:34.183 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/182 2024-02-26T10:36:34.302 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/182 2024-02-26T10:36:34.341 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/182 2024-02-26T10:36:34.381 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/182 2024-02-26T10:36:34.469 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/182 2024-02-26T10:36:34.514 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/182 2024-02-26T10:36:34.577 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/182 2024-02-26T10:36:34.637 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/182 2024-02-26T10:36:34.843 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/182 2024-02-26T10:36:34.903 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/182 2024-02-26T10:36:34.941 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 63/182 2024-02-26T10:36:34.991 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 64/182 2024-02-26T10:36:35.127 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 65/182 2024-02-26T10:36:35.165 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 66/182 2024-02-26T10:36:35.213 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 67/182 2024-02-26T10:36:35.273 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 68/182 2024-02-26T10:36:35.306 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 69/182 2024-02-26T10:36:35.491 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 70/182 2024-02-26T10:36:35.659 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 71/182 2024-02-26T10:36:35.704 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 72/182 2024-02-26T10:36:35.766 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/182 2024-02-26T10:36:35.822 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/182 2024-02-26T10:36:35.894 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/182 2024-02-26T10:36:35.952 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/182 2024-02-26T10:36:36.001 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/182 2024-02-26T10:36:36.047 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/182 2024-02-26T10:36:36.097 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 79/182 2024-02-26T10:36:36.151 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 80/182 2024-02-26T10:36:36.192 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 81/182 2024-02-26T10:36:36.370 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 82/182 2024-02-26T10:36:36.433 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/182 2024-02-26T10:36:36.476 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/182 2024-02-26T10:36:36.545 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/182 2024-02-26T10:36:36.692 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 86/182 2024-02-26T10:36:36.781 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/182 2024-02-26T10:36:37.718 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/182 2024-02-26T10:36:37.775 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/182 2024-02-26T10:36:37.808 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/182 2024-02-26T10:36:37.931 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/182 2024-02-26T10:36:38.002 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 92/182 2024-02-26T10:36:38.101 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/182 2024-02-26T10:36:38.135 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 94/182 2024-02-26T10:36:38.357 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/182 2024-02-26T10:36:38.390 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/182 2024-02-26T10:36:38.490 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/182 2024-02-26T10:36:39.284 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/182 2024-02-26T10:36:39.426 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/182 2024-02-26T10:36:40.929 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/182 2024-02-26T10:36:41.039 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/182 2024-02-26T10:36:41.076 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/182 2024-02-26T10:36:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/182 2024-02-26T10:36:41.463 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/182 2024-02-26T10:36:41.501 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:36:41.621 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 105/182 2024-02-26T10:36:41.649 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 106/182 2024-02-26T10:36:41.733 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/182 2024-02-26T10:36:42.247 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/182 2024-02-26T10:36:42.530 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:36:42.624 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 109/182 2024-02-26T10:36:42.665 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 110/182 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-02-26T10:36:44.308 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:44.537 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:36:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:36:44.588 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:44.588 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T10:36:44.588 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:45.198 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:36:45.262 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:36:45.905 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 111/182 2024-02-26T10:36:45.949 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:36:46.098 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 112/182 2024-02-26T10:36:46.099 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-26T10:36:46.099 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-26T10:36:46.099 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:36:46.141 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:36:46.255 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:36:46.337 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:36:46.337 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:46.338 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T10:36:46.338 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:36:46.338 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:36:46.338 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:46.580 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:36:46.655 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:36:46.655 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:46.655 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T10:36:46.655 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:36:46.655 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:36:46.656 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:47.164 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:36:47.235 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:36:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T10:36:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:36:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:36:47.236 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T10:36:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T10:36:47.533 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T10:36:47.781 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T10:36:47.844 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T10:36:47.934 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T10:36:48.077 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T10:36:48.248 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T10:36:48.308 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T10:36:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T10:36:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T10:36:48.441 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T10:36:48.476 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T10:36:49.084 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:36:49.121 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:36:49.180 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 132/182 2024-02-26T10:36:49.296 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 133/182 2024-02-26T10:36:49.331 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:36:49.669 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:36:49.715 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:36:50.897 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:36:50.940 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:36:51.096 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:36:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 137/182 2024-02-26T10:36:51.309 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:36:51.400 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:36:51.400 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:51.400 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T10:36:51.401 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:36:51.401 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:36:51.401 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:36:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:36:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 140/182 2024-02-26T10:36:56.723 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 141/182 2024-02-26T10:36:56.814 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 142/182 2024-02-26T10:36:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T10:36:57.824 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:36:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:36:57.899 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:57.900 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T10:36:57.900 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:36:57.900 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:36:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:36:58.025 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:36:58.025 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:58.025 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T10:36:58.025 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:58.101 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:36:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:36:58.180 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:58.180 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T10:36:58.180 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:36:58.180 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:36:58.180 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:58.560 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:36:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:36:58.655 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:58.655 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T10:36:58.655 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:36:58.655 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:36:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:36:58.730 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 148/182 2024-02-26T10:36:58.796 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 149/182 2024-02-26T10:36:58.838 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T10:36:58.910 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 113/182 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-02-26T10:36:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-02-26T10:36:59.101 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:36:59.211 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 152/182 2024-02-26T10:36:59.266 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T10:36:59.300 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 154/182 2024-02-26T10:36:59.339 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T10:36:59.350 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:36:59.390 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T10:36:59.399 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 114/182 2024-02-26T10:36:59.400 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:36:59.400 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-26T10:36:59.400 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:36:59.498 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:36:59.605 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T10:36:59.642 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T10:37:00.590 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:37:00.638 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:37:00.726 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:37:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T10:37:01.057 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:37:01.132 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 115/182 2024-02-26T10:37:01.132 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:01.132 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-26T10:37:01.132 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:37:01.132 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-26T10:37:01.133 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:01.381 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:37:01.451 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 116/182 2024-02-26T10:37:01.451 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:01.451 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-26T10:37:01.452 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:37:01.452 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-26T10:37:01.452 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:01.949 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:37:02.017 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 117/182 2024-02-26T10:37:02.017 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:02.017 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-26T10:37:02.017 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:37:02.017 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-26T10:37:02.017 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:02.140 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 118/182 2024-02-26T10:37:02.200 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 119/182 2024-02-26T10:37:02.262 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 162/182 2024-02-26T10:37:02.275 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 120/182 2024-02-26T10:37:02.412 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T10:37:02.499 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 121/182 2024-02-26T10:37:02.542 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T10:37:02.562 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 122/182 2024-02-26T10:37:02.582 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T10:37:02.599 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T10:37:02.624 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T10:37:02.646 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 123/182 2024-02-26T10:37:02.659 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T10:37:02.701 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T10:37:02.701 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:37:02.754 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 124/182 2024-02-26T10:37:02.855 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:37:02.914 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 125/182 2024-02-26T10:37:02.930 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T10:37:02.969 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 126/182 2024-02-26T10:37:02.971 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T10:37:03.005 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T10:37:03.011 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 127/182 2024-02-26T10:37:03.037 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T10:37:03.037 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:37:03.069 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 128/182 2024-02-26T10:37:03.117 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-16.el9.x86_64 129/182 2024-02-26T10:37:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 130/182 2024-02-26T10:37:03.175 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:37:03.205 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T10:37:03.244 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:37:03.244 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:37:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:37:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:37:03.713 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:37:03.749 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T10:37:03.781 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:37:03.791 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T10:37:03.836 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T10:37:03.858 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 131/182 2024-02-26T10:37:03.918 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 132/182 2024-02-26T10:37:03.961 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:37:04.036 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 133/182 2024-02-26T10:37:04.075 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:37:04.267 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:37:04.267 INFO:teuthology.orchestra.run.smithi067.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-26T10:37:04.267 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:04.407 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 134/182 2024-02-26T10:37:04.443 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:37:05.566 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 135/182 2024-02-26T10:37:05.601 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:37:05.743 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 136/182 2024-02-26T10:37:05.916 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 137/182 2024-02-26T10:37:05.945 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:37:06.015 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 138/182 2024-02-26T10:37:06.016 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:06.016 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-26T10:37:06.016 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:37:06.016 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-26T10:37:06.016 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:06.071 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:37:06.107 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 139/182 2024-02-26T10:37:06.235 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 140/182 2024-02-26T10:37:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 1/182 2024-02-26T10:37:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2/182 2024-02-26T10:37:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 3/182 2024-02-26T10:37:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 4/182 2024-02-26T10:37:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 5/182 2024-02-26T10:37:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 6/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 7/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 11/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 12/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 13/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 14/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 15/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 16/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 17/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 18/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 19/182 2024-02-26T10:37:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 20/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 21/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 22/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 23/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 24/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 25/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 26/182 2024-02-26T10:37:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 27/182 2024-02-26T10:37:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 28/182 2024-02-26T10:37:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 29/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 30/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 31/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 32/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 33/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 34/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 35/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 36/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 37/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 38/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 39/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T10:37:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T10:37:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T10:37:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T10:37:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T10:37:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T10:37:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T10:37:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T10:37:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 151/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 153/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T10:37:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T10:37:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T10:37:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T10:37:11.027 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 141/182 2024-02-26T10:37:11.121 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 142/182 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T10:37:11.662 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T10:37:11.663 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.664 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T10:37:11.665 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.6-16.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T10:37:11.666 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T10:37:11.667 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T10:37:11.668 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T10:37:11.669 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T10:37:11.670 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T10:37:11.671 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T10:37:11.672 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T10:37:11.673 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T10:37:11.674 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T10:37:11.674 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T10:37:11.674 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T10:37:11.674 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T10:37:11.674 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:11.674 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-02-26T10:37:12.041 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 143/182 2024-02-26T10:37:12.076 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:37:12.144 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 144/182 2024-02-26T10:37:12.144 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:12.144 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-26T10:37:12.144 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:37:12.144 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-26T10:37:12.145 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:12.194 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:37:12.262 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1686.g6085b 145/182 2024-02-26T10:37:12.262 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:12.262 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-26T10:37:12.262 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:12.336 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:37:12.411 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 146/182 2024-02-26T10:37:12.411 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:12.412 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-26T10:37:12.412 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:37:12.412 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-26T10:37:12.412 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:12.463 DEBUG:teuthology.parallel:result is None 2024-02-26T10:37:12.771 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:37:12.841 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 147/182 2024-02-26T10:37:12.841 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-26T10:37:12.841 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-26T10:37:12.841 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:37:12.841 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-26T10:37:12.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:12.910 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 148/182 2024-02-26T10:37:12.965 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 149/182 2024-02-26T10:37:13.014 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 150/182 2024-02-26T10:37:13.060 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 151/182 2024-02-26T10:37:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 152/182 2024-02-26T10:37:13.404 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 153/182 2024-02-26T10:37:13.438 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 154/182 2024-02-26T10:37:13.476 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 155/182 2024-02-26T10:37:13.526 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 156/182 2024-02-26T10:37:13.615 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:37:13.720 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 158/182 2024-02-26T10:37:13.763 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 159/182 2024-02-26T10:37:14.690 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:37:14.735 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:37:14.820 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 160/182 2024-02-26T10:37:14.920 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/182 2024-02-26T10:37:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 162/182 2024-02-26T10:37:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 163/182 2024-02-26T10:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 164/182 2024-02-26T10:37:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 165/182 2024-02-26T10:37:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 166/182 2024-02-26T10:37:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 167/182 2024-02-26T10:37:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/182 2024-02-26T10:37:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 169/182 2024-02-26T10:37:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:37:16.976 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 170/182 2024-02-26T10:37:17.019 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 171/182 2024-02-26T10:37:17.061 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 172/182 2024-02-26T10:37:17.103 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 173/182 2024-02-26T10:37:17.135 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 174/182 2024-02-26T10:37:17.136 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:37:17.273 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/182 2024-02-26T10:37:17.329 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 176/182 2024-02-26T10:37:17.364 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:37:17.364 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:37:17.661 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 177/182 2024-02-26T10:37:17.661 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:37:17.822 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 178/182 2024-02-26T10:37:17.864 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 179/182 2024-02-26T10:37:17.904 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 180/182 2024-02-26T10:37:17.960 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 181/182 2024-02-26T10:37:18.077 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:37:18.320 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 182/182 2024-02-26T10:37:18.320 INFO:teuthology.orchestra.run.smithi082.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-26T10:37:18.320 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:23.357 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 1/182 2024-02-26T10:37:23.357 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2/182 2024-02-26T10:37:23.357 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 3/182 2024-02-26T10:37:23.357 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 4/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1686.g6085b 5/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 6/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 7/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 8/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 9/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 10/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 11/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 12/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 13/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 14/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_ 15/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 16/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 17/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_6 18/182 2024-02-26T10:37:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86 19/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 20/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el 21/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-1686.g6085b91f.el9. 22/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_6 23/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 24/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 25/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 26/182 2024-02-26T10:37:23.359 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 27/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 28/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 29/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f. 30/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noa 31/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.n 32/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1686.g608 33/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el 34/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 35/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.e 36/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 37/182 2024-02-26T10:37:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 38/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 39/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-16.el9.x86_64 42/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/182 2024-02-26T10:37:23.375 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/182 2024-02-26T10:37:23.376 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/182 2024-02-26T10:37:23.377 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/182 2024-02-26T10:37:23.378 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/182 2024-02-26T10:37:23.379 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/182 2024-02-26T10:37:23.380 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/182 2024-02-26T10:37:23.381 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/182 2024-02-26T10:37:23.382 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 151/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 153/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 155/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 157/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 159/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 161/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/182 2024-02-26T10:37:23.383 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 173/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 174/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 175/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 176/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 177/182 2024-02-26T10:37:23.384 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 178/182 2024-02-26T10:37:23.385 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 179/182 2024-02-26T10:37:23.385 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 180/182 2024-02-26T10:37:23.385 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 181/182 2024-02-26T10:37:25.756 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 182/182 2024-02-26T10:37:25.756 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:25.756 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-02-26T10:37:25.756 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: acl-2.3.1-4.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-26T10:37:25.757 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.758 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.759 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.760 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-1686.g6085b91f.el9.noarch 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-16.el9.x86_64 2024-02-26T10:37:25.761 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.762 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-26T10:37:25.763 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.11-1.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-26T10:37:25.764 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-26T10:37:25.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-26T10:37:25.766 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-26T10:37:25.767 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.768 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-26T10:37:25.769 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-1686.g6085b91f.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-02-26T10:37:25.770 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-26T10:37:25.771 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-26T10:37:25.771 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-26T10:37:25.771 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-26T10:37:25.771 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-26T10:37:25.771 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-26T10:37:25.772 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:25.772 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-02-26T10:37:26.430 DEBUG:teuthology.parallel:result is None 2024-02-26T10:37:26.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:37:26.612 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T10:37:26.691 INFO:teuthology.orchestra.run.smithi067.stdout:19.0.0-1686.g6085b91f.el9 2024-02-26T10:37:26.692 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1686.g6085b91f.el9 2024-02-26T10:37:26.692 INFO:teuthology.task.install:The correct ceph version 19.0.0-1686.g6085b91f is installed. 2024-02-26T10:37:26.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6085b91fefeb74bf4310344a87d81d3217ef7893 2024-02-26T10:37:26.872 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-26T10:37:26.938 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-1686.g6085b91f.el9 2024-02-26T10:37:26.938 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1686.g6085b91f.el9 2024-02-26T10:37:26.939 INFO:teuthology.task.install:The correct ceph version 19.0.0-1686.g6085b91f is installed. 2024-02-26T10:37:26.941 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-26T10:37:26.941 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:26.941 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T10:37:27.052 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:27.052 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-26T10:37:27.129 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-26T10:37:27.129 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:27.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T10:37:27.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T10:37:27.335 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:27.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-02-26T10:37:27.419 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-26T10:37:27.538 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-26T10:37:27.539 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:27.539 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T10:37:27.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T10:37:27.789 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:27.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-26T10:37:27.862 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-26T10:37:27.977 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-26T10:37:27.977 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:27.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T10:37:28.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T10:37:28.183 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:28.184 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-02-26T10:37:28.257 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-26T10:37:28.364 INFO:teuthology.run_tasks:Running task ceph... 2024-02-26T10:37:28.452 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-02-26T10:37:28.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /var/log/ceph 2024-02-26T10:37:28.460 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /var/log/ceph 2024-02-26T10:37:28.560 INFO:tasks.ceph:Disabling ceph logrotate... 2024-02-26T10:37:28.560 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-26T10:37:28.566 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-26T10:37:28.664 INFO:tasks.ceph:Creating extra log directories... 2024-02-26T10:37:28.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-26T10:37:28.670 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-26T10:37:28.772 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-02-26T10:37:28.773 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-02-26T10:37:28.773 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-02-26T10:37:28.773 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:28.773 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-26T10:37:28.830 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-26T10:37:29.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-26T10:37:29.218 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:29.218 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-26T10:37:29.255 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-26T10:37:29.470 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-26T10:37:29.548 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-02-26T10:37:29.548 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-02-26T10:37:29.548 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557', 'branch': 'wip-vshankar-testing-20240226.064629', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7573557', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '7c0fbba6c508351b48a34af7de7b074eb29ad0f9'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240226.064629'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240226.064629', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 1879, 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'sleep_before_teardown': 0, 'subset': '788/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240226.064629', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi067.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNZXEQayW+z0FSWCIGyNxq4hGDs3AtBWyqL+83QzXIFLyaMXmnNDH7mSH/q9RIINXLSmXCuPUhqNV+2Xkv6VY3s=', 'smithi082.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzIN0+PSmnpktoeKuKPW5cX+J0qDvyfnUFf3H5ZNN5S4jHoYDQc8Ab/xnkEM2vKIs4KnCVib96B9xtHEbIUR54='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '7c0fbba6c508351b48a34af7de7b074eb29ad0f9'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '6085b91fefeb74bf4310344a87d81d3217ef7893', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_multifs_auth']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-02-26_10:07:12', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-02-26T10:37:29.549 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-26T10:37:29.557 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-26T10:37:29.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/run/ceph 2024-02-26T10:37:29.618 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/run/ceph 2024-02-26T10:37:29.715 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:29.715 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T10:37:29.776 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T10:37:29.776 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-02-26T10:37:29.829 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-26 10:37:04.706080677 +0000 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-26 10:33:30.986128921 +0000 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-26 10:33:30.986128921 +0000 2024-02-26T10:37:29.830 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-26 10:33:30.986128921 +0000 2024-02-26T10:37:29.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T10:37:29.920 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-26T10:37:29.920 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-26T10:37:29.920 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000360629 s, 1.4 MB/s 2024-02-26T10:37:29.924 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T10:37:29.985 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-02-26T10:37:30.037 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-26 10:37:04.707080654 +0000 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-26 10:33:31.559115683 +0000 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-26 10:33:31.559115683 +0000 2024-02-26T10:37:30.038 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-26 10:33:31.559115683 +0000 2024-02-26T10:37:30.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T10:37:30.120 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-26T10:37:30.120 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-26T10:37:30.120 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000314025 s, 1.6 MB/s 2024-02-26T10:37:30.123 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T10:37:30.180 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-02-26T10:37:30.232 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T10:37:30.232 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:30.232 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-02-26T10:37:30.232 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:30.233 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:30.233 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-26 10:37:04.708080631 +0000 2024-02-26T10:37:30.233 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-26 10:33:32.161101774 +0000 2024-02-26T10:37:30.233 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-26 10:33:32.161101774 +0000 2024-02-26T10:37:30.233 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-26 10:33:32.161101774 +0000 2024-02-26T10:37:30.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T10:37:30.321 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-26T10:37:30.321 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-26T10:37:30.321 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000337575 s, 1.5 MB/s 2024-02-26T10:37:30.326 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T10:37:30.381 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-26 10:37:04.709080608 +0000 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-26 10:33:32.809086803 +0000 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-26 10:33:32.809086803 +0000 2024-02-26T10:37:30.431 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-26 10:33:32.809086803 +0000 2024-02-26T10:37:30.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T10:37:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-26T10:37:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-26T10:37:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000370764 s, 1.4 MB/s 2024-02-26T10:37:30.530 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T10:37:30.590 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:30.590 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:30.590 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-02-26T10:37:30.642 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-26T10:37:30.642 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-26 10:37:18.748707101 +0000 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-26 10:33:11.197924982 +0000 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-26 10:33:11.197924982 +0000 2024-02-26T10:37:30.737 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-26 10:33:11.197924982 +0000 2024-02-26T10:37:30.737 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-26T10:37:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-26T10:37:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-26T10:37:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000396673 s, 1.3 MB/s 2024-02-26T10:37:30.820 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-26T10:37:30.878 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-02-26T10:37:30.926 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-26T10:37:30.926 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-26 10:37:18.749707081 +0000 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-26 10:33:11.784912867 +0000 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-26 10:33:11.784912867 +0000 2024-02-26T10:37:30.927 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-26 10:33:11.784912867 +0000 2024-02-26T10:37:30.927 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-26T10:37:31.011 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-26T10:37:31.011 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-26T10:37:31.011 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000306972 s, 1.7 MB/s 2024-02-26T10:37:31.015 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-26T10:37:31.068 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-26 10:37:18.750707061 +0000 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-26 10:33:12.357901041 +0000 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-26 10:33:12.357901041 +0000 2024-02-26T10:37:31.113 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-26 10:33:12.357901041 +0000 2024-02-26T10:37:31.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-26T10:37:31.248 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-26T10:37:31.248 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-26T10:37:31.248 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000300866 s, 1.7 MB/s 2024-02-26T10:37:31.252 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-26T10:37:31.304 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-02-26T10:37:31.350 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-26T10:37:31.350 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-02-26 10:37:18.750707061 +0000 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-02-26 10:33:12.915889524 +0000 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-02-26 10:33:12.915889524 +0000 2024-02-26T10:37:31.351 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-02-26 10:33:12.915889524 +0000 2024-02-26T10:37:31.351 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-26T10:37:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-02-26T10:37:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-02-26T10:37:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.00031582 s, 1.6 MB/s 2024-02-26T10:37:31.488 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-26T10:37:31.543 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:31.543 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-26T10:37:31.544 INFO:tasks.ceph:Generating config... 2024-02-26T10:37:31.545 INFO:tasks.ceph:[client] client mount timeout = 600 2024-02-26T10:37:31.545 INFO:tasks.ceph:[client] debug client = 20 2024-02-26T10:37:31.545 INFO:tasks.ceph:[client] debug ms = 1 2024-02-26T10:37:31.545 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-02-26T10:37:31.545 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-02-26T10:37:31.545 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-02-26T10:37:31.545 INFO:tasks.ceph:[mds] debug mds = 20 2024-02-26T10:37:31.545 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-02-26T10:37:31.545 INFO:tasks.ceph:[mds] debug ms = 1 2024-02-26T10:37:31.545 INFO:tasks.ceph:[mds] mds debug frag = True 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mgr] debug ms = 1 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mon] debug mon = 20 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mon] debug ms = 1 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mon] debug paxos = 20 2024-02-26T10:37:31.546 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-02-26T10:37:31.546 INFO:tasks.ceph:[osd] bdev async discard = True 2024-02-26T10:37:31.546 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-02-26T10:37:31.546 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-02-26T10:37:31.546 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-02-26T10:37:31.546 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] debug ms = 1 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] debug osd = 20 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-02-26T10:37:31.547 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-02-26T10:37:31.547 INFO:tasks.ceph:Setting up mon.a... 2024-02-26T10:37:31.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-02-26T10:37:31.696 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.keyring 2024-02-26T10:37:31.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-02-26T10:37:31.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-26T10:37:31.943 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.67'), ('mon.b', '172.21.15.82'), ('mon.c', '[v2:172.21.15.82:3301,v1:172.21.15.82:6790]')] 2024-02-26T10:37:31.944 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.67,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-02-26T10:37:31.945 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:31.945 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-02-26T10:37:32.001 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.67 --add b 172.21.15.82 --addv c '[v2:172.21.15.82:3301,v1:172.21.15.82:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:32.161 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:32.162 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: generated fsid f4a3a702-2ce8-46d3-b32d-e4f6872b8b58 2024-02-26T10:37:32.162 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-02-26T10:37:32.162 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 0 2024-02-26T10:37:32.162 INFO:teuthology.orchestra.run.smithi067.stdout:fsid f4a3a702-2ce8-46d3-b32d-e4f6872b8b58 2024-02-26T10:37:32.162 INFO:teuthology.orchestra.run.smithi067.stdout:last_changed 2024-02-26T10:37:32.158640+0000 2024-02-26T10:37:32.162 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-02-26T10:37:32.158640+0000 2024-02-26T10:37:32.163 INFO:teuthology.orchestra.run.smithi067.stdout:min_mon_release 17 (quincy) 2024-02-26T10:37:32.163 INFO:teuthology.orchestra.run.smithi067.stdout:election_strategy: 1 2024-02-26T10:37:32.163 INFO:teuthology.orchestra.run.smithi067.stdout:0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-02-26T10:37:32.163 INFO:teuthology.orchestra.run.smithi067.stdout:1: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-02-26T10:37:32.163 INFO:teuthology.orchestra.run.smithi067.stdout:2: [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] mon.c 2024-02-26T10:37:32.163 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-02-26T10:37:32.165 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-02-26T10:37:32.218 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f4a3a702-2ce8-46d3-b32d-e4f6872b8b58... 2024-02-26T10:37:32.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-26T10:37:32.226 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-02-26T10:37:32.398 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-02-26T10:37:32.399 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.400 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.67,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = f4a3a702-2ce8-46d3-b32d-e4f6872b8b58 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-02-26T10:37:32.401 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.402 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: bdev async discard = True 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: bdev enable discard = True 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore allocator = bitmap 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 1/20 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 1/20 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 4/10 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-02-26T10:37:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.404 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-02-26T10:37:32.405 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-26T10:37:32.406 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-02-26T10:37:32.407 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-02-26T10:37:32.408 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: chdir = "" 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: auth supported = cephx 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: filestore xattr use omap = true 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: mon clock drift allowed = 1.000 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: osd crush chooseleaf type = 0 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: auth debug = true 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on old message = true 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on bug = true 2024-02-26T10:37:32.409 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon pg warn max object skew = 0 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: # disable pg_autoscaler by default for new pools 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default size = 2 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow primary affinity = true 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow pg remap = true 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on legacy crush tunables = false 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on crush straw calc version zero = false 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on no sortbitwise = false 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on osd down out interval zero = false 2024-02-26T10:37:32.410 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on too few osds = false 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon_allow_pool_size_one = true 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: osd default data pool replay window = 5 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon allow pool delete = true 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon cluster log file level = debug 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: debug asserts on shutdown = true 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon health detail to clog = false 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: mon host = "172.21.15.67,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-02-26T10:37:32.411 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on skipped message = False 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = f4a3a702-2ce8-46d3-b32d-e4f6872b8b58 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout:[osd] 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd journal size = 100 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub load threshold = 5.0 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub max interval = 600 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd mclock profile = high_recovery_ops 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd recover clone overlap = true 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd recovery max chunk = 1048576 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug shutdown = true 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug op order = true 2024-02-26T10:37:32.412 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify stray on activate = true 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug trim objects = true 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: osd open classes on start = true 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug pg log writeout = true 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: osd deep scrub update digest min age = 30 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: osd map max advance = 10 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: journal zero on create = true 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: filestore ondisk finisher threads = 3 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: filestore apply finisher threads = 3 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: bdev debug aio = true 2024-02-26T10:37:32.413 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug misdirected ops = true 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: bdev async discard = True 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: bdev enable discard = True 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore allocator = bitmap 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore block size = 96636764160 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore fsck on mount = True 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluefs = 1/20 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluestore = 1/20 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: debug osd = 20 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: debug rocksdb = 4/10 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd backfillfull_ratio = 0.85 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd full ratio = 0.9 2024-02-26T10:37:32.414 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd nearfull ratio = 0.8 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: osd failsafe full ratio = 0.95 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: osd objectstore = bluestore 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr] 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: debug mgr = 20 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: mgr/telemetry/nag = false 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout:[mon] 2024-02-26T10:37:32.415 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: debug paxos = 20 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: mon data avail warn = 5 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: mon mgr mkfs grace = 240 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd reporter subtree level = osd 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd prime pg temp = true 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: auth mon ticket ttl = 660 # 11m 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: auth service ticket ttl = 240 # 4m 2024-02-26T10:37:32.416 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: # don't complain about insecure global_id in the test suite 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: # 1m isn't quite enough 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: mon_down_mkfs_grace = 2m 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_filestore_osds = false 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: mon op complaint time = 120 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout:[client] 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: rgw cache enabled = true 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable ops log = true 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable usage log = true 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-26T10:37:32.417 INFO:teuthology.orchestra.run.smithi067.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout: client mount timeout = 600 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout: debug client = 20 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.a] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.b] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.c] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout:[mds] 2024-02-26T10:37:32.418 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds = 20 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds balancer = 20 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug frag = True 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug scatterstat = True 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: mds op complaint time = 180 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: mds verify scatter = True 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2024-02-26T10:37:32.419 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2024-02-26T10:37:32.448 INFO:tasks.ceph:Creating admin key on mon.a... 2024-02-26T10:37:32.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-02-26T10:37:32.600 INFO:tasks.ceph:Copying monmap to all nodes... 2024-02-26T10:37:32.600 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:32.601 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-02-26T10:37:32.656 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:32.656 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-02-26T10:37:32.709 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:37:32.709 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:32.709 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-26T10:37:32.709 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-26T10:37:32.863 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:37:32.863 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:32.962 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:37:32.963 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:32.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-26T10:37:32.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-26T10:37:33.111 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:37:33.112 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:33.210 INFO:tasks.ceph:Setting up mon nodes... 2024-02-26T10:37:33.211 INFO:tasks.ceph:Setting up mgr nodes... 2024-02-26T10:37:33.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-02-26T10:37:33.402 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-02-26T10:37:33.413 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-02-26T10:37:33.605 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-02-26T10:37:33.616 INFO:tasks.ceph:Setting up mds nodes... 2024-02-26T10:37:33.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-02-26T10:37:33.810 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-02-26T10:37:33.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-02-26T10:37:33.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-02-26T10:37:34.109 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-02-26T10:37:34.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-02-26T10:37:34.221 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-02-26T10:37:34.410 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-02-26T10:37:34.420 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-02-26T10:37:34.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-02-26T10:37:34.663 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-02-26T10:37:34.672 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-02-26T10:37:34.756 INFO:tasks.ceph_client:Setting up client nodes... 2024-02-26T10:37:34.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-26T10:37:34.892 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-02-26T10:37:34.942 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-26T10:37:35.067 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-02-26T10:37:35.127 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-02-26T10:37:35.128 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-26T10:37:35.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-02-26T10:37:35.232 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:35.232 INFO:tasks.ceph:role: osd.0 2024-02-26T10:37:35.232 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:37:35.233 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:35.336 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:35.850 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-02-26T10:37:35.858 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-02-26T10:37:35.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-02-26T10:37:36.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-02-26T10:37:36.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-02-26T10:37:36.233 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:36.234 INFO:tasks.ceph:role: osd.1 2024-02-26T10:37:36.234 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:37:36.234 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-26T10:37:36.325 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:36.326 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:36.326 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:36.327 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:36.327 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:36.785 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-02-26T10:37:36.792 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-02-26T10:37:36.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-02-26T10:37:36.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-02-26T10:37:37.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-02-26T10:37:37.102 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:37.102 INFO:tasks.ceph:role: osd.2 2024-02-26T10:37:37.103 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:37:37.103 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-26T10:37:37.191 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:37.191 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:37.191 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:37.191 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:37.191 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:37.191 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:37.192 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:37.192 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:37.192 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:37.192 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:37.711 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-02-26T10:37:37.716 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-02-26T10:37:37.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-02-26T10:37:37.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-02-26T10:37:37.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-02-26T10:37:38.088 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:38.089 INFO:tasks.ceph:role: osd.3 2024-02-26T10:37:38.089 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:37:38.089 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-26T10:37:38.194 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:38.195 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:38.670 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-02-26T10:37:38.679 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-02-26T10:37:38.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-02-26T10:37:38.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-02-26T10:37:38.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:39.068+0000 7f44e591d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-02-26T10:37:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:39.071+0000 7f44e591d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-02-26T10:37:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:39.071+0000 7f44e591d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-26T10:37:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:39.071+0000 7f44e591d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-26T10:37:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:39.071+0000 7f44e591d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-26T10:37:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:39.080+0000 7f44e591d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-02-26T10:37:42.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-26T10:37:42.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:42.490+0000 7f36016c1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-02-26T10:37:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:42.491+0000 7f36016c1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-02-26T10:37:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:42.491+0000 7f36016c1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-26T10:37:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:42.491+0000 7f36016c1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-26T10:37:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:42.491+0000 7f36016c1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-26T10:37:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:42.494+0000 7f36016c1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-02-26T10:37:45.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-26T10:37:45.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:45.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:45.928+0000 7f88e9abf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-02-26T10:37:45.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:45.928+0000 7f88e9abf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-02-26T10:37:45.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:45.929+0000 7f88e9abf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-26T10:37:45.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:45.929+0000 7f88e9abf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-26T10:37:45.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:45.929+0000 7f88e9abf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-26T10:37:45.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:45.931+0000 7f88e9abf640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-02-26T10:37:49.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-26T10:37:49.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:49.365+0000 7f91e3973640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-02-26T10:37:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:49.366+0000 7f91e3973640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-02-26T10:37:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:49.366+0000 7f91e3973640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-26T10:37:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:49.366+0000 7f91e3973640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-26T10:37:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:49.366+0000 7f91e3973640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-26T10:37:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:37:49.368+0000 7f91e3973640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-02-26T10:37:52.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-26T10:37:52.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-02-26T10:37:52.750 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:52.750 INFO:tasks.ceph:role: osd.4 2024-02-26T10:37:52.750 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:37:52.751 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-26T10:37:52.849 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:52.849 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:52.849 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:52.849 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:52.849 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:52.849 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:52.850 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:52.850 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:52.850 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:52.850 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:53.356 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-02-26T10:37:53.362 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-02-26T10:37:53.362 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-02-26T10:37:53.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-02-26T10:37:53.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-02-26T10:37:53.716 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:53.716 INFO:tasks.ceph:role: osd.5 2024-02-26T10:37:53.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:37:53.716 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-26T10:37:53.798 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:53.799 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:54.254 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-02-26T10:37:54.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-02-26T10:37:54.260 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-02-26T10:37:54.384 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-02-26T10:37:54.467 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-02-26T10:37:54.540 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:54.540 INFO:tasks.ceph:role: osd.6 2024-02-26T10:37:54.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:37:54.540 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-26T10:37:54.626 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:54.627 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:55.138 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-02-26T10:37:55.144 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-02-26T10:37:55.144 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-02-26T10:37:55.306 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-02-26T10:37:55.401 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-02-26T10:37:55.488 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-26T10:37:55.488 INFO:tasks.ceph:role: osd.7 2024-02-26T10:37:55.488 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:37:55.488 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-02-26T10:37:55.585 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-26T10:37:55.586 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-26T10:37:55.586 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-26T10:37:55.586 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-26T10:37:56.045 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-02-26T10:37:56.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-02-26T10:37:56.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-02-26T10:37:56.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-02-26T10:37:56.231 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:56.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:56.374+0000 7ffbfba13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-02-26T10:37:56.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:56.375+0000 7ffbfba13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-02-26T10:37:56.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:56.375+0000 7ffbfba13640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-26T10:37:56.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:56.375+0000 7ffbfba13640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-26T10:37:56.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:56.375+0000 7ffbfba13640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-26T10:37:56.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:56.378+0000 7ffbfba13640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-02-26T10:37:59.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:37:59.554 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-26T10:37:59.558 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:37:59.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:37:59.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:59.805+0000 7fc96c211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-02-26T10:37:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:59.805+0000 7fc96c211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-02-26T10:37:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:59.805+0000 7fc96c211640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-26T10:37:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:59.805+0000 7fc96c211640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-26T10:37:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:59.805+0000 7fc96c211640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-26T10:37:59.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:37:59.808+0000 7fc96c211640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-02-26T10:38:02.731 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-26T10:38:02.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:38:02.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:02.959+0000 7f7afa928640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-02-26T10:38:02.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:02.960+0000 7f7afa928640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-02-26T10:38:02.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:02.960+0000 7f7afa928640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-26T10:38:02.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:02.960+0000 7f7afa928640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-26T10:38:02.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:02.960+0000 7f7afa928640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-26T10:38:02.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:02.962+0000 7f7afa928640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-02-26T10:38:06.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-02-26T10:38:06.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:38:06.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:06.410+0000 7fe020cc8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-02-26T10:38:06.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:06.411+0000 7fe020cc8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-02-26T10:38:06.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:06.411+0000 7fe020cc8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-26T10:38:06.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:06.411+0000 7fe020cc8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-26T10:38:06.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:06.411+0000 7fe020cc8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-26T10:38:06.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:06.413+0000 7fe020cc8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-02-26T10:38:09.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-02-26T10:38:09.672 INFO:tasks.ceph:Reading keys from all nodes... 2024-02-26T10:38:09.672 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:09.672 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-02-26T10:38:09.775 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:09.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-02-26T10:38:09.875 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:09.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-02-26T10:38:09.975 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:09.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-02-26T10:38:10.079 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:10.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-02-26T10:38:10.184 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:10.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-02-26T10:38:10.280 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:10.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-02-26T10:38:10.383 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:10.383 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-02-26T10:38:10.483 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:10.483 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-02-26T10:38:10.576 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:10.576 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-02-26T10:38:10.672 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:10.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-02-26T10:38:10.766 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:10.766 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-02-26T10:38:10.904 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:10.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-02-26T10:38:11.001 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:11.001 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-02-26T10:38:11.091 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:11.091 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-02-26T10:38:11.151 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:11.151 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-02-26T10:38:11.207 INFO:tasks.ceph:Adding keys to all mons... 2024-02-26T10:38:11.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-26T10:38:11.215 DEBUG:teuthology.orchestra.run.smithi082:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-26T10:38:11.300 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.y] 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDtadxlMa/TFxAAAa7erEK+tXM+k5KOfx2JgA== 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDtadxlrPwnMBAADC1UqiHSleXCZZ8LRLN/MQ== 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDuadxl2Z1dBhAAT9aRD1Te0TwHE93ZvNdInA== 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.0] 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDzadxlki0qBBAAWmQAFvxfJJ5kYMrk0ChwLQ== 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.1] 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQD2adxl4X5OHRAAHtaf00M+kXgvYOn1/ddr7A== 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.2] 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQD5adxlQqxnNxAAWzDQ/1DzO3r1ztTrociQzQ== 2024-02-26T10:38:11.301 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.3] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQD9adxlgCXOFRAAHdACX7JyyLLY0KPVHOycGw== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.x] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDtadxlQMXpIxAA0W1caz5FAL2wDMrNG3j5Mg== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDuadxlXEpIGBAANii7fywaaL0+vjyqLX6Atw== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDuadxlDx1gJxAAYGDhukySlrxjHaHWn+707g== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.4] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAEatxlJ8FqFhAA2XFN8d8NUYW1rITUu3tQnA== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.5] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAHatxlO9YRMBAA6z8hYbTwHnU0BV3OMxKSjA== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.6] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAKatxlzGhIORAANuWy2QCgtFcJRO4vQV4EkQ== 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.7] 2024-02-26T10:38:11.302 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAOatxlUwuJGBAAYZlIHvgD3suXzh5kyeUWqQ== 2024-02-26T10:38:11.303 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-02-26T10:38:11.303 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw== 2024-02-26T10:38:11.303 INFO:teuthology.orchestra.run.smithi082.stdout:[client.1] 2024-02-26T10:38:11.303 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw== 2024-02-26T10:38:11.307 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.y] 2024-02-26T10:38:11.307 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDtadxlMa/TFxAAAa7erEK+tXM+k5KOfx2JgA== 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDtadxlrPwnMBAADC1UqiHSleXCZZ8LRLN/MQ== 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxl2Z1dBhAAT9aRD1Te0TwHE93ZvNdInA== 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.0] 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDzadxlki0qBBAAWmQAFvxfJJ5kYMrk0ChwLQ== 2024-02-26T10:38:11.308 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.1] 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQD2adxl4X5OHRAAHtaf00M+kXgvYOn1/ddr7A== 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.2] 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQD5adxlQqxnNxAAWzDQ/1DzO3r1ztTrociQzQ== 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.3] 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQD9adxlgCXOFRAAHdACX7JyyLLY0KPVHOycGw== 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.x] 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDtadxlQMXpIxAA0W1caz5FAL2wDMrNG3j5Mg== 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxlXEpIGBAANii7fywaaL0+vjyqLX6Atw== 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxlDx1gJxAAYGDhukySlrxjHaHWn+707g== 2024-02-26T10:38:11.309 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.4] 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAEatxlJ8FqFhAA2XFN8d8NUYW1rITUu3tQnA== 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.5] 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAHatxlO9YRMBAA6z8hYbTwHnU0BV3OMxKSjA== 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.6] 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAKatxlzGhIORAANuWy2QCgtFcJRO4vQV4EkQ== 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.7] 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAOatxlUwuJGBAAYZlIHvgD3suXzh5kyeUWqQ== 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw== 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-02-26T10:38:11.310 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw== 2024-02-26T10:38:11.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-26T10:38:11.318 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-26T10:38:11.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:11.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:11.643 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:11.649 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:11.794 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:11.798 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:11.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:11.949 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:12.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:12.102 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:12.243 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:12.249 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:12.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-26T10:38:12.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-26T10:38:12.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:12.553 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:12.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:12.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-26T10:38:12.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:12.860 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.008 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.154 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-26T10:38:13.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-26T10:38:13.451 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-26T10:38:13.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-26T10:38:13.604 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-26T10:38:13.749 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-02-26T10:38:13.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-02-26T10:38:13.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-26T10:38:14.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-02-26T10:38:14.097 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-02-26T10:38:14.193 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-26T10:38:14.349 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-02-26T10:38:14.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-02-26T10:38:14.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-26T10:38:14.684 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-02-26T10:38:14.775 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:38:14.782 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-26T10:38:14.835 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-02-26T10:38:14.836 INFO:tasks.ceph.mon.a:Restarting daemon 2024-02-26T10:38:14.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-02-26T10:38:14.841 INFO:tasks.ceph.mon.a:Started 2024-02-26T10:38:14.841 INFO:tasks.ceph.mon.b:Restarting daemon 2024-02-26T10:38:14.841 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-02-26T10:38:14.846 INFO:tasks.ceph.mon.b:Started 2024-02-26T10:38:14.846 INFO:tasks.ceph.mon.c:Restarting daemon 2024-02-26T10:38:14.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-02-26T10:38:14.851 INFO:tasks.ceph.mon.c:Started 2024-02-26T10:38:14.851 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-02-26T10:38:14.851 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-02-26T10:38:14.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-02-26T10:38:14.856 INFO:tasks.ceph.mgr.y:Started 2024-02-26T10:38:14.856 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-02-26T10:38:14.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-02-26T10:38:14.861 INFO:tasks.ceph.mgr.x:Started 2024-02-26T10:38:14.861 INFO:tasks.ceph:Setting crush tunables to default 2024-02-26T10:38:14.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd crush tunables default 2024-02-26T10:38:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.038+0000 7f167f200640 1 Processor -- start 2024-02-26T10:38:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.039+0000 7f167f200640 1 -- start start 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f167f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f167f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 0x7f1678153730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f167f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f167f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16781532a0 con 0x7f16780aacd0 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f167f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1678157340 con 0x7f167813c3e0 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f167f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1678156d80 con 0x7f1678156410 2024-02-26T10:38:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f1677e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f1677e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55632/0 (socket says 172.21.15.67:55632) 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f1677e00640 1 -- 172.21.15.67:0/3992981903 learned_addr learned my addr 172.21.15.67:0/3992981903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f1677e00640 1 -- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 msgr2=0x7f1678154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f1677e00640 1 -- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 msgr2=0x7f1678154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.040+0000 7f1677e00640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f1677e00640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f167d200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f167d200640 1 -- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 msgr2=0x7f167806e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f167d200640 1 -- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 msgr2=0x7f167806e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f167d200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f167d200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 0x7f1678153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f1677400640 1 -- 172.21.15.67:0/3992981903 >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 msgr2=0x7f1678153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-02-26T10:38:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f1677400640 1 -- 172.21.15.67:0/3992981903 >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 msgr2=0x7f1678153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-26T10:38:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 0x7f1678153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-26T10:38:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.041+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 0x7f1678153730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-26T10:38:15.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.242+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.67:3300/0 conn(0x7f16780aacd0 0x7f1678153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.242+0000 7f167d200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.242+0000 7f1677e00640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.243+0000 7f1677400640 1 -- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 msgr2=0x7f167806e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.243+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.243+0000 7f1677400640 1 -- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 msgr2=0x7f1678154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.243+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.243+0000 7f1677400640 1 -- 172.21.15.67:0/3992981903 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678157060 con 0x7f16780aacd0 2024-02-26T10:38:15.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.245+0000 7f1677400640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780aacd0 0x7f1678153730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f167806f490 tx=0x7f166402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42753c5934003c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.245+0000 7f1676a00640 1 -- 172.21.15.67:0/3992981903 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664032740 con 0x7f16780aacd0 2024-02-26T10:38:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.245+0000 7f1676a00640 1 -- 172.21.15.67:0/3992981903 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16640328e0 con 0x7f16780aacd0 2024-02-26T10:38:15.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.245+0000 7f1676a00640 1 -- 172.21.15.67:0/3992981903 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664032cd0 con 0x7f16780aacd0 2024-02-26T10:38:15.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 -- 172.21.15.67:0/3992981903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780aacd0 msgr2=0x7f1678153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:15.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780aacd0 0x7f1678153730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f167806f490 tx=0x7f166402b870 comp rx=0 tx=0).stop 2024-02-26T10:38:15.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 -- 172.21.15.67:0/3992981903 shutdown_connections 2024-02-26T10:38:15.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1678156410 0x7f167806e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 --2- 172.21.15.67:0/3992981903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780aacd0 0x7f1678153730 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 --2- 172.21.15.67:0/3992981903 >> v2:172.21.15.82:3300/0 conn(0x7f167813c3e0 0x7f1678154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 -- 172.21.15.67:0/3992981903 >> 172.21.15.67:0/3992981903 conn(0x7f16780a84a0 msgr2=0x7f16780a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 -- 172.21.15.67:0/3992981903 shutdown_connections 2024-02-26T10:38:15.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.246+0000 7f167f200640 1 -- 172.21.15.67:0/3992981903 wait complete. 2024-02-26T10:38:15.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.249+0000 7f167f200640 1 Processor -- start 2024-02-26T10:38:15.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.249+0000 7f167f200640 1 -- start start 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.250+0000 7f167f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16780aacd0 0x7f1678135040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.250+0000 7f167f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f167813c3e0 0x7f16780c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.250+0000 7f167f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 0x7f16780c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.250+0000 7f167f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678153df0 con 0x7f1678156410 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.250+0000 7f167f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678153c70 con 0x7f16780aacd0 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.250+0000 7f167f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16781568c0 con 0x7f167813c3e0 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 0x7f16780c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 0x7f16780c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35644/0 (socket says 172.21.15.67:35644) 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 -- 172.21.15.67:0/2246299382 learned_addr learned my addr 172.21.15.67:0/2246299382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f1677e00640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16780aacd0 0x7f1678135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f1677400640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f167813c3e0 0x7f16780c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 -- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f167813c3e0 msgr2=0x7f16780c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f167813c3e0 0x7f16780c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 -- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16780aacd0 msgr2=0x7f1678135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16780aacd0 0x7f1678135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.251+0000 7f167d200640 1 -- 172.21.15.67:0/2246299382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678072360 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.252+0000 7f167d200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 0x7f16780c4740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1670004650 tx=0x7f1670007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.252+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670014610 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.252+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670019070 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.252+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1678071da0 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.252+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670014a10 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.252+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16780c4220 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.253+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f167001d410 con 0x7f1678156410 2024-02-26T10:38:15.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.253+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f16700086f0 con 0x7f1678156410 2024-02-26T10:38:15.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.254+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678074a30 con 0x7f1678156410 2024-02-26T10:38:15.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.257+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f167001d8e0 con 0x7f1678156410 2024-02-26T10:38:15.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.333+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f167806e260 con 0x7f1678156410 2024-02-26T10:38:15.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.335+0000 7f166fe00640 1 -- 172.21.15.67:0/2246299382 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f167001dac0 con 0x7f1678156410 2024-02-26T10:38:15.339 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.337+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 msgr2=0x7f16780c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.337+0000 7f167f200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 0x7f16780c4740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1670004650 tx=0x7f1670007700 comp rx=0 tx=0).stop 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 shutdown_connections 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1678156410 0x7f16780c4740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f167813c3e0 0x7f16780c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 --2- 172.21.15.67:0/2246299382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16780aacd0 0x7f1678135040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 >> 172.21.15.67:0/2246299382 conn(0x7f16780a84a0 msgr2=0x7f167813b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 shutdown_connections 2024-02-26T10:38:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:15.338+0000 7f167f200640 1 -- 172.21.15.67:0/2246299382 wait complete. 2024-02-26T10:38:15.359 INFO:tasks.ceph:check_enable_crimson: False 2024-02-26T10:38:15.359 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-02-26T10:38:15.359 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:15.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-02-26T10:38:15.447 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:15.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-02-26T10:38:15.576 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:15.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-02-26T10:38:15.663 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:38:15.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-02-26T10:38:15.798 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:15.798 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-02-26T10:38:15.814 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:15.811+0000 7f79c862b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:38:15.822 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:15.819+0000 7efd9a440040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-26T10:38:15.884 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:15.880+0000 7f79c862b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:38:15.888 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:15.888 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-02-26T10:38:15.889 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:15.886+0000 7efd9a440040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-26T10:38:16.016 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:16.016 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-02-26T10:38:16.094 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:38:16.095 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-02-26T10:38:16.216 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 239e9e71-e60d-4d32-b8a3-4a3dce0021cc 0 2024-02-26T10:38:16.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.447+0000 7f9a09800640 1 Processor -- start 2024-02-26T10:38:16.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.447+0000 7f9a09800640 1 -- start start 2024-02-26T10:38:16.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.448+0000 7f9a09800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a04152a80 0x7f9a040c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.448+0000 7f9a09800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a04155e40 0x7f9a0415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.448+0000 7f9a09800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a040c54e0 con 0x7f9a040c74d0 2024-02-26T10:38:16.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.448+0000 7f9a09800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040c9270 con 0x7f9a04152a80 2024-02-26T10:38:16.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.448+0000 7f9a09800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040c57c0 con 0x7f9a04155e40 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a04155e40 0x7f9a0415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a04155e40 0x7f9a0415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47794/0 (socket says 172.21.15.82:47794) 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 -- 172.21.15.82:0/3027890060 learned_addr learned my addr 172.21.15.82:0/3027890060 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a03e00640 1 --2- 172.21.15.82:0/3027890060 >> v2:172.21.15.67:3300/0 conn(0x7f9a04152a80 0x7f9a040c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 -- 172.21.15.82:0/3027890060 >> v2:172.21.15.67:3300/0 conn(0x7f9a04152a80 msgr2=0x7f9a040c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 --2- 172.21.15.82:0/3027890060 >> v2:172.21.15.67:3300/0 conn(0x7f9a04152a80 0x7f9a040c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 -- 172.21.15.82:0/3027890060 >> v1:172.21.15.82:6789/0 conn(0x7f9a040c74d0 legacy=0x7f9a04155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.449+0000 7f9a08e00640 1 -- 172.21.15.82:0/3027890060 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040c8e90 con 0x7f9a04155e40 2024-02-26T10:38:16.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.451+0000 7f9a08e00640 1 --2- 172.21.15.82:0/3027890060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0415f9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a04159630 tx=0x7f99fc02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a173a006d652482d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:16.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.451+0000 7f9a03400640 1 -- 172.21.15.82:0/3027890060 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc02ca80 con 0x7f9a04155e40 2024-02-26T10:38:16.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.451+0000 7f9a03400640 1 -- 172.21.15.82:0/3027890060 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc02cc20 con 0x7f9a04155e40 2024-02-26T10:38:16.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.451+0000 7f9a03400640 1 -- 172.21.15.82:0/3027890060 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0343f0 con 0x7f9a04155e40 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 -- 172.21.15.82:0/3027890060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 msgr2=0x7f9a0415f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 --2- 172.21.15.82:0/3027890060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0415f9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a04159630 tx=0x7f99fc02bee0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 -- 172.21.15.82:0/3027890060 shutdown_connections 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 --2- 172.21.15.82:0/3027890060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0415f9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 --2- 172.21.15.82:0/3027890060 >> v2:172.21.15.67:3300/0 conn(0x7f9a04152a80 0x7f9a040c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 -- 172.21.15.82:0/3027890060 >> 172.21.15.82:0/3027890060 conn(0x7f9a040ac270 msgr2=0x7f9a040ac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 -- 172.21.15.82:0/3027890060 shutdown_connections 2024-02-26T10:38:16.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.452+0000 7f9a09800640 1 -- 172.21.15.82:0/3027890060 wait complete. 2024-02-26T10:38:16.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.453+0000 7f9a09800640 1 Processor -- start 2024-02-26T10:38:16.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.453+0000 7f9a09800640 1 -- start start 2024-02-26T10:38:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.454+0000 7f9a09800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a040c74d0 0x7f9a0414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.454+0000 7f9a09800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 0x7f9a041e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.454+0000 7f9a09800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0414b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.454+0000 7f9a09800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0406ca20 con 0x7f9a040c74d0 2024-02-26T10:38:16.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.454+0000 7f9a09800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0406c8a0 con 0x7f9a04155e40 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.454+0000 7f9a09800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040c7ce0 con 0x7f9a04152a80 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 0x7f9a041e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f9a08e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0414b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 0x7f9a041e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:60064/0 (socket says 172.21.15.82:60064) 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 -- 172.21.15.82:0/3256131337 learned_addr learned my addr 172.21.15.82:0/3256131337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f9a08e00640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0414b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47798/0 (socket says 172.21.15.82:47798) 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f9a03e00640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a040c74d0 0x7f9a0414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 -- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 msgr2=0x7f9a0414b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0414b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 -- 172.21.15.82:0/3256131337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a040c74d0 msgr2=0x7f9a0414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a040c74d0 0x7f9a0414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 -- 172.21.15.82:0/3256131337 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0414d410 con 0x7f9a04152a80 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f9a03e00640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a040c74d0 0x7f9a0414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:16.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.455+0000 7f99fbe00640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 0x7f9a041e1d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c470 tx=0x7f99ec00c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:16.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0052d0 con 0x7f9a04152a80 2024-02-26T10:38:16.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec005490 con 0x7f9a04152a80 2024-02-26T10:38:16.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0414cbf0 con 0x7f9a04152a80 2024-02-26T10:38:16.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec007260 con 0x7f9a04152a80 2024-02-26T10:38:16.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a041e2260 con 0x7f9a04152a80 2024-02-26T10:38:16.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f99ec007400 con 0x7f9a04152a80 2024-02-26T10:38:16.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.456+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f99ec003a90 con 0x7f9a04152a80 2024-02-26T10:38:16.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.457+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a04073180 con 0x7f9a04152a80 2024-02-26T10:38:16.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.460+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9a04073180 con 0x7f9a04152a80 2024-02-26T10:38:16.513 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.510+0000 7efd9a440040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:38:16.521 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.518+0000 7f79c862b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-26T10:38:16.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.539+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "239e9e71-e60d-4d32-b8a3-4a3dce0021cc", "id": 0} v 0) v1 -- 0x7f9a040a9d20 con 0x7f9a04152a80 2024-02-26T10:38:16.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.546+0000 7f9a00c00640 1 -- 172.21.15.82:0/3256131337 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "239e9e71-e60d-4d32-b8a3-4a3dce0021cc", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a040a9d20 con 0x7f9a04152a80 2024-02-26T10:38:16.549 INFO:teuthology.orchestra.run.smithi082.stdout:0 2024-02-26T10:38:16.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 msgr2=0x7f9a041e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 0x7f9a041e1d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c470 tx=0x7f99ec00c930 comp rx=0 tx=0).stop 2024-02-26T10:38:16.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 shutdown_connections 2024-02-26T10:38:16.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a04155e40 0x7f9a0414b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a04152a80 0x7f9a041e1d20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 --2- 172.21.15.82:0/3256131337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a040c74d0 0x7f9a0414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 >> 172.21.15.82:0/3256131337 conn(0x7f9a040ac270 msgr2=0x7f9a040cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:16.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 shutdown_connections 2024-02-26T10:38:16.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.548+0000 7f9a09800640 1 -- 172.21.15.82:0/3256131337 wait complete. 2024-02-26T10:38:16.567 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new f6dd1fff-5c4f-4c8a-954a-49f49124cd7a 1 2024-02-26T10:38:16.580 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.577+0000 7efd9a440040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:38:16.590 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.587+0000 7f79c862b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-26T10:38:16.633 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.630+0000 7efd9a440040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:38:16.644 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.641+0000 7f79c862b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-26T10:38:16.699 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.696+0000 7efd9a440040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:38:16.710 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.707+0000 7f79c862b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-26T10:38:16.751 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.748+0000 7efd9a440040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:38:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.755+0000 7fa964e00640 1 Processor -- start 2024-02-26T10:38:16.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.756+0000 7fa964e00640 1 -- start start 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa964e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa960152380 0x7fa9600c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa964e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9600c54d0 0x7fa9600c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa964e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 0x7fa960154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa964e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa960152760 con 0x7fa960152380 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa964e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9600cb4e0 con 0x7fa9600c54d0 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa964e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96006b900 con 0x7fa9600c9960 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa95fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 0x7fa960154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa957e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9600c54d0 0x7fa9600c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa95fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 0x7fa960154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:60066/0 (socket says 172.21.15.82:60066) 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa95fe00640 1 -- 172.21.15.82:0/4219927747 learned_addr learned my addr 172.21.15.82:0/4219927747 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa957e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9600c54d0 0x7fa9600c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47806/0 (socket says 172.21.15.82:47806) 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.757+0000 7fa95f400640 1 --2- 172.21.15.82:0/4219927747 >> v2:172.21.15.67:3300/0 conn(0x7fa960152380 0x7fa9600c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.758+0000 7fa95fe00640 1 -- 172.21.15.82:0/4219927747 >> v2:172.21.15.82:3300/0 conn(0x7fa9600c54d0 msgr2=0x7fa9600c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.758+0000 7fa95fe00640 1 --2- 172.21.15.82:0/4219927747 >> v2:172.21.15.82:3300/0 conn(0x7fa9600c54d0 0x7fa9600c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.758+0000 7fa95fe00640 1 -- 172.21.15.82:0/4219927747 >> v2:172.21.15.67:3300/0 conn(0x7fa960152380 msgr2=0x7fa9600c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.758+0000 7fa95fe00640 1 --2- 172.21.15.82:0/4219927747 >> v2:172.21.15.67:3300/0 conn(0x7fa960152380 0x7fa9600c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.758+0000 7fa95fe00640 1 -- 172.21.15.82:0/4219927747 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa960159010 con 0x7fa9600c9960 2024-02-26T10:38:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.759+0000 7fa95fe00640 1 --2- 172.21.15.82:0/4219927747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 0x7fa960154ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa96008c7c0 tx=0x7fa95802bdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=357508745728c94f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.759+0000 7fa95ea00640 1 -- 172.21.15.82:0/4219927747 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95802c960 con 0x7fa9600c9960 2024-02-26T10:38:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.759+0000 7fa95ea00640 1 -- 172.21.15.82:0/4219927747 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95802cb00 con 0x7fa9600c9960 2024-02-26T10:38:16.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.759+0000 7fa95ea00640 1 -- 172.21.15.82:0/4219927747 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9580343f0 con 0x7fa9600c9960 2024-02-26T10:38:16.764 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.760+0000 7f79c862b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-26T10:38:16.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 -- 172.21.15.82:0/4219927747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 msgr2=0x7fa960154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 --2- 172.21.15.82:0/4219927747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 0x7fa960154ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa96008c7c0 tx=0x7fa95802bdc0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 -- 172.21.15.82:0/4219927747 shutdown_connections 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 --2- 172.21.15.82:0/4219927747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c9960 0x7fa960154ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 --2- 172.21.15.82:0/4219927747 >> v2:172.21.15.82:3300/0 conn(0x7fa9600c54d0 0x7fa9600c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 --2- 172.21.15.82:0/4219927747 >> v2:172.21.15.67:3300/0 conn(0x7fa960152380 0x7fa9600c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 -- 172.21.15.82:0/4219927747 >> 172.21.15.82:0/4219927747 conn(0x7fa9600ca610 msgr2=0x7fa9600caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 -- 172.21.15.82:0/4219927747 shutdown_connections 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.760+0000 7fa964e00640 1 -- 172.21.15.82:0/4219927747 wait complete. 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.761+0000 7fa964e00640 1 Processor -- start 2024-02-26T10:38:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 -- start start 2024-02-26T10:38:16.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 0x7fa96014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9600c9960 0x7fa9600ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960152380 0x7fa9600bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96006dca0 con 0x7fa960152380 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9600c8bf0 con 0x7fa9600c9960 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.762+0000 7fa964e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa960158c70 con 0x7fa9600c54d0 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 0x7fa96014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 0x7fa96014b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:60072/0 (socket says 172.21.15.82:60072) 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 -- 172.21.15.82:0/1240420554 learned_addr learned my addr 172.21.15.82:0/1240420554 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa957e00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9600c9960 0x7fa9600ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95fe00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960152380 0x7fa9600bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 -- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9600c9960 msgr2=0x7fa9600ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9600c9960 0x7fa9600ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 -- 172.21.15.82:0/1240420554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960152380 msgr2=0x7fa9600bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960152380 0x7fa9600bf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 -- 172.21.15.82:0/1240420554 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9600c31e0 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95fe00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960152380 0x7fa9600bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.763+0000 7fa95f400640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 0x7fa96014b830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa95000d800 tx=0x7fa95000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000b6c0 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa950019070 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000ba80 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9600c34c0 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9600c3a80 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa95000bc20 con 0x7fa9600c54d0 2024-02-26T10:38:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.764+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa950014910 con 0x7fa9600c54d0 2024-02-26T10:38:16.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.765+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa960074a30 con 0x7fa9600c54d0 2024-02-26T10:38:16.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.769+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa950014b10 con 0x7fa9600c54d0 2024-02-26T10:38:16.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.848+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f6dd1fff-5c4f-4c8a-954a-49f49124cd7a", "id": 1} v 0) v1 -- 0x7fa9600c8510 con 0x7fa9600c54d0 2024-02-26T10:38:16.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.854+0000 7fa957400640 1 -- 172.21.15.82:0/1240420554 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6dd1fff-5c4f-4c8a-954a-49f49124cd7a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa950014dc0 con 0x7fa9600c54d0 2024-02-26T10:38:16.857 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.855+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 msgr2=0x7fa96014b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.855+0000 7fa964e00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 0x7fa96014b830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa95000d800 tx=0x7fa95000dcc0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 shutdown_connections 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960152380 0x7fa9600bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9600c9960 0x7fa9600ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 --2- 172.21.15.82:0/1240420554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9600c54d0 0x7fa96014b830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 >> 172.21.15.82:0/1240420554 conn(0x7fa9600ca610 msgr2=0x7fa96006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:16.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 shutdown_connections 2024-02-26T10:38:16.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:16.856+0000 7fa964e00640 1 -- 172.21.15.82:0/1240420554 wait complete. 2024-02-26T10:38:16.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new d5b606f9-72f7-49f5-9c8c-cf21d2fd263d 2 2024-02-26T10:38:16.887 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.884+0000 7efd9a440040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:38:16.903 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.899+0000 7f79c862b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-26T10:38:16.971 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:16.968+0000 7efd9a440040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:38:16.988 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:16.985+0000 7f79c862b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-26T10:38:17.027 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.024+0000 7efd9a440040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:38:17.045 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:17.042+0000 7f79c862b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-26T10:38:17.091 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.088+0000 7efd9a440040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:38:17.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.088+0000 7fb107e00640 1 Processor -- start 2024-02-26T10:38:17.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.088+0000 7fb107e00640 1 -- start start 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.089+0000 7fb107e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb100157d30 0x7fb100158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.089+0000 7fb107e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb1000bdef0 0x7fb1000c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.089+0000 7fb107e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb100154f50 con 0x7fb1000c56e0 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.089+0000 7fb107e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1000c7440 con 0x7fb1000bdef0 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.089+0000 7fb107e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1000c7720 con 0x7fb100157d30 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.089+0000 7fb105400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb100157d30 0x7fb100158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb105400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb100157d30 0x7fb100158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47828/0 (socket says 172.21.15.82:47828) 2024-02-26T10:38:17.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb105400640 1 -- 172.21.15.82:0/3955729380 learned_addr learned my addr 172.21.15.82:0/3955729380 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:17.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb104a00640 1 --2- 172.21.15.82:0/3955729380 >> v2:172.21.15.67:3300/0 conn(0x7fb1000bdef0 0x7fb1000c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb105400640 1 -- 172.21.15.82:0/3955729380 >> v2:172.21.15.67:3300/0 conn(0x7fb1000bdef0 msgr2=0x7fb1000c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:17.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb105400640 1 --2- 172.21.15.82:0/3955729380 >> v2:172.21.15.67:3300/0 conn(0x7fb1000bdef0 0x7fb1000c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb105400640 1 -- 172.21.15.82:0/3955729380 >> v1:172.21.15.82:6789/0 conn(0x7fb1000c56e0 legacy=0x7fb100155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.090+0000 7fb105400640 1 -- 172.21.15.82:0/3955729380 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000ad2a0 con 0x7fb100157d30 2024-02-26T10:38:17.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.091+0000 7fb105400640 1 --2- 172.21.15.82:0/3955729380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb100158e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb100073330 tx=0x7fb0f402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b04c4432fcf491c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.091+0000 7fb0ffe00640 1 -- 172.21.15.82:0/3955729380 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4035020 con 0x7fb100157d30 2024-02-26T10:38:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb0ffe00640 1 -- 172.21.15.82:0/3955729380 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4031890 con 0x7fb100157d30 2024-02-26T10:38:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb0ffe00640 1 -- 172.21.15.82:0/3955729380 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4031c60 con 0x7fb100157d30 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 -- 172.21.15.82:0/3955729380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 msgr2=0x7fb100158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 --2- 172.21.15.82:0/3955729380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb100158e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb100073330 tx=0x7fb0f402b930 comp rx=0 tx=0).stop 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 -- 172.21.15.82:0/3955729380 shutdown_connections 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 --2- 172.21.15.82:0/3955729380 >> v2:172.21.15.67:3300/0 conn(0x7fb1000bdef0 0x7fb1000c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 --2- 172.21.15.82:0/3955729380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb100158e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 -- 172.21.15.82:0/3955729380 >> 172.21.15.82:0/3955729380 conn(0x7fb100072980 msgr2=0x7fb1000ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 -- 172.21.15.82:0/3955729380 shutdown_connections 2024-02-26T10:38:17.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.092+0000 7fb107e00640 1 -- 172.21.15.82:0/3955729380 wait complete. 2024-02-26T10:38:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.094+0000 7fb107e00640 1 Processor -- start 2024-02-26T10:38:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.094+0000 7fb107e00640 1 -- start start 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb107e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1000bdef0 0x7fb1000cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb107e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 0x7fb1001ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb107e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb1001ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb107e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb100156330 con 0x7fb1000bdef0 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb107e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000c7a00 con 0x7fb100157d30 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb107e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb100155b10 con 0x7fb1000c56e0 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb104a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 0x7fb1001ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb105e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb1001ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb104a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 0x7fb1001ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:60082/0 (socket says 172.21.15.82:60082) 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.095+0000 7fb104a00640 1 -- 172.21.15.82:0/3535652412 learned_addr learned my addr 172.21.15.82:0/3535652412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb105400640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1000bdef0 0x7fb1000cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb104a00640 1 -- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 msgr2=0x7fb1001ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb104a00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb1001ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb104a00640 1 -- 172.21.15.82:0/3535652412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1000bdef0 msgr2=0x7fb1000cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb104a00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1000bdef0 0x7fb1000cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb104a00640 1 -- 172.21.15.82:0/3535652412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000c4530 con 0x7fb1000c56e0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb105e00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb1001ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb104a00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 0x7fb1001ee430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00d800 tx=0x7fb0ec00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00b6c0 con 0x7fb1000c56e0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.096+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec019070 con 0x7fb1000c56e0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.097+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00ba80 con 0x7fb1000c56e0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.097+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1000ae3d0 con 0x7fb1000c56e0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.097+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1001ef090 con 0x7fb1000c56e0 2024-02-26T10:38:17.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.097+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb0ec00bc20 con 0x7fb1000c56e0 2024-02-26T10:38:17.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.097+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb0ec01e880 con 0x7fb1000c56e0 2024-02-26T10:38:17.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.098+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1001eeeb0 con 0x7fb1000c56e0 2024-02-26T10:38:17.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.100+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb1001eeeb0 con 0x7fb1000c56e0 2024-02-26T10:38:17.110 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:17.107+0000 7f79c862b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-26T10:38:17.148 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.145+0000 7efd9a440040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:38:17.168 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:17.164+0000 7f79c862b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-26T10:38:17.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.179+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d5b606f9-72f7-49f5-9c8c-cf21d2fd263d", "id": 2} v 0) v1 -- 0x7fb100074a30 con 0x7fb1000c56e0 2024-02-26T10:38:17.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.186+0000 7fb0fd600640 1 -- 172.21.15.82:0/3535652412 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5b606f9-72f7-49f5-9c8c-cf21d2fd263d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb0ec01eb50 con 0x7fb1000c56e0 2024-02-26T10:38:17.189 INFO:teuthology.orchestra.run.smithi082.stdout:2 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 msgr2=0x7fb1001ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 0x7fb1001ee430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00d800 tx=0x7fb0ec00dcc0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 shutdown_connections 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb100157d30 0x7fb1001ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1000c56e0 0x7fb1001ee430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 --2- 172.21.15.82:0/3535652412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1000bdef0 0x7fb1000cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 >> 172.21.15.82:0/3535652412 conn(0x7fb100072980 msgr2=0x7fb100153520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.187+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 shutdown_connections 2024-02-26T10:38:17.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.188+0000 7fb107e00640 1 -- 172.21.15.82:0/3535652412 wait complete. 2024-02-26T10:38:17.207 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 6f93e7e6-8121-4666-8641-c3c6d1e43fea 3 2024-02-26T10:38:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.387+0000 7fd24d000640 1 Processor -- start 2024-02-26T10:38:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.387+0000 7fd24d000640 1 -- start start 2024-02-26T10:38:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd24d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd24815e640 0x7fd248160a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd24d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd248152760 con 0x7fd248152380 2024-02-26T10:38:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd24d000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2480cb4e0 con 0x7fd2480c9420 2024-02-26T10:38:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd24d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24806b900 con 0x7fd24815e640 2024-02-26T10:38:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd23fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd2480c9420 0x7fd2480c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51088/0 (socket says 172.21.15.82:51088) 2024-02-26T10:38:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd23fe00640 1 -- 172.21.15.82:0/4116311995 learned_addr learned my addr 172.21.15.82:0/4116311995 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-02-26T10:38:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd247e00640 1 --2- 172.21.15.82:0/4116311995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd24815e640 0x7fd248160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd246a00640 1 -- 172.21.15.82:0/4116311995 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863023868 0 0) 0x7fd2480cb4e0 con 0x7fd2480c9420 2024-02-26T10:38:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.389+0000 7fd246a00640 1 -- 172.21.15.82:0/4116311995 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003520 con 0x7fd2480c9420 2024-02-26T10:38:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.390+0000 7fd247e00640 1 -- 172.21.15.82:0/4116311995 >> v1:172.21.15.82:6789/0 conn(0x7fd2480c9420 legacy=0x7fd2480c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.390+0000 7fd247e00640 1 -- 172.21.15.82:0/4116311995 >> v1:172.21.15.67:6789/0 conn(0x7fd248152380 legacy=0x7fd2480c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.390+0000 7fd247e00640 1 -- 172.21.15.82:0/4116311995 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd248159010 con 0x7fd24815e640 2024-02-26T10:38:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.391+0000 7fd247e00640 1 --2- 172.21.15.82:0/4116311995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd24815e640 0x7fd248160a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd240009dc0 tx=0x7fd24002bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e824a6eadc05ff8a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd246a00640 1 -- 172.21.15.82:0/4116311995 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240039070 con 0x7fd24815e640 2024-02-26T10:38:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd246a00640 1 -- 172.21.15.82:0/4116311995 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24002cba0 con 0x7fd24815e640 2024-02-26T10:38:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd246a00640 1 -- 172.21.15.82:0/4116311995 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240034440 con 0x7fd24815e640 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 -- 172.21.15.82:0/4116311995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd24815e640 msgr2=0x7fd248160a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 --2- 172.21.15.82:0/4116311995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd24815e640 0x7fd248160a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd240009dc0 tx=0x7fd24002bdf0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 -- 172.21.15.82:0/4116311995 shutdown_connections 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 --2- 172.21.15.82:0/4116311995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd24815e640 0x7fd248160a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 -- 172.21.15.82:0/4116311995 >> 172.21.15.82:0/4116311995 conn(0x7fd2480ca610 msgr2=0x7fd2480caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 -- 172.21.15.82:0/4116311995 shutdown_connections 2024-02-26T10:38:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.392+0000 7fd24d000640 1 -- 172.21.15.82:0/4116311995 wait complete. 2024-02-26T10:38:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.394+0000 7fd24d000640 1 Processor -- start 2024-02-26T10:38:17.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.394+0000 7fd24d000640 1 -- start start 2024-02-26T10:38:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd24d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 0x7fd2480af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd24d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd248152380 0x7fd24814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd24d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd24815e640 0x7fd24814ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd24d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24806dca0 con 0x7fd248152380 2024-02-26T10:38:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd24d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480c8bf0 con 0x7fd24815e640 2024-02-26T10:38:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd24d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480c9fe0 con 0x7fd2480c9420 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 0x7fd2480af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 0x7fd2480af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:60090/0 (socket says 172.21.15.82:60090) 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 -- 172.21.15.82:0/4183336260 learned_addr learned my addr 172.21.15.82:0/4183336260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247e00640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd24815e640 0x7fd24814ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd23fe00640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd248152380 0x7fd24814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 -- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd24815e640 msgr2=0x7fd24814ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd24815e640 0x7fd24814ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 -- 172.21.15.82:0/4183336260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd248152380 msgr2=0x7fd24814e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd248152380 0x7fd24814e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.395+0000 7fd247400640 1 -- 172.21.15.82:0/4183336260 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2481eae80 con 0x7fd2480c9420 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd247400640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 0x7fd2480af940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c450 tx=0x7fd23800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380052d0 con 0x7fd2480c9420 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238005470 con 0x7fd2480c9420 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238012870 con 0x7fd2480c9420 2024-02-26T10:38:17.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd248152da0 con 0x7fd2480c9420 2024-02-26T10:38:17.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd248153360 con 0x7fd2480c9420 2024-02-26T10:38:17.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd238012a10 con 0x7fd2480c9420 2024-02-26T10:38:17.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.396+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd238012d00 con 0x7fd2480c9420 2024-02-26T10:38:17.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.397+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd248153560 con 0x7fd2480c9420 2024-02-26T10:38:17.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.399+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd248152760 con 0x7fd2480c9420 2024-02-26T10:38:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.477+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6f93e7e6-8121-4666-8641-c3c6d1e43fea", "id": 3} v 0) v1 -- 0x7fd248074a30 con 0x7fd2480c9420 2024-02-26T10:38:17.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.485+0000 7fd23f400640 1 -- 172.21.15.82:0/4183336260 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f93e7e6-8121-4666-8641-c3c6d1e43fea", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd2380036c0 con 0x7fd2480c9420 2024-02-26T10:38:17.488 INFO:teuthology.orchestra.run.smithi082.stdout:3 2024-02-26T10:38:17.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 msgr2=0x7fd2480af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 0x7fd2480af940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c450 tx=0x7fd23800c910 comp rx=0 tx=0).stop 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 shutdown_connections 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd24815e640 0x7fd24814ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd248152380 0x7fd24814e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 --2- 172.21.15.82:0/4183336260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2480c9420 0x7fd2480af940 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 >> 172.21.15.82:0/4183336260 conn(0x7fd2480ca610 msgr2=0x7fd24814fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 shutdown_connections 2024-02-26T10:38:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.486+0000 7fd24d000640 1 -- 172.21.15.82:0/4183336260 wait complete. 2024-02-26T10:38:17.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new e6238c1b-6838-4531-86e8-8decc09c7a29 4 2024-02-26T10:38:17.620 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.617+0000 7efd9a440040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:38:17.646 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:17.642+0000 7f79c862b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-26T10:38:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.692+0000 7f1ca9800640 1 Processor -- start 2024-02-26T10:38:17.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.692+0000 7f1ca9800640 1 -- start start 2024-02-26T10:38:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.693+0000 7f1ca9800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ca40c9db0 0x7f1ca4155730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.693+0000 7f1ca9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ca4155c70 0x7f1ca415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca9800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca40c54e0 con 0x7f1ca4152a80 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca9800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40c9270 con 0x7f1ca4155c70 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca9800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40c57c0 con 0x7f1ca40c9db0 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ca40c9db0 0x7f1ca4155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ca40c9db0 0x7f1ca4155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47860/0 (socket says 172.21.15.82:47860) 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 -- 172.21.15.82:0/2100624882 learned_addr learned my addr 172.21.15.82:0/2100624882 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca8e00640 1 --2- 172.21.15.82:0/2100624882 >> v2:172.21.15.67:3300/0 conn(0x7f1ca4155c70 0x7f1ca415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 -- 172.21.15.82:0/2100624882 >> v2:172.21.15.67:3300/0 conn(0x7f1ca4155c70 msgr2=0x7f1ca415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 --2- 172.21.15.82:0/2100624882 >> v2:172.21.15.67:3300/0 conn(0x7f1ca4155c70 0x7f1ca415f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 -- 172.21.15.82:0/2100624882 >> v1:172.21.15.67:6789/0 conn(0x7f1ca4152a80 legacy=0x7f1ca40c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:17.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.694+0000 7f1ca3400640 1 -- 172.21.15.82:0/2100624882 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40c76a0 con 0x7f1ca40c9db0 2024-02-26T10:38:17.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.695+0000 7f1ca3400640 1 --2- 172.21.15.82:0/2100624882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca4155730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ca406db30 tx=0x7f1c9002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5660c3437be55b36 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:17.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.696+0000 7f1ca2a00640 1 -- 172.21.15.82:0/2100624882 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90035020 con 0x7f1ca40c9db0 2024-02-26T10:38:17.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.696+0000 7f1ca2a00640 1 -- 172.21.15.82:0/2100624882 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c90031870 con 0x7f1ca40c9db0 2024-02-26T10:38:17.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.696+0000 7f1ca2a00640 1 -- 172.21.15.82:0/2100624882 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90031c40 con 0x7f1ca40c9db0 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.696+0000 7f1ca9800640 1 -- 172.21.15.82:0/2100624882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 msgr2=0x7f1ca4155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.696+0000 7f1ca9800640 1 --2- 172.21.15.82:0/2100624882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca4155730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ca406db30 tx=0x7f1c9002b910 comp rx=0 tx=0).stop 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.697+0000 7f1ca9800640 1 -- 172.21.15.82:0/2100624882 shutdown_connections 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.697+0000 7f1ca9800640 1 --2- 172.21.15.82:0/2100624882 >> v2:172.21.15.67:3300/0 conn(0x7f1ca4155c70 0x7f1ca415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.697+0000 7f1ca9800640 1 --2- 172.21.15.82:0/2100624882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca4155730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.697+0000 7f1ca9800640 1 -- 172.21.15.82:0/2100624882 >> 172.21.15.82:0/2100624882 conn(0x7f1ca40ac270 msgr2=0x7f1ca40ac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:17.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.697+0000 7f1ca9800640 1 -- 172.21.15.82:0/2100624882 shutdown_connections 2024-02-26T10:38:17.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.697+0000 7f1ca9800640 1 -- 172.21.15.82:0/2100624882 wait complete. 2024-02-26T10:38:17.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.698+0000 7f1ca9800640 1 Processor -- start 2024-02-26T10:38:17.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.698+0000 7f1ca9800640 1 -- start start 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca40c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ca4152a80 0x7f1ca41db280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ca4155c70 0x7f1ca41ede70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40c5c20 con 0x7f1ca4152a80 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40c5aa0 con 0x7f1ca40c9db0 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca406c8a0 con 0x7f1ca4155c70 2024-02-26T10:38:17.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca40c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca40c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47872/0 (socket says 172.21.15.82:47872) 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca3e00640 1 -- 172.21.15.82:0/4287423283 learned_addr learned my addr 172.21.15.82:0/4287423283 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.699+0000 7f1ca8e00640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ca4155c70 0x7f1ca41ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3400640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ca4152a80 0x7f1ca41db280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3e00640 1 -- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ca4155c70 msgr2=0x7f1ca41ede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3e00640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ca4155c70 0x7f1ca41ede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3e00640 1 -- 172.21.15.82:0/4287423283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ca4152a80 msgr2=0x7f1ca41db280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3e00640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ca4152a80 0x7f1ca41db280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3e00640 1 -- 172.21.15.82:0/4287423283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40bee80 con 0x7f1ca40c9db0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca8e00640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ca4155c70 0x7f1ca41ede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca3e00640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca40c2eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c460 tx=0x7f1c9400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940052d0 con 0x7f1ca40c9db0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94005470 con 0x7f1ca40c9db0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.700+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca41de130 con 0x7f1ca40c9db0 2024-02-26T10:38:17.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.701+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94012870 con 0x7f1ca40c9db0 2024-02-26T10:38:17.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.701+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca41de6f0 con 0x7f1ca40c9db0 2024-02-26T10:38:17.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.701+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1c94012a10 con 0x7f1ca40c9db0 2024-02-26T10:38:17.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.701+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f1c94003b00 con 0x7f1ca40c9db0 2024-02-26T10:38:17.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.702+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca4075020 con 0x7f1ca40c9db0 2024-02-26T10:38:17.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.704+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1ca4075020 con 0x7f1ca40c9db0 2024-02-26T10:38:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.783+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6238c1b-6838-4531-86e8-8decc09c7a29", "id": 4} v 0) v1 -- 0x7f1ca4073180 con 0x7f1ca40c9db0 2024-02-26T10:38:17.789 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.786+0000 7efd9a440040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:38:17.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.790+0000 7f1c9be00640 1 -- 172.21.15.82:0/4287423283 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6238c1b-6838-4531-86e8-8decc09c7a29", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ca4073180 con 0x7f1ca40c9db0 2024-02-26T10:38:17.793 INFO:teuthology.orchestra.run.smithi082.stdout:4 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 msgr2=0x7f1ca40c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca40c2eb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c460 tx=0x7f1c9400c920 comp rx=0 tx=0).stop 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 shutdown_connections 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ca4155c70 0x7f1ca41ede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ca4152a80 0x7f1ca41db280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 --2- 172.21.15.82:0/4287423283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca40c9db0 0x7f1ca40c2eb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.791+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 >> 172.21.15.82:0/4287423283 conn(0x7f1ca40ac270 msgr2=0x7f1ca40ac650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.792+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 shutdown_connections 2024-02-26T10:38:17.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.792+0000 7f1ca9800640 1 -- 172.21.15.82:0/4287423283 wait complete. 2024-02-26T10:38:17.812 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 5f87545d-44e7-41af-bbcc-306a21a8159d 5 2024-02-26T10:38:17.819 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:17.811+0000 7f79c862b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-26T10:38:17.937 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.934+0000 7efd9a440040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:38:17.959 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:17.956+0000 7f79c862b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-26T10:38:17.993 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:17.990+0000 7efd9a440040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:38:18.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.998+0000 7fd421e00640 1 Processor -- start 2024-02-26T10:38:18.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:17.999+0000 7fd421e00640 1 -- start start 2024-02-26T10:38:18.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd41c152380 0x7fd41c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd41c0c9b30 0x7fd41c15f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c152760 con 0x7fd41c0c54d0 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c0cb4e0 con 0x7fd41c152380 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c06b900 con 0x7fd41c0c9b30 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd41c0c9b30 0x7fd41c15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd41c0c9b30 0x7fd41c15f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37392/0 (socket says 172.21.15.82:37392) 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421400640 1 -- 172.21.15.82:0/901921761 learned_addr learned my addr 172.21.15.82:0/901921761 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd420a00640 1 --2- 172.21.15.82:0/901921761 >> v2:172.21.15.67:3300/0 conn(0x7fd41c152380 0x7fd41c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421400640 1 -- 172.21.15.82:0/901921761 >> v2:172.21.15.67:3300/0 conn(0x7fd41c152380 msgr2=0x7fd41c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421400640 1 --2- 172.21.15.82:0/901921761 >> v2:172.21.15.67:3300/0 conn(0x7fd41c152380 0x7fd41c0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.000+0000 7fd421400640 1 -- 172.21.15.82:0/901921761 >> v1:172.21.15.67:6789/0 conn(0x7fd41c0c54d0 legacy=0x7fd41c0c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:18.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.001+0000 7fd421400640 1 -- 172.21.15.82:0/901921761 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c159010 con 0x7fd41c0c9b30 2024-02-26T10:38:18.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.001+0000 7fd420a00640 1 --2- 172.21.15.82:0/901921761 >> v2:172.21.15.67:3300/0 conn(0x7fd41c152380 0x7fd41c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd421400640 1 --2- 172.21.15.82:0/901921761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c9b30 0x7fd41c15f640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0bdcd0 tx=0x7fd41402bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4827e30144b271e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd41b400640 1 -- 172.21.15.82:0/901921761 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41402c960 con 0x7fd41c0c9b30 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd41b400640 1 -- 172.21.15.82:0/901921761 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41402cb00 con 0x7fd41c0c9b30 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd41b400640 1 -- 172.21.15.82:0/901921761 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4140343f0 con 0x7fd41c0c9b30 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd421e00640 1 -- 172.21.15.82:0/901921761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c9b30 msgr2=0x7fd41c15f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd421e00640 1 --2- 172.21.15.82:0/901921761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c9b30 0x7fd41c15f640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0bdcd0 tx=0x7fd41402bdc0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.002+0000 7fd421e00640 1 -- 172.21.15.82:0/901921761 shutdown_connections 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.003+0000 7fd421e00640 1 --2- 172.21.15.82:0/901921761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c9b30 0x7fd41c15f640 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.003+0000 7fd421e00640 1 --2- 172.21.15.82:0/901921761 >> v2:172.21.15.67:3300/0 conn(0x7fd41c152380 0x7fd41c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.003+0000 7fd421e00640 1 -- 172.21.15.82:0/901921761 >> 172.21.15.82:0/901921761 conn(0x7fd41c0ca610 msgr2=0x7fd41c0caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.003+0000 7fd421e00640 1 -- 172.21.15.82:0/901921761 shutdown_connections 2024-02-26T10:38:18.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.003+0000 7fd421e00640 1 -- 172.21.15.82:0/901921761 wait complete. 2024-02-26T10:38:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.004+0000 7fd421e00640 1 Processor -- start 2024-02-26T10:38:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.004+0000 7fd421e00640 1 -- start start 2024-02-26T10:38:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd421e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 0x7fd41c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd421e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd41c0c9b30 0x7fd41c0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd421e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c152380 0x7fd41c0c3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd421e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c06dca0 con 0x7fd41c152380 2024-02-26T10:38:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd421e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0c8bf0 con 0x7fd41c0c54d0 2024-02-26T10:38:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd421e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c158c70 con 0x7fd41c0c9b30 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd420a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 0x7fd41c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd41be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd41c0c9b30 0x7fd41c0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd420a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 0x7fd41c14b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37396/0 (socket says 172.21.15.82:37396) 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.005+0000 7fd420a00640 1 -- 172.21.15.82:0/2800117062 learned_addr learned my addr 172.21.15.82:0/2800117062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd421400640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c152380 0x7fd41c0c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd41be00640 1 -- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 msgr2=0x7fd41c14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd41be00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 0x7fd41c14b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd41be00640 1 -- 172.21.15.82:0/2800117062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c152380 msgr2=0x7fd41c0c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd41be00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c152380 0x7fd41c0c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd41be00640 1 -- 172.21.15.82:0/2800117062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c1583b0 con 0x7fd41c0c9b30 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd420a00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 0x7fd41c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd421400640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c152380 0x7fd41c0c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd41be00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd41c0c9b30 0x7fd41c0c4310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd40800b3e0 tx=0x7fd40800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408015030 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408004020 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c0c2190 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.006+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408004450 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.007+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0c2750 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.007+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd408018070 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.007+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd408011630 con 0x7fd41c0c9b30 2024-02-26T10:38:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.008+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c074a30 con 0x7fd41c0c9b30 2024-02-26T10:38:18.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.010+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd408003910 con 0x7fd41c0c9b30 2024-02-26T10:38:18.016 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:18.013+0000 7f79c862b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-26T10:38:18.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.091+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5f87545d-44e7-41af-bbcc-306a21a8159d", "id": 5} v 0) v1 -- 0x7fd41c0c8510 con 0x7fd41c0c9b30 2024-02-26T10:38:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.098+0000 7fd418c00640 1 -- 172.21.15.82:0/2800117062 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5f87545d-44e7-41af-bbcc-306a21a8159d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd408003af0 con 0x7fd41c0c9b30 2024-02-26T10:38:18.101 INFO:teuthology.orchestra.run.smithi082.stdout:5 2024-02-26T10:38:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd41c0c9b30 msgr2=0x7fd41c0c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd41c0c9b30 0x7fd41c0c4310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd40800b3e0 tx=0x7fd40800b8a0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 shutdown_connections 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c152380 0x7fd41c0c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd41c0c9b30 0x7fd41c0c4310 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 --2- 172.21.15.82:0/2800117062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0c54d0 0x7fd41c14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 >> 172.21.15.82:0/2800117062 conn(0x7fd41c0ca610 msgr2=0x7fd41c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 shutdown_connections 2024-02-26T10:38:18.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.099+0000 7fd421e00640 1 -- 172.21.15.82:0/2800117062 wait complete. 2024-02-26T10:38:18.119 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new b510a5fe-eeb5-4ae9-807e-95e4e00db1cd 6 2024-02-26T10:38:18.126 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:18.123+0000 7efd9a440040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:38:18.150 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:18.147+0000 7f79c862b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:38:18.242 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:18.239+0000 7efd9a440040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:38:18.269 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:18.266+0000 7f79c862b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-26T10:38:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.308+0000 7fbee7200640 1 Processor -- start 2024-02-26T10:38:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.308+0000 7fbee7200640 1 -- start start 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbee8152380 0x7fbee80c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee80c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee8152760 con 0x7fbee80c9960 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee80cb4e0 con 0x7fbee8152380 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee806b900 con 0x7fbee80c54d0 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee80c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbedfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee80c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:43526/0 (socket says 172.21.15.82:43526) 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbedfe00640 1 -- 172.21.15.82:0/2368199700 learned_addr learned my addr 172.21.15.82:0/2368199700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee5e00640 1 --2- 172.21.15.82:0/2368199700 >> v2:172.21.15.67:3300/0 conn(0x7fbee8152380 0x7fbee80c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee5400640 1 -- 172.21.15.82:0/2368199700 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824638486 0 0) 0x7fbee0001490 con 0x7fbee80c9960 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.310+0000 7fbee5400640 1 -- 172.21.15.82:0/2368199700 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc003520 con 0x7fbee80c9960 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.311+0000 7fbedfe00640 1 -- 172.21.15.82:0/2368199700 >> v2:172.21.15.67:3300/0 conn(0x7fbee8152380 msgr2=0x7fbee80c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.311+0000 7fbedfe00640 1 --2- 172.21.15.82:0/2368199700 >> v2:172.21.15.67:3300/0 conn(0x7fbee8152380 0x7fbee80c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.311+0000 7fbedfe00640 1 -- 172.21.15.82:0/2368199700 >> v1:172.21.15.82:6789/0 conn(0x7fbee80c9960 legacy=0x7fbee8154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.311+0000 7fbedfe00640 1 -- 172.21.15.82:0/2368199700 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8159010 con 0x7fbee80c54d0 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.312+0000 7fbedfe00640 1 --2- 172.21.15.82:0/2368199700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee80c9420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009870 tx=0x7fbed002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae0cd5d4e1292c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.312+0000 7fbee5400640 1 -- 172.21.15.82:0/2368199700 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00317c0 con 0x7fbee80c54d0 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.312+0000 7fbee5400640 1 -- 172.21.15.82:0/2368199700 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0037070 con 0x7fbee80c54d0 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.312+0000 7fbee5400640 1 -- 172.21.15.82:0/2368199700 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0031b80 con 0x7fbee80c54d0 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.312+0000 7fbee7200640 1 -- 172.21.15.82:0/2368199700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 msgr2=0x7fbee80c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.312+0000 7fbee7200640 1 --2- 172.21.15.82:0/2368199700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee80c9420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009870 tx=0x7fbed002b860 comp rx=0 tx=0).stop 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.313+0000 7fbee7200640 1 -- 172.21.15.82:0/2368199700 shutdown_connections 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.313+0000 7fbee7200640 1 --2- 172.21.15.82:0/2368199700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee80c9420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.313+0000 7fbee7200640 1 --2- 172.21.15.82:0/2368199700 >> v2:172.21.15.67:3300/0 conn(0x7fbee8152380 0x7fbee80c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.313+0000 7fbee7200640 1 -- 172.21.15.82:0/2368199700 >> 172.21.15.82:0/2368199700 conn(0x7fbee80ca610 msgr2=0x7fbee80caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.313+0000 7fbee7200640 1 -- 172.21.15.82:0/2368199700 shutdown_connections 2024-02-26T10:38:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.313+0000 7fbee7200640 1 -- 172.21.15.82:0/2368199700 wait complete. 2024-02-26T10:38:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.314+0000 7fbee7200640 1 Processor -- start 2024-02-26T10:38:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.314+0000 7fbee7200640 1 -- start start 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee814dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee80c9960 0x7fbee814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee8152380 0x7fbee80c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee806dca0 con 0x7fbee80c9960 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee80c8bf0 con 0x7fbee8152380 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8158c70 con 0x7fbee80c54d0 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee814dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee814dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:43538/0 (socket says 172.21.15.82:43538) 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.315+0000 7fbee6800640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee8152380 0x7fbee80c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee5e00640 1 -- 172.21.15.82:0/2734642546 learned_addr learned my addr 172.21.15.82:0/2734642546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbedfe00640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee80c9960 0x7fbee814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee6800640 1 -- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 msgr2=0x7fbee814dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee6800640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee814dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee6800640 1 -- 172.21.15.82:0/2734642546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee80c9960 msgr2=0x7fbee814e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee6800640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee80c9960 0x7fbee814e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee6800640 1 -- 172.21.15.82:0/2734642546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80c43a0 con 0x7fbee8152380 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee5e00640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee814dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbee6800640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee8152380 0x7fbee80c2cb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbee0002230 tx=0x7fbee000cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.316+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0017070 con 0x7fbee8152380 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.317+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee0003b70 con 0x7fbee8152380 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.317+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee00050d0 con 0x7fbee8152380 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.317+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee81eb490 con 0x7fbee8152380 2024-02-26T10:38:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.317+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee8145fa0 con 0x7fbee8152380 2024-02-26T10:38:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.317+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbee001a070 con 0x7fbee8152380 2024-02-26T10:38:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.317+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fbee0012a10 con 0x7fbee8152380 2024-02-26T10:38:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.318+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee8074a30 con 0x7fbee8152380 2024-02-26T10:38:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.321+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbee0021080 con 0x7fbee8152380 2024-02-26T10:38:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.399+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b510a5fe-eeb5-4ae9-807e-95e4e00db1cd", "id": 6} v 0) v1 -- 0x7fbee80c8510 con 0x7fbee8152380 2024-02-26T10:38:18.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.407+0000 7fbede000640 1 -- 172.21.15.82:0/2734642546 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b510a5fe-eeb5-4ae9-807e-95e4e00db1cd", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbee8152760 con 0x7fbee8152380 2024-02-26T10:38:18.410 INFO:teuthology.orchestra.run.smithi082.stdout:6 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee8152380 msgr2=0x7fbee80c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee8152380 0x7fbee80c2cb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbee0002230 tx=0x7fbee000cb90 comp rx=0 tx=0).stop 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 shutdown_connections 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee8152380 0x7fbee80c2cb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbee80c9960 0x7fbee814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 --2- 172.21.15.82:0/2734642546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbee80c54d0 0x7fbee814dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 >> 172.21.15.82:0/2734642546 conn(0x7fbee80ca610 msgr2=0x7fbee806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.408+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 shutdown_connections 2024-02-26T10:38:18.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.409+0000 7fbee7200640 1 -- 172.21.15.82:0/2734642546 wait complete. 2024-02-26T10:38:18.427 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new dcd292f3-26a3-47f5-ba3d-ebb61f72348b 7 2024-02-26T10:38:18.427 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:18.424+0000 7efd9a440040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:38:18.457 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:18.453+0000 7f79c862b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-26T10:38:18.479 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:18.476+0000 7efd9a440040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:38:18.509 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:18.506+0000 7f79c862b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-26T10:38:18.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.642+0000 7f5aa2000640 1 Processor -- start 2024-02-26T10:38:18.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.642+0000 7f5aa2000640 1 -- start start 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a9c0c9de0 0x7f5a9c15f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa2000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c152760 con 0x7f5a9c152380 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c0cb4e0 con 0x7f5a9c0c9de0 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c06b900 con 0x7f5a9c0c9420 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5a9be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa0c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5a9c152380 0x7f5a9c0c4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36992/0 (socket says 172.21.15.82:36992) 2024-02-26T10:38:18.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5a9be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:43548/0 (socket says 172.21.15.82:43548) 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.644+0000 7f5aa0c00640 1 -- 172.21.15.82:0/3396810949 learned_addr learned my addr 172.21.15.82:0/3396810949 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5aa1600640 1 --2- 172.21.15.82:0/3396810949 >> v2:172.21.15.67:3300/0 conn(0x7f5a9c0c9de0 0x7f5a9c15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5a9be00640 1 -- 172.21.15.82:0/3396810949 >> v2:172.21.15.67:3300/0 conn(0x7f5a9c0c9de0 msgr2=0x7f5a9c15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5a9be00640 1 --2- 172.21.15.82:0/3396810949 >> v2:172.21.15.67:3300/0 conn(0x7f5a9c0c9de0 0x7f5a9c15f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5a9be00640 1 -- 172.21.15.82:0/3396810949 >> v1:172.21.15.82:6789/0 conn(0x7f5a9c152380 legacy=0x7f5a9c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5a9be00640 1 -- 172.21.15.82:0/3396810949 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c159010 con 0x7f5a9c0c9420 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5a9b400640 1 -- 172.21.15.82:0/3396810949 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728246618 0 0) 0x7f5a90001110 con 0x7f5a9c152380 2024-02-26T10:38:18.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.645+0000 7f5aa1600640 1 --2- 172.21.15.82:0/3396810949 >> v2:172.21.15.67:3300/0 conn(0x7f5a9c0c9de0 0x7f5a9c15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:18.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.646+0000 7f5a9be00640 1 --2- 172.21.15.82:0/3396810949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0c98a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a880097a0 tx=0x7f5a8802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41e5b08d95f55ac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:18.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.646+0000 7f5a9b400640 1 -- 172.21.15.82:0/3396810949 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88038070 con 0x7f5a9c0c9420 2024-02-26T10:38:18.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.646+0000 7f5a9b400640 1 -- 172.21.15.82:0/3396810949 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a88031700 con 0x7f5a9c0c9420 2024-02-26T10:38:18.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.646+0000 7f5a9b400640 1 -- 172.21.15.82:0/3396810949 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88031ad0 con 0x7f5a9c0c9420 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 -- 172.21.15.82:0/3396810949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 msgr2=0x7f5a9c0c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 --2- 172.21.15.82:0/3396810949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0c98a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a880097a0 tx=0x7f5a8802b7a0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 -- 172.21.15.82:0/3396810949 shutdown_connections 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 --2- 172.21.15.82:0/3396810949 >> v2:172.21.15.67:3300/0 conn(0x7f5a9c0c9de0 0x7f5a9c15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 --2- 172.21.15.82:0/3396810949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0c98a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 -- 172.21.15.82:0/3396810949 >> 172.21.15.82:0/3396810949 conn(0x7f5a9c0ca610 msgr2=0x7f5a9c0caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 -- 172.21.15.82:0/3396810949 shutdown_connections 2024-02-26T10:38:18.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.647+0000 7f5aa2000640 1 -- 172.21.15.82:0/3396810949 wait complete. 2024-02-26T10:38:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.648+0000 7f5aa2000640 1 Processor -- start 2024-02-26T10:38:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 -- start start 2024-02-26T10:38:18.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c0c9de0 0x7f5a9c1ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a9c152380 0x7f5a9c153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:18.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c06dca0 con 0x7f5a9c0c9de0 2024-02-26T10:38:18.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c8bf0 con 0x7f5a9c152380 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.649+0000 7f5aa2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c158c70 con 0x7f5a9c0c9420 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a9c152380 0x7f5a9c153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:43564/0 (socket says 172.21.15.82:43564) 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 -- 172.21.15.82:0/2672767516 learned_addr learned my addr 172.21.15.82:0/2672767516 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5a9be00640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c0c9de0 0x7f5a9c1ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 -- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a9c152380 msgr2=0x7f5a9c153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a9c152380 0x7f5a9c153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 -- 172.21.15.82:0/2672767516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c0c9de0 msgr2=0x7f5a9c1ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c0c9de0 0x7f5a9c1ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 -- 172.21.15.82:0/2672767516 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c154930 con 0x7f5a9c0c9420 2024-02-26T10:38:18.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.650+0000 7f5aa0c00640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0af280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c780 tx=0x7f5a9000cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:18.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90018070 con 0x7f5a9c0c9420 2024-02-26T10:38:18.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a900072d0 con 0x7f5a9c0c9420 2024-02-26T10:38:18.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c14bdc0 con 0x7f5a9c0c9420 2024-02-26T10:38:18.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a900047e0 con 0x7f5a9c0c9420 2024-02-26T10:38:18.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c1eeae0 con 0x7f5a9c0c9420 2024-02-26T10:38:18.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5a9001b070 con 0x7f5a9c0c9420 2024-02-26T10:38:18.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.651+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f5a90012a60 con 0x7f5a9c0c9420 2024-02-26T10:38:18.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.652+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c075020 con 0x7f5a9c0c9420 2024-02-26T10:38:18.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.654+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5a9c075020 con 0x7f5a9c0c9420 2024-02-26T10:38:18.661 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:18.658+0000 7efd9a440040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:38:18.693 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:18.690+0000 7f79c862b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-26T10:38:18.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.732+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dcd292f3-26a3-47f5-ba3d-ebb61f72348b", "id": 7} v 0) v1 -- 0x7f5a9c074a30 con 0x7f5a9c0c9420 2024-02-26T10:38:18.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.740+0000 7f5a98c00640 1 -- 172.21.15.82:0/2672767516 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dcd292f3-26a3-47f5-ba3d-ebb61f72348b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a9c152760 con 0x7f5a9c0c9420 2024-02-26T10:38:18.743 INFO:teuthology.orchestra.run.smithi082.stdout:7 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.741+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 msgr2=0x7f5a9c0af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.741+0000 7f5aa2000640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0af280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c780 tx=0x7f5a9000cc40 comp rx=0 tx=0).stop 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 shutdown_connections 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a9c152380 0x7f5a9c153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c0c9de0 0x7f5a9c1ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 --2- 172.21.15.82:0/2672767516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a9c0c9420 0x7f5a9c0af280 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 >> 172.21.15.82:0/2672767516 conn(0x7f5a9c0ca610 msgr2=0x7f5a9c0aaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 shutdown_connections 2024-02-26T10:38:18.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-02-26T10:38:18.742+0000 7f5aa2000640 1 -- 172.21.15.82:0/2672767516 wait complete. 2024-02-26T10:38:18.762 INFO:tasks.ceph.osd.0:Restarting daemon 2024-02-26T10:38:18.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-02-26T10:38:18.769 INFO:tasks.ceph.osd.0:Started 2024-02-26T10:38:18.769 INFO:tasks.ceph.osd.1:Restarting daemon 2024-02-26T10:38:18.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-02-26T10:38:18.776 INFO:tasks.ceph.osd.1:Started 2024-02-26T10:38:18.776 INFO:tasks.ceph.osd.2:Restarting daemon 2024-02-26T10:38:18.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-02-26T10:38:18.782 INFO:tasks.ceph.osd.2:Started 2024-02-26T10:38:18.782 INFO:tasks.ceph.osd.3:Restarting daemon 2024-02-26T10:38:18.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-02-26T10:38:18.788 INFO:tasks.ceph.osd.3:Started 2024-02-26T10:38:18.788 INFO:tasks.ceph.osd.4:Restarting daemon 2024-02-26T10:38:18.788 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-02-26T10:38:18.795 INFO:tasks.ceph.osd.4:Started 2024-02-26T10:38:18.796 INFO:tasks.ceph.osd.5:Restarting daemon 2024-02-26T10:38:18.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-02-26T10:38:18.802 INFO:tasks.ceph.osd.5:Started 2024-02-26T10:38:18.802 INFO:tasks.ceph.osd.6:Restarting daemon 2024-02-26T10:38:18.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-02-26T10:38:18.809 INFO:tasks.ceph.osd.6:Started 2024-02-26T10:38:18.809 INFO:tasks.ceph.osd.7:Restarting daemon 2024-02-26T10:38:18.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-02-26T10:38:18.815 INFO:tasks.ceph.osd.7:Started 2024-02-26T10:38:18.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:19.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.018+0000 7fccd3e00640 1 Processor -- start 2024-02-26T10:38:19.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.018+0000 7fccd3e00640 1 -- start start 2024-02-26T10:38:19.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.019+0000 7fccd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fccd40c8510 0x7fccd40c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:19.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.019+0000 7fccd3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd4152760 con 0x7fccd4152380 2024-02-26T10:38:19.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.019+0000 7fccd3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd406d9a0 con 0x7fccd40c7020 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.019+0000 7fccd3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd40c7550 con 0x7fccd40c8510 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.020+0000 7fccd2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fccd4152380 0x7fccd4155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48098/0 (socket says 172.21.15.67:48098) 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.020+0000 7fccd2a00640 1 -- 172.21.15.67:0/176367343 learned_addr learned my addr 172.21.15.67:0/176367343 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.020+0000 7fccd3400640 1 --2- 172.21.15.67:0/176367343 >> v2:172.21.15.67:3300/0 conn(0x7fccd40c8510 0x7fccd40c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.020+0000 7fccd3400640 1 -- 172.21.15.67:0/176367343 >> v1:172.21.15.82:6789/0 conn(0x7fccd40c7020 legacy=0x7fccd406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.020+0000 7fccd3400640 1 -- 172.21.15.67:0/176367343 >> v1:172.21.15.67:6789/0 conn(0x7fccd4152380 legacy=0x7fccd4155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:19.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.020+0000 7fccd3400640 1 -- 172.21.15.67:0/176367343 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40c7830 con 0x7fccd40c8510 2024-02-26T10:38:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.021+0000 7fccd3400640 1 --2- 172.21.15.67:0/176367343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c8510 0x7fccd40c4f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009d90 tx=0x7fcccc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e51e24caa7a389b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:19.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.021+0000 7fccd2000640 1 -- 172.21.15.67:0/176367343 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc039070 con 0x7fccd40c8510 2024-02-26T10:38:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.021+0000 7fccd2000640 1 -- 172.21.15.67:0/176367343 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc02cc60 con 0x7fccd40c8510 2024-02-26T10:38:19.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.021+0000 7fccd2000640 1 -- 172.21.15.67:0/176367343 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc034660 con 0x7fccd40c8510 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 -- 172.21.15.67:0/176367343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c8510 msgr2=0x7fccd40c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 --2- 172.21.15.67:0/176367343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c8510 0x7fccd40c4f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009d90 tx=0x7fcccc02beb0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 -- 172.21.15.67:0/176367343 shutdown_connections 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 --2- 172.21.15.67:0/176367343 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c8510 0x7fccd40c4f90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 -- 172.21.15.67:0/176367343 >> 172.21.15.67:0/176367343 conn(0x7fccd40ac1a0 msgr2=0x7fccd40ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 -- 172.21.15.67:0/176367343 shutdown_connections 2024-02-26T10:38:19.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.022+0000 7fccd3e00640 1 -- 172.21.15.67:0/176367343 wait complete. 2024-02-26T10:38:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.024+0000 7fccd3e00640 1 Processor -- start 2024-02-26T10:38:19.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.024+0000 7fccd3e00640 1 -- start start 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 0x7fccd40ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccd40c8510 0x7fccd41477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd4152380 0x7fccd4147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40c54d0 con 0x7fccd40c7020 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd406cd70 con 0x7fccd4152380 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40c5650 con 0x7fccd40c8510 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 0x7fccd40ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 0x7fccd40ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35682/0 (socket says 172.21.15.67:35682) 2024-02-26T10:38:19.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.025+0000 7fccd2a00640 1 -- 172.21.15.67:0/593975974 learned_addr learned my addr 172.21.15.67:0/593975974 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd3400640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd4152380 0x7fccd4147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fcccbe00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccd40c8510 0x7fccd41477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd2a00640 1 -- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccd40c8510 msgr2=0x7fccd41477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd2a00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccd40c8510 0x7fccd41477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd2a00640 1 -- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd4152380 msgr2=0x7fccd4147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd2a00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd4152380 0x7fccd4147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd2a00640 1 -- 172.21.15.67:0/593975974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40c0730 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fccd2a00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 0x7fccd40ab400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c3c0 tx=0x7fccc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4017070 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc4005ce0 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.026+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40072f0 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.027+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd40c0a10 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.027+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd40b0090 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.027+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fccc401a070 con 0x7fccd40c7020 2024-02-26T10:38:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.027+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fccc4012ad0 con 0x7fccd40c7020 2024-02-26T10:38:19.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.028+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd4074a30 con 0x7fccd40c7020 2024-02-26T10:38:19.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.030+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fccc4021020 con 0x7fccd40c7020 2024-02-26T10:38:19.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.096+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccd40c7c50 con 0x7fccd40c7020 2024-02-26T10:38:19.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.097+0000 7fcccb400640 1 -- 172.21.15.67:0/593975974 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fccc40074f0 con 0x7fccd40c7020 2024-02-26T10:38:19.100 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:19.100 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:18.735515+0000","last_up_change":"0.000000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 msgr2=0x7fccd40ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:19.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 0x7fccd40ab400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c3c0 tx=0x7fccc400c880 comp rx=0 tx=0).stop 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 shutdown_connections 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd4152380 0x7fccd4147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccd40c8510 0x7fccd41477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 --2- 172.21.15.67:0/593975974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd40c7020 0x7fccd40ab400 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.098+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 >> 172.21.15.67:0/593975974 conn(0x7fccd40ac1a0 msgr2=0x7fccd41504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.099+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 shutdown_connections 2024-02-26T10:38:19.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:19.099+0000 7fccd3e00640 1 -- 172.21.15.67:0/593975974 wait complete. 2024-02-26T10:38:19.117 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-02-26T10:38:19.117 INFO:tasks.ceph:Waiting for OSDs to come up 2024-02-26T10:38:19.475 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-02-26T10:38:19.471+0000 7fa07a3f8640 -1 Falling back to public interface 2024-02-26T10:38:19.482 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-02-26T10:38:19.479+0000 7fd7ed40d640 -1 Falling back to public interface 2024-02-26T10:38:19.499 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-02-26T10:38:19.497+0000 7fd6b09e1640 -1 Falling back to public interface 2024-02-26T10:38:19.504 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-02-26T10:38:19.501+0000 7ffafe949640 -1 Falling back to public interface 2024-02-26T10:38:19.506 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-02-26T10:38:19.503+0000 7f7d418bd640 -1 Falling back to public interface 2024-02-26T10:38:19.507 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-02-26T10:38:19.504+0000 7f7d40000640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2024-02-26T10:38:19.509 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-02-26T10:38:19.507+0000 7faaae0d1640 -1 Falling back to public interface 2024-02-26T10:38:19.523 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-02-26T10:38:19.520+0000 7f1b42421640 -1 Falling back to public interface 2024-02-26T10:38:19.533 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-02-26T10:38:19.530+0000 7f172f2bd640 -1 Falling back to public interface 2024-02-26T10:38:19.797 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:19.794+0000 7f79c862b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:38:19.848 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:19.846+0000 7efd9a440040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-26T10:38:19.923 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:20.094 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.091+0000 7f8d5f000640 1 Processor -- start 2024-02-26T10:38:20.094 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.091+0000 7f8d5f000640 1 -- start start 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d5f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d5806aef0 0x7f8d5814c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d5f000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8d58153a20 0x7f8d5814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d5f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d580c2a00 con 0x7f8d580c5ba0 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d5f000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d580c6110 con 0x7f8d58153a20 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d5f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580c2ce0 con 0x7f8d5806aef0 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d4fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8d580c5ba0 0x7f8d58153310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53990/0 (socket says 172.21.15.67:53990) 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.092+0000 7f8d4fe00640 1 -- 172.21.15.67:0/2371983562 learned_addr learned my addr 172.21.15.67:0/2371983562 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57e00640 1 --2- 172.21.15.67:0/2371983562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d5806aef0 0x7f8d5814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d5d000640 1 --2- 172.21.15.67:0/2371983562 >> v2:172.21.15.82:3300/0 conn(0x7f8d58153a20 0x7f8d5814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:20.096 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57400640 1 -- 172.21.15.67:0/2371983562 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296383359 0 0) 0x7f8d580c2a00 con 0x7f8d580c5ba0 2024-02-26T10:38:20.097 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57400640 1 -- 172.21.15.67:0/2371983562 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d3c003520 con 0x7f8d580c5ba0 2024-02-26T10:38:20.097 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57e00640 1 -- 172.21.15.67:0/2371983562 >> v2:172.21.15.82:3300/0 conn(0x7f8d58153a20 msgr2=0x7f8d5814cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:20.097 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57e00640 1 --2- 172.21.15.67:0/2371983562 >> v2:172.21.15.82:3300/0 conn(0x7f8d58153a20 0x7f8d5814cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.097 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57e00640 1 -- 172.21.15.67:0/2371983562 >> v1:172.21.15.67:6789/0 conn(0x7f8d580c5ba0 legacy=0x7f8d58153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:20.097 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.093+0000 7f8d57e00640 1 -- 172.21.15.67:0/2371983562 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580c2fc0 con 0x7f8d5806aef0 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.094+0000 7f8d57e00640 1 --2- 172.21.15.67:0/2371983562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d5806aef0 0x7f8d5814c850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d480097a0 tx=0x7f8d4802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f36618cf9a51414 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d57400640 1 -- 172.21.15.67:0/2371983562 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48031780 con 0x7f8d5806aef0 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d57400640 1 -- 172.21.15.67:0/2371983562 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48037070 con 0x7f8d5806aef0 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d57400640 1 -- 172.21.15.67:0/2371983562 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48031b40 con 0x7f8d5806aef0 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 -- 172.21.15.67:0/2371983562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d5806aef0 msgr2=0x7f8d5814c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 --2- 172.21.15.67:0/2371983562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d5806aef0 0x7f8d5814c850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d480097a0 tx=0x7f8d4802b820 comp rx=0 tx=0).stop 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 -- 172.21.15.67:0/2371983562 shutdown_connections 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 --2- 172.21.15.67:0/2371983562 >> v2:172.21.15.82:3300/0 conn(0x7f8d58153a20 0x7f8d5814cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 --2- 172.21.15.67:0/2371983562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d5806aef0 0x7f8d5814c850 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.098 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 -- 172.21.15.67:0/2371983562 >> 172.21.15.67:0/2371983562 conn(0x7f8d580a8580 msgr2=0x7f8d580a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:20.099 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 -- 172.21.15.67:0/2371983562 shutdown_connections 2024-02-26T10:38:20.099 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.095+0000 7f8d5f000640 1 -- 172.21.15.67:0/2371983562 wait complete. 2024-02-26T10:38:20.100 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.098+0000 7f8d5f000640 1 Processor -- start 2024-02-26T10:38:20.100 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.098+0000 7f8d5f000640 1 -- start start 2024-02-26T10:38:20.102 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.099+0000 7f8d5f000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d5806aef0 0x7f8d58151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.099+0000 7f8d5f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d580c5ba0 0x7f8d5806e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 0x7f8d580bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580c0ef0 con 0x7f8d58153a20 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5f000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580c0d70 con 0x7f8d5806aef0 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580730b0 con 0x7f8d580c5ba0 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 0x7f8d580bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d580c5ba0 0x7f8d5806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d57e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d5806aef0 0x7f8d58151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 0x7f8d580bfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35798/0 (socket says 172.21.15.67:35798) 2024-02-26T10:38:20.103 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 -- 172.21.15.67:0/3386927207 learned_addr learned my addr 172.21.15.67:0/3386927207 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 -- 172.21.15.67:0/3386927207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d580c5ba0 msgr2=0x7f8d5806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d580c5ba0 0x7f8d5806e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 -- 172.21.15.67:0/3386927207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d5806aef0 msgr2=0x7f8d58151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d5806aef0 0x7f8d58151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.100+0000 7f8d5d000640 1 -- 172.21.15.67:0/3386927207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5814d060 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d5d000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 0x7f8d580bfc00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000efc0 tx=0x7f8d5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50018020 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d50014480 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50014870 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5814d340 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d581523f0 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8d50005d10 con 0x7f8d58153a20 2024-02-26T10:38:20.105 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.101+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8d50014da0 con 0x7f8d58153a20 2024-02-26T10:38:20.106 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.102+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d580c6590 con 0x7f8d58153a20 2024-02-26T10:38:20.109 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.107+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8d580c6590 con 0x7f8d58153a20 2024-02-26T10:38:20.111 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:20.108+0000 7f79c862b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:38:20.171 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:20.168+0000 7f79c862b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:38:20.178 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.175+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d58074930 con 0x7f8d58153a20 2024-02-26T10:38:20.179 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.176+0000 7f8d54c00640 1 -- 172.21.15.67:0/3386927207 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f8d50022070 con 0x7f8d58153a20 2024-02-26T10:38:20.179 INFO:teuthology.misc.health.smithi067.stdout: 2024-02-26T10:38:20.179 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":10,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:18.735515+0000","last_up_change":"0.000000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 msgr2=0x7f8d580bfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 0x7f8d580bfc00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000efc0 tx=0x7f8d5000c490 comp rx=0 tx=0).stop 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 shutdown_connections 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d58153a20 0x7f8d580bfc00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d580c5ba0 0x7f8d5806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 --2- 172.21.15.67:0/3386927207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d5806aef0 0x7f8d58151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 >> 172.21.15.67:0/3386927207 conn(0x7f8d580a8580 msgr2=0x7f8d580a7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 shutdown_connections 2024-02-26T10:38:20.181 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:20.178+0000 7f8d5f000640 1 -- 172.21.15.67:0/3386927207 wait complete. 2024-02-26T10:38:20.184 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:20.182+0000 7efd9a440040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-26T10:38:20.193 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-02-26T10:38:20.247 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:20.244+0000 7efd9a440040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-26T10:38:20.299 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:20.296+0000 7f79c862b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:38:20.388 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-02-26T10:38:20.385+0000 7efd9a440040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-26T10:38:21.347 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:21.344+0000 7f7968400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-26T10:38:21.348 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-02-26T10:38:21.345+0000 7f7968400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-26T10:38:23.056 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-02-26T10:38:23.054+0000 7fd7ed40d640 -1 osd.6 0 log_to_monitors true 2024-02-26T10:38:23.082 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-02-26T10:38:23.079+0000 7f172f2bd640 -1 osd.7 0 log_to_monitors true 2024-02-26T10:38:23.085 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-02-26T10:38:23.083+0000 7fd6b09e1640 -1 osd.0 0 log_to_monitors true 2024-02-26T10:38:23.090 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-02-26T10:38:23.088+0000 7ffafe949640 -1 osd.3 0 log_to_monitors true 2024-02-26T10:38:23.248 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-02-26T10:38:23.245+0000 7fa07a3f8640 -1 osd.5 0 log_to_monitors true 2024-02-26T10:38:23.319 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-02-26T10:38:23.315+0000 7faaae0d1640 -1 osd.1 0 log_to_monitors true 2024-02-26T10:38:23.323 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-02-26T10:38:23.320+0000 7f1b42421640 -1 osd.4 0 log_to_monitors true 2024-02-26T10:38:23.352 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-02-26T10:38:23.349+0000 7f7d418bd640 -1 osd.2 0 log_to_monitors true 2024-02-26T10:38:26.387 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-02-26T10:38:26.382+0000 7fd6aaa00640 -1 osd.0 0 waiting for initial osdmap 2024-02-26T10:38:26.391 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-02-26T10:38:26.388+0000 7fd6a1400640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.393 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-02-26T10:38:26.391+0000 7faaaa000640 -1 osd.1 0 waiting for initial osdmap 2024-02-26T10:38:26.400 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-02-26T10:38:26.397+0000 7faa9ec00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.414 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-02-26T10:38:26.411+0000 7ffafa800640 -1 osd.3 0 waiting for initial osdmap 2024-02-26T10:38:26.421 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-02-26T10:38:26.418+0000 7ffaef400640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.696 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-02-26T10:38:26.694+0000 7f172b200640 -1 osd.7 0 waiting for initial osdmap 2024-02-26T10:38:26.704 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-02-26T10:38:26.703+0000 7fd7e8c00640 -1 osd.6 0 waiting for initial osdmap 2024-02-26T10:38:26.710 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-02-26T10:38:26.708+0000 7f1b3be00640 -1 osd.4 0 waiting for initial osdmap 2024-02-26T10:38:26.711 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-02-26T10:38:26.709+0000 7f171fe00640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.713 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-02-26T10:38:26.711+0000 7fa074400640 -1 osd.5 0 waiting for initial osdmap 2024-02-26T10:38:26.714 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-02-26T10:38:26.712+0000 7fd7dd800640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.717 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-02-26T10:38:26.715+0000 7f1b32800640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.721 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-02-26T10:38:26.719+0000 7fa06ae00640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:26.848 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-02-26T10:38:26.846+0000 7f7d3d800640 -1 osd.2 0 waiting for initial osdmap 2024-02-26T10:38:26.860 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-02-26T10:38:26.858+0000 7f7d32400640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-26T10:38:27.001 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:27.193 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.190+0000 7ff6fb200640 1 Processor -- start 2024-02-26T10:38:27.193 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.190+0000 7ff6fb200640 1 -- start start 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6fb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff6fc0a9730 0x7ff6fc0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc154f60 0x7ff6fc14eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6fb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6fc0c1640 con 0x7ff6fc0c6570 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6fb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc152180 con 0x7ff6fc0a9730 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6fb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc152460 con 0x7ff6fc154f60 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6f9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff6fc0a9730 0x7ff6fc0c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.195 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6f9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff6fc0a9730 0x7ff6fc0c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35840/0 (socket says 172.21.15.67:35840) 2024-02-26T10:38:27.196 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6f9400640 1 -- 172.21.15.67:0/530846534 learned_addr learned my addr 172.21.15.67:0/530846534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:27.196 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6fa800640 1 --2- 172.21.15.67:0/530846534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc154f60 0x7ff6fc14eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.196 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6f9400640 1 -- 172.21.15.67:0/530846534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc154f60 msgr2=0x7ff6fc14eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:27.196 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.192+0000 7ff6f9400640 1 --2- 172.21.15.67:0/530846534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc154f60 0x7ff6fc14eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.196 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.193+0000 7ff6f9400640 1 -- 172.21.15.67:0/530846534 >> v1:172.21.15.82:6789/0 conn(0x7ff6fc0c6570 legacy=0x7ff6fc151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.196 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.193+0000 7ff6f9400640 1 -- 172.21.15.67:0/530846534 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc152740 con 0x7ff6fc0a9730 2024-02-26T10:38:27.197 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.194+0000 7ff6f9400640 1 --2- 172.21.15.67:0/530846534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc0c1e30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009870 tx=0x7ff6e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1aa72e57417f79 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:27.197 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.194+0000 7ff6f8a00640 1 -- 172.21.15.67:0/530846534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8035020 con 0x7ff6fc0a9730 2024-02-26T10:38:27.197 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.194+0000 7ff6f8a00640 1 -- 172.21.15.67:0/530846534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8031640 con 0x7ff6fc0a9730 2024-02-26T10:38:27.197 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.194+0000 7ff6f8a00640 1 -- 172.21.15.67:0/530846534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e802d040 con 0x7ff6fc0a9730 2024-02-26T10:38:27.197 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 -- 172.21.15.67:0/530846534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 msgr2=0x7ff6fc0c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.197 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 --2- 172.21.15.67:0/530846534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc0c1e30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009870 tx=0x7ff6e802b8f0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.198 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 -- 172.21.15.67:0/530846534 shutdown_connections 2024-02-26T10:38:27.198 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 --2- 172.21.15.67:0/530846534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc154f60 0x7ff6fc14eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.198 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 --2- 172.21.15.67:0/530846534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc0c1e30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.198 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 -- 172.21.15.67:0/530846534 >> 172.21.15.67:0/530846534 conn(0x7ff6fc06cb60 msgr2=0x7ff6fc06cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:27.198 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 -- 172.21.15.67:0/530846534 shutdown_connections 2024-02-26T10:38:27.198 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.195+0000 7ff6fb200640 1 -- 172.21.15.67:0/530846534 wait complete. 2024-02-26T10:38:27.199 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.196+0000 7ff6fb200640 1 Processor -- start 2024-02-26T10:38:27.199 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.196+0000 7ff6fb200640 1 -- start start 2024-02-26T10:38:27.201 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.201 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc0c6570 0x7ff6fc1dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.201 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc154f60 0x7ff6fc1e9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.201 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6fb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0c24f0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.201 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6fb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0c2370 con 0x7ff6fc154f60 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6fb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0c2670 con 0x7ff6fc0c6570 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.197+0000 7ff6f9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35852/0 (socket says 172.21.15.67:35852) 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 -- 172.21.15.67:0/1676566585 learned_addr learned my addr 172.21.15.67:0/1676566585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6fa800640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc154f60 0x7ff6fc1e9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9400640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc0c6570 0x7ff6fc1dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 -- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc0c6570 msgr2=0x7ff6fc1dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc0c6570 0x7ff6fc1dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 -- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc154f60 msgr2=0x7ff6fc1e9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc154f60 0x7ff6fc1e9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 -- 172.21.15.67:0/1676566585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc146ce0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6fa800640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc154f60 0x7ff6fc1e9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.198+0000 7ff6f9e00640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc146fc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c8a0 tx=0x7ff6f400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.199+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4003b70 con 0x7ff6fc0a9730 2024-02-26T10:38:27.202 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.199+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f4003d30 con 0x7ff6fc0a9730 2024-02-26T10:38:27.203 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.199+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400aa90 con 0x7ff6fc0a9730 2024-02-26T10:38:27.203 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.199+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc0710f0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.203 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.199+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc1dde20 con 0x7ff6fc0a9730 2024-02-26T10:38:27.203 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.200+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc04efc0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.203 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.201+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7ff6f400ac30 con 0x7ff6fc0a9730 2024-02-26T10:38:27.204 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.202+0000 7ff6f2000640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff6dc06b440 0x7ff6dc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.204 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.202+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2231+0+0 (secure 0 0 0) 0x7ff6f4089300 con 0x7ff6fc0a9730 2024-02-26T10:38:27.205 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.202+0000 7ff6f9400640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff6dc06b440 0x7ff6dc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.206 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.204+0000 7ff6f9400640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff6dc06b440 0x7ff6dc06d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:27.209 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.207+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6f4058fe0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.405 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.403+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6fc1515c0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.416 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.413+0000 7ff6f2000640 1 -- 172.21.15.67:0/1676566585 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10837 (secure 0 0 0) 0x7ff6f40588e0 con 0x7ff6fc0a9730 2024-02-26T10:38:27.416 INFO:teuthology.misc.health.smithi067.stdout: 2024-02-26T10:38:27.417 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":14,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:27.389640+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.417+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff6dc06b440 msgr2=0x7ff6dc06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.417+0000 7ff6fb200640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff6dc06b440 0x7ff6dc06d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e8008030 comp rx=0 tx=0).stop 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 msgr2=0x7ff6fc146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc146fc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c8a0 tx=0x7ff6f400cd60 comp rx=0 tx=0).stop 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 shutdown_connections 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff6dc06b440 0x7ff6dc06d900 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036000 tx=0x7ff6e8008030 comp rx=0 tx=0).stop 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc154f60 0x7ff6fc1e9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6fc0c6570 0x7ff6fc1dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 --2- 172.21.15.67:0/1676566585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6fc0a9730 0x7ff6fc146fc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.421 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 >> 172.21.15.67:0/1676566585 conn(0x7ff6fc06cb60 msgr2=0x7ff6fc0c3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:27.422 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.418+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 shutdown_connections 2024-02-26T10:38:27.422 INFO:teuthology.misc.health.smithi067.stderr:2024-02-26T10:38:27.419+0000 7ff6fb200640 1 -- 172.21.15.67:0/1676566585 wait complete. 2024-02-26T10:38:27.436 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-02-26T10:38:27.436 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-02-26T10:38:27.437 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-26T10:38:27.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-26T10:38:27.441 INFO:tasks.ceph.mds.a:Started 2024-02-26T10:38:27.441 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-26T10:38:27.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-26T10:38:27.445 INFO:tasks.ceph.mds.c:Started 2024-02-26T10:38:27.445 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-26T10:38:27.445 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-26T10:38:27.450 INFO:tasks.ceph.mds.b:Started 2024-02-26T10:38:27.450 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-26T10:38:27.450 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-26T10:38:27.455 INFO:tasks.ceph.mds.d:Started 2024-02-26T10:38:27.457 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-02-26T10:38:27.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:27.583 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-02-26T10:38:27.587 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:38:27.599 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-02-26T10:38:27.602 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-02-26T10:38:27.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.636+0000 7fb317200640 1 Processor -- start 2024-02-26T10:38:27.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.636+0000 7fb317200640 1 -- start start 2024-02-26T10:38:27.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb317200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3180c8510 0x7fb3180c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb317200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb318152760 con 0x7fb318152380 2024-02-26T10:38:27.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb317200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31806d9a0 con 0x7fb3180c7020 2024-02-26T10:38:27.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb317200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c7550 con 0x7fb3180c8510 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb315e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb318152380 0x7fb318155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54006/0 (socket says 172.21.15.67:54006) 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb315e00640 1 -- 172.21.15.67:0/3793926776 learned_addr learned my addr 172.21.15.67:0/3793926776 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.637+0000 7fb316800640 1 --2- 172.21.15.67:0/3793926776 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3180c8510 0x7fb3180c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.638+0000 7fb314a00640 1 -- 172.21.15.67:0/3793926776 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779564150 0 0) 0x7fb31806d9a0 con 0x7fb3180c7020 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.638+0000 7fb314a00640 1 -- 172.21.15.67:0/3793926776 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc003520 con 0x7fb3180c7020 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.638+0000 7fb314a00640 1 -- 172.21.15.67:0/3793926776 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180928007 0 0) 0x7fb318152760 con 0x7fb318152380 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.638+0000 7fb316800640 1 -- 172.21.15.67:0/3793926776 >> v1:172.21.15.82:6789/0 conn(0x7fb3180c7020 legacy=0x7fb31806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.638+0000 7fb316800640 1 -- 172.21.15.67:0/3793926776 >> v1:172.21.15.67:6789/0 conn(0x7fb318152380 legacy=0x7fb318155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.638+0000 7fb316800640 1 -- 172.21.15.67:0/3793926776 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180c7830 con 0x7fb3180c8510 2024-02-26T10:38:27.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.639+0000 7fb316800640 1 --2- 172.21.15.67:0/3793926776 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3180c8510 0x7fb3180c4f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb310009dc0 tx=0x7fb31002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de780703ba6295a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:27.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.639+0000 7fb314a00640 1 -- 172.21.15.67:0/3793926776 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310039070 con 0x7fb3180c8510 2024-02-26T10:38:27.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.639+0000 7fb314a00640 1 -- 172.21.15.67:0/3793926776 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31002cc40 con 0x7fb3180c8510 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.639+0000 7fb314a00640 1 -- 172.21.15.67:0/3793926776 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310034440 con 0x7fb3180c8510 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 -- 172.21.15.67:0/3793926776 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3180c8510 msgr2=0x7fb3180c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 --2- 172.21.15.67:0/3793926776 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3180c8510 0x7fb3180c4f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb310009dc0 tx=0x7fb31002be90 comp rx=0 tx=0).stop 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 -- 172.21.15.67:0/3793926776 shutdown_connections 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 --2- 172.21.15.67:0/3793926776 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3180c8510 0x7fb3180c4f90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 -- 172.21.15.67:0/3793926776 >> 172.21.15.67:0/3793926776 conn(0x7fb3180ac1a0 msgr2=0x7fb3180ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 -- 172.21.15.67:0/3793926776 shutdown_connections 2024-02-26T10:38:27.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.640+0000 7fb317200640 1 -- 172.21.15.67:0/3793926776 wait complete. 2024-02-26T10:38:27.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.641+0000 7fb317200640 1 Processor -- start 2024-02-26T10:38:27.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.641+0000 7fb317200640 1 -- start start 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.642+0000 7fb317200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180c7020 0x7fb3180c1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.642+0000 7fb317200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 0x7fb3181ee530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.642+0000 7fb317200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318152380 0x7fb3181543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.642+0000 7fb317200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c54d0 con 0x7fb3180c8510 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.642+0000 7fb317200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31806cd70 con 0x7fb3180c7020 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.642+0000 7fb317200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180c5650 con 0x7fb318152380 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 0x7fb3181ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 0x7fb3181ee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35888/0 (socket says 172.21.15.67:35888) 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 -- 172.21.15.67:0/3548040515 learned_addr learned my addr 172.21.15.67:0/3548040515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb316800640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318152380 0x7fb3181543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315e00640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180c7020 0x7fb3180c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 -- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318152380 msgr2=0x7fb3181543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318152380 0x7fb3181543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 -- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180c7020 msgr2=0x7fb3180c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180c7020 0x7fb3180c1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 -- 172.21.15.67:0/3548040515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3181eea70 con 0x7fb3180c8510 2024-02-26T10:38:27.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb316800640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318152380 0x7fb3181543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:27.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315e00640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180c7020 0x7fb3180c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:27.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.643+0000 7fb315400640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 0x7fb3181ee530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c4c0 tx=0x7fb30400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.644+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3040052d0 con 0x7fb3180c8510 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.644+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb304005470 con 0x7fb3180c8510 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.644+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304012870 con 0x7fb3180c8510 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.644+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3181579c0 con 0x7fb3180c8510 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.644+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb318157f80 con 0x7fb3180c8510 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.645+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb318074a30 con 0x7fb3180c8510 2024-02-26T10:38:27.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.646+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fb304012a10 con 0x7fb3180c8510 2024-02-26T10:38:27.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.647+0000 7fb30e000640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2f806b4c0 0x7fb2f806d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:27.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.647+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb304089700 con 0x7fb3180c8510 2024-02-26T10:38:27.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.647+0000 7fb315e00640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2f806b4c0 0x7fb2f806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:27.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.648+0000 7fb315e00640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2f806b4c0 0x7fb2f806d980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb308000f60 tx=0x7fb308007e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:27.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.651+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb304058550 con 0x7fb3180c8510 2024-02-26T10:38:27.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.846+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3180c7c50 con 0x7fb3180c8510 2024-02-26T10:38:27.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.848+0000 7fb30e000640 1 -- 172.21.15.67:0/3548040515 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10837 (secure 0 0 0) 0x7fb30405c5f0 con 0x7fb3180c8510 2024-02-26T10:38:27.851 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:27.851 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:27.389640+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:27.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.850+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2f806b4c0 msgr2=0x7fb2f806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.850+0000 7fb317200640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2f806b4c0 0x7fb2f806d980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb308000f60 tx=0x7fb308007e40 comp rx=0 tx=0).stop 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 msgr2=0x7fb3181ee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 0x7fb3181ee530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c4c0 tx=0x7fb30400c980 comp rx=0 tx=0).stop 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 shutdown_connections 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318152380 0x7fb3181543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3180c8510 0x7fb3181ee530 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2f806b4c0 0x7fb2f806d980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 --2- 172.21.15.67:0/3548040515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180c7020 0x7fb3180c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 >> 172.21.15.67:0/3548040515 conn(0x7fb3180ac1a0 msgr2=0x7fb318154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 shutdown_connections 2024-02-26T10:38:27.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:27.851+0000 7fb317200640 1 -- 172.21.15.67:0/3548040515 wait complete. 2024-02-26T10:38:27.870 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-02-26T10:38:27.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:38:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.086+0000 7f8568800640 1 Processor -- start 2024-02-26T10:38:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.086+0000 7f8568800640 1 -- start start 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8568800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f856014f3d0 0x7f856006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8568800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856006ed10 con 0x7f8560071700 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8568800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8560155c00 con 0x7f85600c2870 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8568800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856006dd80 con 0x7f856014f3d0 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8565e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f856014f3d0 0x7f856006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8565e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f856014f3d0 0x7f856006e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35892/0 (socket says 172.21.15.67:35892) 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8565e00640 1 -- 172.21.15.67:0/4046157176 learned_addr learned my addr 172.21.15.67:0/4046157176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.088+0000 7f8566800640 1 --1- 172.21.15.67:0/4046157176 >> v1:172.21.15.67:6789/0 conn(0x7f8560071700 0x7f856015c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54022/0 (socket says 172.21.15.67:54022) 2024-02-26T10:38:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.089+0000 7f8565e00640 1 -- 172.21.15.67:0/4046157176 >> v1:172.21.15.82:6789/0 conn(0x7f85600c2870 legacy=0x7f8560155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.089+0000 7f8565e00640 1 -- 172.21.15.67:0/4046157176 >> v1:172.21.15.67:6789/0 conn(0x7f8560071700 legacy=0x7f856015c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.089+0000 7f8565e00640 1 -- 172.21.15.67:0/4046157176 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560155ee0 con 0x7f856014f3d0 2024-02-26T10:38:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.089+0000 7f8564a00640 1 -- 172.21.15.67:0/4046157176 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753273992 0 0) 0x7f856006ed10 con 0x7f8560071700 2024-02-26T10:38:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.090+0000 7f8565e00640 1 --2- 172.21.15.67:0/4046157176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f856014f3d0 0x7f856006e7d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f854c009870 tx=0x7f854c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8d61fa663c464 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:28.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.090+0000 7f8564a00640 1 -- 172.21.15.67:0/4046157176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c035020 con 0x7f856014f3d0 2024-02-26T10:38:28.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.090+0000 7f8564a00640 1 -- 172.21.15.67:0/4046157176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c031870 con 0x7f856014f3d0 2024-02-26T10:38:28.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.090+0000 7f8564a00640 1 -- 172.21.15.67:0/4046157176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c031c40 con 0x7f856014f3d0 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 -- 172.21.15.67:0/4046157176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f856014f3d0 msgr2=0x7f856006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 --2- 172.21.15.67:0/4046157176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f856014f3d0 0x7f856006e7d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f854c009870 tx=0x7f854c02b910 comp rx=0 tx=0).stop 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 -- 172.21.15.67:0/4046157176 shutdown_connections 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 --2- 172.21.15.67:0/4046157176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f856014f3d0 0x7f856006e7d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 -- 172.21.15.67:0/4046157176 >> 172.21.15.67:0/4046157176 conn(0x7f85600a84a0 msgr2=0x7f85600a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 -- 172.21.15.67:0/4046157176 shutdown_connections 2024-02-26T10:38:28.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.091+0000 7f8568800640 1 -- 172.21.15.67:0/4046157176 wait complete. 2024-02-26T10:38:28.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.092+0000 7f8568800640 1 Processor -- start 2024-02-26T10:38:28.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.093+0000 7f8568800640 1 -- start start 2024-02-26T10:38:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.093+0000 7f8568800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8560071700 0x7f85601eba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:28.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8568800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 0x7f8560145c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8568800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f856014f3d0 0x7f8560146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8568800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600c1810 con 0x7f85600c2870 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8568800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856006e2b0 con 0x7f8560071700 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8568800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8560155870 con 0x7f856014f3d0 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 0x7f8560145c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 0x7f8560145c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35904/0 (socket says 172.21.15.67:35904) 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 -- 172.21.15.67:0/4150021935 learned_addr learned my addr 172.21.15.67:0/4150021935 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565e00640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8560071700 0x7f85601eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8566800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f856014f3d0 0x7f8560146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 -- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f856014f3d0 msgr2=0x7f8560146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f856014f3d0 0x7f8560146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 -- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8560071700 msgr2=0x7f85601eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:28.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8560071700 0x7f85601eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565400640 1 -- 172.21.15.67:0/4150021935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85601ec4c0 con 0x7f85600c2870 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.094+0000 7f8565e00640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8560071700 0x7f85601eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.095+0000 7f8565400640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 0x7f8560145c30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f855400b3f0 tx=0x7f855400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.095+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554015020 con 0x7f85600c2870 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.095+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8554011830 con 0x7f85600c2870 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.095+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554011c20 con 0x7f85600c2870 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.095+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8560145430 con 0x7f85600c2870 2024-02-26T10:38:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.095+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8560144d90 con 0x7f85600c2870 2024-02-26T10:38:28.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.096+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8560145220 con 0x7f85600c2870 2024-02-26T10:38:28.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.097+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f8554018020 con 0x7f85600c2870 2024-02-26T10:38:28.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.098+0000 7f855a000640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f854006b440 0x7f854006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:28.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.098+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f8554089850 con 0x7f85600c2870 2024-02-26T10:38:28.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.099+0000 7f8565e00640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f854006b440 0x7f854006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:28.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.100+0000 7f8565e00640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f854006b440 0x7f854006d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f854c036000 tx=0x7f854c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:28.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.103+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f855401f030 con 0x7f85600c2870 2024-02-26T10:38:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.339+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8560074a30 con 0x7f85600c2870 2024-02-26T10:38:28.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.394+0000 7f855a000640 1 -- 172.21.15.67:0/4150021935 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f85540172c0 con 0x7f85600c2870 2024-02-26T10:38:28.398 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:28.398 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-26T10:38:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2024-02-26T10:38:28.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f854006b440 msgr2=0x7f854006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:28.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f854006b440 0x7f854006d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f854c036000 tx=0x7f854c03c040 comp rx=0 tx=0).stop 2024-02-26T10:38:28.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 msgr2=0x7f8560145c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 0x7f8560145c30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f855400b3f0 tx=0x7f855400b8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 shutdown_connections 2024-02-26T10:38:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f856014f3d0 0x7f8560146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85600c2870 0x7f8560145c30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f854006b440 0x7f854006d900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 --2- 172.21.15.67:0/4150021935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8560071700 0x7f85601eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 >> 172.21.15.67:0/4150021935 conn(0x7f85600a84a0 msgr2=0x7f85600a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 shutdown_connections 2024-02-26T10:38:28.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:28.398+0000 7f8568800640 1 -- 172.21.15.67:0/4150021935 wait complete. 2024-02-26T10:38:29.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:38:29.716 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:38:30.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:38:30.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.613+0000 7f48a6600640 1 Processor -- start 2024-02-26T10:38:30.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.613+0000 7f48a6600640 1 -- start start 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f48a6600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48a00bdef0 0x7f48a0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f48a6600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 0x7f48a0155840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f48a6600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a0154f50 con 0x7f48a0157d30 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f48a6600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a00c7440 con 0x7f48a00bdef0 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f48a6600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00c7720 con 0x7f48a0155400 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f489f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f48a0157d30 0x7f48a00c51a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40538/0 (socket says 172.21.15.67:40538) 2024-02-26T10:38:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.614+0000 7f489f400640 1 -- 172.21.15.67:0/1725824228 learned_addr learned my addr 172.21.15.67:0/1725824228 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489fe00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 0x7f48a0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489ea00640 1 --2- 172.21.15.67:0/1725824228 >> v2:172.21.15.82:3300/0 conn(0x7f48a00bdef0 0x7f48a0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816339605 0 0) 0x7f48a0154f50 con 0x7f48a0157d30 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4884003520 con 0x7f48a0157d30 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 378280934 0 0) 0x7f4884003520 con 0x7f48a0157d30 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 msgr2=0x7f48a0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 0x7f48a0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 >> v2:172.21.15.82:3300/0 conn(0x7f48a00bdef0 msgr2=0x7f48a0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 --2- 172.21.15.67:0/1725824228 >> v2:172.21.15.82:3300/0 conn(0x7f48a00bdef0 0x7f48a0158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00ad2a0 con 0x7f48a0157d30 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f4890001670 con 0x7f48a0157d30 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 >> v1:172.21.15.67:6789/0 conn(0x7f48a0157d30 legacy=0x7f48a00c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 0x7f48840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884003f80 0x7f4884004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884004d40 0x7f48840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00c7440 con 0x7f4884004d40 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00c7720 con 0x7f4884003f80 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.615+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488400ba70 con 0x7f48a0155400 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489fe00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884004d40 0x7f48840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489ea00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884003f80 0x7f4884004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489f400640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 0x7f48840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489fe00640 1 -- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 msgr2=0x7f48840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489fe00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a0155400 0x7f48840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489fe00640 1 -- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884003f80 msgr2=0x7f4884004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:30.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489fe00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884003f80 0x7f4884004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489fe00640 1 -- 172.21.15.67:0/1725824228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884003520 con 0x7f4884004d40 2024-02-26T10:38:30.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.616+0000 7f489ea00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884003f80 0x7f4884004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.617+0000 7f489fe00640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884004d40 0x7f48840051c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4898009fc0 tx=0x7f4884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0b4b87a2ec34cc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898039070 con 0x7f4884004d40 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489802cc20 con 0x7f4884004d40 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f489e000640 1 -- 172.21.15.67:0/1725824228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898009c10 con 0x7f4884004d40 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f48a6600640 1 -- 172.21.15.67:0/1725824228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884004d40 msgr2=0x7f48840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f48a6600640 1 --2- 172.21.15.67:0/1725824228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4884004d40 0x7f48840051c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4898009fc0 tx=0x7f4884005830 comp rx=0 tx=0).stop 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f489fe00640 1 -- 172.21.15.67:0/1725824228 reap_dead start 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f48a6600640 1 -- 172.21.15.67:0/1725824228 shutdown_connections 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.618+0000 7f48a6600640 1 -- 172.21.15.67:0/1725824228 >> 172.21.15.67:0/1725824228 conn(0x7f48a0072980 msgr2=0x7f48a00ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.619+0000 7f48a6600640 1 -- 172.21.15.67:0/1725824228 shutdown_connections 2024-02-26T10:38:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.619+0000 7f48a6600640 1 -- 172.21.15.67:0/1725824228 wait complete. 2024-02-26T10:38:30.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.621+0000 7f48a6600640 1 Processor -- start 2024-02-26T10:38:30.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.621+0000 7f48a6600640 1 -- start start 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f48a6600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884004d40 0x7f48a00c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f48a6600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a00c3980 0x7f48a00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f48a6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 0x7f48a01eb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f48a6600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0156330 con 0x7f48a00c3e40 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f48a6600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00c7a00 con 0x7f4884004d40 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f48a6600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00c58b0 con 0x7f48a00c3980 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f489fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 0x7f48a01eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f489fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 0x7f48a01eb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37500/0 (socket says 172.21.15.67:37500) 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f489fe00640 1 -- 172.21.15.67:0/1107789444 learned_addr learned my addr 172.21.15.67:0/1107789444 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f489f400640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884004d40 0x7f48a00c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.622+0000 7f489ea00640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a00c3980 0x7f48a00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489fe00640 1 -- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a00c3980 msgr2=0x7f48a00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489fe00640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a00c3980 0x7f48a00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489fe00640 1 -- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884004d40 msgr2=0x7f48a00c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489fe00640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884004d40 0x7f48a00c18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489fe00640 1 -- 172.21.15.67:0/1107789444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00c3000 con 0x7f48a00c3e40 2024-02-26T10:38:30.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489f400640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884004d40 0x7f48a00c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.623+0000 7f489fe00640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 0x7f48a01eb320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4898002230 tx=0x7f4898009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.624+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898039030 con 0x7f48a00c3e40 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.624+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898043070 con 0x7f48a00c3e40 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.624+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489803d7f0 con 0x7f48a00c3e40 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.624+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a00c32e0 con 0x7f48a00c3e40 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.624+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a00af2a0 con 0x7f48a00c3e40 2024-02-26T10:38:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.625+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a01518b0 con 0x7f48a00c3e40 2024-02-26T10:38:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.626+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f4898005d10 con 0x7f48a00c3e40 2024-02-26T10:38:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.627+0000 7f4897400640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f488006b440 0x7f488006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.627+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f48980abff0 con 0x7f48a00c3e40 2024-02-26T10:38:30.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.628+0000 7f489f400640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f488006b440 0x7f488006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:30.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.629+0000 7f489f400640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f488006b440 0x7f488006d900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4890001050 tx=0x7f4890009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.633+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48a01518b0 con 0x7f48a00c3e40 2024-02-26T10:38:30.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.874+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a0074a30 con 0x7f48a00c3e40 2024-02-26T10:38:30.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.875+0000 7f4897400640 1 -- 172.21.15.67:0/1107789444 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f489807b2f0 con 0x7f48a00c3e40 2024-02-26T10:38:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:30.877 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-26T10:38:30.878 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-02-26T10:38:30.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f488006b440 msgr2=0x7f488006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f488006b440 0x7f488006d900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4890001050 tx=0x7f4890009040 comp rx=0 tx=0).stop 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 msgr2=0x7f48a01eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 0x7f48a01eb320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4898002230 tx=0x7f4898009750 comp rx=0 tx=0).stop 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 shutdown_connections 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48a00c3e40 0x7f48a01eb320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48a00c3980 0x7f48a00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f488006b440 0x7f488006d900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 --2- 172.21.15.67:0/1107789444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4884004d40 0x7f48a00c18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 >> 172.21.15.67:0/1107789444 conn(0x7f48a0072980 msgr2=0x7f48a01509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 shutdown_connections 2024-02-26T10:38:30.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:30.878+0000 7f48a6600640 1 -- 172.21.15.67:0/1107789444 wait complete. 2024-02-26T10:38:30.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:31.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.091+0000 7fa8ed600640 1 Processor -- start 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.091+0000 7fa8ed600640 1 -- start start 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.092+0000 7fa8ed600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 0x7fa8e8070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.092+0000 7fa8ed600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e8071390 0x7fa8e8152390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.092+0000 7fa8ed600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e80c4510 con 0x7fa8e81528d0 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.092+0000 7fa8ed600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80c59a0 con 0x7fa8e80c4f00 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.092+0000 7fa8ed600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80c3990 con 0x7fa8e8071390 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8e6a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 0x7fa8e8070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8de000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e8071390 0x7fa8e8152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8e6a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 0x7fa8e8070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34132/0 (socket says 172.21.15.67:34132) 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8e6a00640 1 -- 172.21.15.67:0/1361984668 learned_addr learned my addr 172.21.15.67:0/1361984668 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8de000640 1 -- 172.21.15.67:0/1361984668 >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 msgr2=0x7fa8e8070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8de000640 1 --2- 172.21.15.67:0/1361984668 >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 0x7fa8e8070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8de000640 1 -- 172.21.15.67:0/1361984668 >> v1:172.21.15.82:6789/0 conn(0x7fa8e81528d0 legacy=0x7fa8e8155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8de000640 1 -- 172.21.15.67:0/1361984668 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80c3c70 con 0x7fa8e8071390 2024-02-26T10:38:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.093+0000 7fa8e6a00640 1 --2- 172.21.15.67:0/1361984668 >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 0x7fa8e8070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.095+0000 7fa8de000640 1 --2- 172.21.15.67:0/1361984668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e8071390 0x7fa8e8152390 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00097a0 tx=0x7fa8d002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b238043a2e64d8f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.095+0000 7fa8e6000640 1 -- 172.21.15.67:0/1361984668 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0035020 con 0x7fa8e8071390 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.095+0000 7fa8e6000640 1 -- 172.21.15.67:0/1361984668 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d0031810 con 0x7fa8e8071390 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.095+0000 7fa8e6000640 1 -- 172.21.15.67:0/1361984668 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0031be0 con 0x7fa8e8071390 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 -- 172.21.15.67:0/1361984668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e8071390 msgr2=0x7fa8e8152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1361984668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e8071390 0x7fa8e8152390 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00097a0 tx=0x7fa8d002b8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 -- 172.21.15.67:0/1361984668 shutdown_connections 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1361984668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e8071390 0x7fa8e8152390 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1361984668 >> v2:172.21.15.82:3300/0 conn(0x7fa8e80c4f00 0x7fa8e8070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 -- 172.21.15.67:0/1361984668 >> 172.21.15.67:0/1361984668 conn(0x7fa8e80c8620 msgr2=0x7fa8e80c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 -- 172.21.15.67:0/1361984668 shutdown_connections 2024-02-26T10:38:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.096+0000 7fa8ed600640 1 -- 172.21.15.67:0/1361984668 wait complete. 2024-02-26T10:38:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.099+0000 7fa8ed600640 1 Processor -- start 2024-02-26T10:38:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.099+0000 7fa8ed600640 1 -- start start 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.100+0000 7fa8ed600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 0x7fa8e8154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.100+0000 7fa8ed600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8e80c4f00 0x7fa8e81eb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.100+0000 7fa8ed600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e81528d0 0x7fa8e80adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.100+0000 7fa8ed600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80c52e0 con 0x7fa8e8071390 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.100+0000 7fa8ed600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80c4230 con 0x7fa8e80c4f00 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.100+0000 7fa8ed600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80c5460 con 0x7fa8e81528d0 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 0x7fa8e8154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 0x7fa8e8154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37512/0 (socket says 172.21.15.67:37512) 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 -- 172.21.15.67:0/1936998204 learned_addr learned my addr 172.21.15.67:0/1936998204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8de000640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8e80c4f00 0x7fa8e81eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e7400640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e81528d0 0x7fa8e80adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 -- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e81528d0 msgr2=0x7fa8e80adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e81528d0 0x7fa8e80adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 -- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8e80c4f00 msgr2=0x7fa8e81eb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8e80c4f00 0x7fa8e81eb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e6a00640 1 -- 172.21.15.67:0/1936998204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80aabd0 con 0x7fa8e8071390 2024-02-26T10:38:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8de000640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8e80c4f00 0x7fa8e81eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.101+0000 7fa8e7400640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e81528d0 0x7fa8e80adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.102+0000 7fa8e6a00640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 0x7fa8e8154760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800efc0 tx=0x7fa8d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.102+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8014480 con 0x7fa8e8071390 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.102+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8019070 con 0x7fa8e8071390 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.102+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e80aaeb0 con 0x7fa8e8071390 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.102+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8014820 con 0x7fa8e8071390 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.102+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e8154ca0 con 0x7fa8e8071390 2024-02-26T10:38:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.104+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e81eab50 con 0x7fa8e8071390 2024-02-26T10:38:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.105+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fa8d8005d10 con 0x7fa8e8071390 2024-02-26T10:38:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.106+0000 7fa8df400640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8c406b440 0x7fa8c406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.107+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8d808a360 con 0x7fa8e8071390 2024-02-26T10:38:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.107+0000 7fa8de000640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8c406b440 0x7fa8c406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.108+0000 7fa8de000640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8c406b440 0x7fa8c406d900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0036000 tx=0x7fa8d0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.111+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8d8010040 con 0x7fa8e8071390 2024-02-26T10:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.314+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8e8074a30 con 0x7fa8e8071390 2024-02-26T10:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.315+0000 7fa8df400640 1 -- 172.21.15.67:0/1936998204 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12812 (secure 0 0 0) 0x7fa8d8059600 con 0x7fa8e8071390 2024-02-26T10:38:31.318 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:31.318 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:30.621060+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8c406b440 msgr2=0x7fa8c406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8c406b440 0x7fa8c406d900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0036000 tx=0x7fa8d0008030 comp rx=0 tx=0).stop 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 msgr2=0x7fa8e8154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 0x7fa8e8154760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800efc0 tx=0x7fa8d800c490 comp rx=0 tx=0).stop 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 shutdown_connections 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8e81528d0 0x7fa8e80adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8c406b440 0x7fa8c406d900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8e80c4f00 0x7fa8e81eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 --2- 172.21.15.67:0/1936998204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8e8071390 0x7fa8e8154760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 >> 172.21.15.67:0/1936998204 conn(0x7fa8e80c8620 msgr2=0x7fa8e8150d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 shutdown_connections 2024-02-26T10:38:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.318+0000 7fa8ed600640 1 -- 172.21.15.67:0/1936998204 wait complete. 2024-02-26T10:38:31.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:38:31.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:38:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.538+0000 7fd89de00640 1 Processor -- start 2024-02-26T10:38:31.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.538+0000 7fd89de00640 1 -- start start 2024-02-26T10:38:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd89de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8980c4630 0x7fd8980c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd89de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8980c0b40 0x7fd898154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd89de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8980c3b00 con 0x7fd898071ff0 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd89de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8980c38d0 con 0x7fd8980c4630 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd89de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd898155b30 con 0x7fd8980c0b40 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8980c4630 0x7fd8980c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8980c4630 0x7fd8980c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37528/0 (socket says 172.21.15.67:37528) 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 -- 172.21.15.67:0/2372175997 learned_addr learned my addr 172.21.15.67:0/2372175997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd88fe00640 1 --2- 172.21.15.67:0/2372175997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8980c0b40 0x7fd898154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 -- 172.21.15.67:0/2372175997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8980c0b40 msgr2=0x7fd898154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 --2- 172.21.15.67:0/2372175997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8980c0b40 0x7fd898154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 -- 172.21.15.67:0/2372175997 >> v1:172.21.15.82:6789/0 conn(0x7fd898071ff0 legacy=0x7fd898155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:31.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.540+0000 7fd896a00640 1 -- 172.21.15.67:0/2372175997 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8980c3de0 con 0x7fd8980c4630 2024-02-26T10:38:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.541+0000 7fd896a00640 1 --2- 172.21.15.67:0/2372175997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c4a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd888009870 tx=0x7fd88802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62681345b43054d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.541+0000 7fd896000640 1 -- 172.21.15.67:0/2372175997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888035020 con 0x7fd8980c4630 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.541+0000 7fd896000640 1 -- 172.21.15.67:0/2372175997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888031640 con 0x7fd8980c4630 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.541+0000 7fd896000640 1 -- 172.21.15.67:0/2372175997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88802d040 con 0x7fd8980c4630 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 -- 172.21.15.67:0/2372175997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 msgr2=0x7fd8980c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 --2- 172.21.15.67:0/2372175997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c4a10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd888009870 tx=0x7fd88802b8f0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 -- 172.21.15.67:0/2372175997 shutdown_connections 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 --2- 172.21.15.67:0/2372175997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8980c0b40 0x7fd898154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 --2- 172.21.15.67:0/2372175997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c4a10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 -- 172.21.15.67:0/2372175997 >> 172.21.15.67:0/2372175997 conn(0x7fd8980c7b00 msgr2=0x7fd8980c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 -- 172.21.15.67:0/2372175997 shutdown_connections 2024-02-26T10:38:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.542+0000 7fd89de00640 1 -- 172.21.15.67:0/2372175997 wait complete. 2024-02-26T10:38:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.544+0000 7fd89de00640 1 Processor -- start 2024-02-26T10:38:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.544+0000 7fd89de00640 1 -- start start 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd89de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd898071ff0 0x7fd898154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd89de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8980c0b40 0x7fd8980c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd89de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd89de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd898072490 con 0x7fd8980c4630 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd89de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8980c2340 con 0x7fd8980c0b40 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd89de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd898152770 con 0x7fd898071ff0 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37532/0 (socket says 172.21.15.67:37532) 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 -- 172.21.15.67:0/536212094 learned_addr learned my addr 172.21.15.67:0/536212094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd88fe00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8980c0b40 0x7fd8980c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd896a00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd898071ff0 0x7fd898154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 -- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd898071ff0 msgr2=0x7fd898154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd898071ff0 0x7fd898154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 -- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8980c0b40 msgr2=0x7fd8980c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8980c0b40 0x7fd8980c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd897400640 1 -- 172.21.15.67:0/536212094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8981eb580 con 0x7fd8980c4630 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.545+0000 7fd896a00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd898071ff0 0x7fd898154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.546+0000 7fd897400640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c5840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd89000ba10 tx=0x7fd89000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.546+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890019070 con 0x7fd8980c4630 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.546+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89000cb40 con 0x7fd8980c4630 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.546+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd898148cf0 con 0x7fd8980c4630 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.546+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890014440 con 0x7fd8980c4630 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.546+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8980c67a0 con 0x7fd8980c4630 2024-02-26T10:38:31.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.547+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8980c69a0 con 0x7fd8980c4630 2024-02-26T10:38:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.548+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fd89001c070 con 0x7fd8980c4630 2024-02-26T10:38:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.549+0000 7fd88f400640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd87406b440 0x7fd87406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.549+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd890089970 con 0x7fd8980c4630 2024-02-26T10:38:31.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.550+0000 7fd896a00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd87406b440 0x7fd87406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.551+0000 7fd896a00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd87406b440 0x7fd87406d900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd888036000 tx=0x7fd88803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.554+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd890058c10 con 0x7fd8980c4630 2024-02-26T10:38:31.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.756+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd898074a30 con 0x7fd8980c4630 2024-02-26T10:38:31.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.756+0000 7fd88f400640 1 -- 172.21.15.67:0/536212094 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd890058510 con 0x7fd8980c4630 2024-02-26T10:38:31.759 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:38:31.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.759+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd87406b440 msgr2=0x7fd87406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.759+0000 7fd89de00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd87406b440 0x7fd87406d900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd888036000 tx=0x7fd88803c040 comp rx=0 tx=0).stop 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.759+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 msgr2=0x7fd8980c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.759+0000 7fd89de00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c5840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd89000ba10 tx=0x7fd89000bed0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 shutdown_connections 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8980c4630 0x7fd8980c5840 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd87406b440 0x7fd87406d900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8980c0b40 0x7fd8980c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 --2- 172.21.15.67:0/536212094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd898071ff0 0x7fd898154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 >> 172.21.15.67:0/536212094 conn(0x7fd8980c7b00 msgr2=0x7fd89806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 shutdown_connections 2024-02-26T10:38:31.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.760+0000 7fd89de00640 1 -- 172.21.15.67:0/536212094 wait complete. 2024-02-26T10:38:31.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:38:31.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.989+0000 7f1f5b200640 1 Processor -- start 2024-02-26T10:38:31.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.990+0000 7f1f5b200640 1 -- start start 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f5b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f5c0c8510 0x7f1f5c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f5b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c152760 con 0x7f1f5c152380 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f5b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c06d9a0 con 0x7f1f5c0c7020 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f5b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c7550 con 0x7f1f5c0c8510 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f59e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1f5c152380 0x7f1f5c155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40540/0 (socket says 172.21.15.67:40540) 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f59e00640 1 -- 172.21.15.67:0/2021473115 learned_addr learned my addr 172.21.15.67:0/2021473115 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:31.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.991+0000 7f1f5a800640 1 --2- 172.21.15.67:0/2021473115 >> v2:172.21.15.82:3300/0 conn(0x7f1f5c0c8510 0x7f1f5c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.992+0000 7f1f5a800640 1 -- 172.21.15.67:0/2021473115 >> v1:172.21.15.82:6789/0 conn(0x7f1f5c0c7020 legacy=0x7f1f5c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.992+0000 7f1f5a800640 1 -- 172.21.15.67:0/2021473115 >> v1:172.21.15.67:6789/0 conn(0x7f1f5c152380 legacy=0x7f1f5c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.992+0000 7f1f5a800640 1 -- 172.21.15.67:0/2021473115 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0c7830 con 0x7f1f5c0c8510 2024-02-26T10:38:31.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.993+0000 7f1f5a800640 1 --2- 172.21.15.67:0/2021473115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c0c4f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009d90 tx=0x7f1f5402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca5b799d4dca89e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f59400640 1 -- 172.21.15.67:0/2021473115 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54039070 con 0x7f1f5c0c8510 2024-02-26T10:38:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f59400640 1 -- 172.21.15.67:0/2021473115 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5402cc60 con 0x7f1f5c0c8510 2024-02-26T10:38:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f59400640 1 -- 172.21.15.67:0/2021473115 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54034660 con 0x7f1f5c0c8510 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 -- 172.21.15.67:0/2021473115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c0c8510 msgr2=0x7f1f5c0c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 --2- 172.21.15.67:0/2021473115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c0c4f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009d90 tx=0x7f1f5402beb0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 -- 172.21.15.67:0/2021473115 shutdown_connections 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 --2- 172.21.15.67:0/2021473115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c0c4f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 -- 172.21.15.67:0/2021473115 >> 172.21.15.67:0/2021473115 conn(0x7f1f5c0ac1a0 msgr2=0x7f1f5c0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 -- 172.21.15.67:0/2021473115 shutdown_connections 2024-02-26T10:38:31.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.994+0000 7f1f5b200640 1 -- 172.21.15.67:0/2021473115 wait complete. 2024-02-26T10:38:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.996+0000 7f1f5b200640 1 Processor -- start 2024-02-26T10:38:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.996+0000 7f1f5b200640 1 -- start start 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f5c0c7020 0x7f1f5c0ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c1477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f5b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c152380 0x7f1f5c147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f5b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c54d0 con 0x7f1f5c0c8510 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f5b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c06cd70 con 0x7f1f5c152380 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f5b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c5650 con 0x7f1f5c0c7020 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f51400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c1477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f51400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c1477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37544/0 (socket says 172.21.15.67:37544) 2024-02-26T10:38:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.997+0000 7f1f51400640 1 -- 172.21.15.67:0/73213354 learned_addr learned my addr 172.21.15.67:0/73213354 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f59e00640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f5c0c7020 0x7f1f5c0ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f5a800640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c152380 0x7f1f5c147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f51400640 1 -- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f5c0c7020 msgr2=0x7f1f5c0ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f51400640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f5c0c7020 0x7f1f5c0ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f51400640 1 -- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c152380 msgr2=0x7f1f5c147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f51400640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c152380 0x7f1f5c147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f51400640 1 -- 172.21.15.67:0/73213354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0c0730 con 0x7f1f5c0c8510 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f59e00640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f5c0c7020 0x7f1f5c0ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f51400640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c1477f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400c400 tx=0x7f1f4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.998+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f440052d0 con 0x7f1f5c0c8510 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.999+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f44005490 con 0x7f1f5c0c8510 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.999+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44007260 con 0x7f1f5c0c8510 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.999+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0c0a10 con 0x7f1f5c0c8510 2024-02-26T10:38:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:31.999+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0b0090 con 0x7f1f5c0c8510 2024-02-26T10:38:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.000+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c074a30 con 0x7f1f5c0c8510 2024-02-26T10:38:32.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.001+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f1f44007400 con 0x7f1f5c0c8510 2024-02-26T10:38:32.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.002+0000 7f1f52a00640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f3806b440 0x7f1f3806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.002+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f440892e0 con 0x7f1f5c0c8510 2024-02-26T10:38:32.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.003+0000 7f1f59e00640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f3806b440 0x7f1f3806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.004+0000 7f1f59e00640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f3806b440 0x7f1f3806d900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005880 tx=0x7f1f4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.007+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f44057ec0 con 0x7f1f5c0c8510 2024-02-26T10:38:32.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.208+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f5c0c7c50 con 0x7f1f5c0c8510 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.209+0000 7f1f52a00640 1 -- 172.21.15.67:0/73213354 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1f4405bf60 con 0x7f1f5c0c8510 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:38:32.211 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:38:32.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.211+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f3806b440 msgr2=0x7f1f3806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.211+0000 7f1f5b200640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f3806b440 0x7f1f3806d900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005880 tx=0x7f1f4c009040 comp rx=0 tx=0).stop 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.211+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 msgr2=0x7f1f5c1477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.211+0000 7f1f5b200640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c1477f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400c400 tx=0x7f1f4400c8c0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 shutdown_connections 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f3806b440 0x7f1f3806d900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f5c152380 0x7f1f5c147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f5c0c8510 0x7f1f5c1477f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 --2- 172.21.15.67:0/73213354 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f5c0c7020 0x7f1f5c0ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 >> 172.21.15.67:0/73213354 conn(0x7f1f5c0ac1a0 msgr2=0x7f1f5c1504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 shutdown_connections 2024-02-26T10:38:32.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.212+0000 7f1f5b200640 1 -- 172.21.15.67:0/73213354 wait complete. 2024-02-26T10:38:32.231 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T10:38:32.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T10:38:32.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.451+0000 7f772f200640 1 Processor -- start 2024-02-26T10:38:32.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.451+0000 7f772f200640 1 -- start start 2024-02-26T10:38:32.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77300a9430 0x7f77300ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7730150780 con 0x7f773006e2d0 2024-02-26T10:38:32.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300a98c0 con 0x7f77301503a0 2024-02-26T10:38:32.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77300a9ba0 con 0x7f77300a9430 2024-02-26T10:38:32.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f773006e2d0 0x7f77300c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40554/0 (socket says 172.21.15.67:40554) 2024-02-26T10:38:32.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772e800640 1 -- 172.21.15.67:0/3532819928 learned_addr learned my addr 172.21.15.67:0/3532819928 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.453+0000 7f772d400640 1 --2- 172.21.15.67:0/3532819928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77300a9430 0x7f77300ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022681887 0 0) 0x7f7730150780 con 0x7f773006e2d0 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7714003520 con 0x7f773006e2d0 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704365854 0 0) 0x7f77300a98c0 con 0x7f77301503a0 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7730150780 con 0x7f77301503a0 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1694298067 0 0) 0x7f7714003520 con 0x7f773006e2d0 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772d400640 1 -- 172.21.15.67:0/3532819928 >> v1:172.21.15.82:6789/0 conn(0x7f77301503a0 legacy=0x7f77301535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772d400640 1 -- 172.21.15.67:0/3532819928 >> v1:172.21.15.67:6789/0 conn(0x7f773006e2d0 legacy=0x7f77300c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.454+0000 7f772d400640 1 -- 172.21.15.67:0/3532819928 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300a9e80 con 0x7f77300a9430 2024-02-26T10:38:32.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.455+0000 7f772d400640 1 --2- 172.21.15.67:0/3532819928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77300a9430 0x7f77300ab480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f771c0097d0 tx=0x7f771c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bdfb111e9e34 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:32.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.456+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c037070 con 0x7f77300a9430 2024-02-26T10:38:32.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.456+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c030510 con 0x7f77300a9430 2024-02-26T10:38:32.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.456+0000 7f772ca00640 1 -- 172.21.15.67:0/3532819928 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280019f0 con 0x7f77300a9430 2024-02-26T10:38:32.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 -- 172.21.15.67:0/3532819928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77300a9430 msgr2=0x7f77300ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 --2- 172.21.15.67:0/3532819928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77300a9430 0x7f77300ab480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f771c0097d0 tx=0x7f771c02b800 comp rx=0 tx=0).stop 2024-02-26T10:38:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 -- 172.21.15.67:0/3532819928 shutdown_connections 2024-02-26T10:38:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 --2- 172.21.15.67:0/3532819928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77300a9430 0x7f77300ab480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 -- 172.21.15.67:0/3532819928 >> 172.21.15.67:0/3532819928 conn(0x7f77300c6380 msgr2=0x7f77300c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 -- 172.21.15.67:0/3532819928 shutdown_connections 2024-02-26T10:38:32.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.457+0000 7f772f200640 1 -- 172.21.15.67:0/3532819928 wait complete. 2024-02-26T10:38:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.458+0000 7f772f200640 1 Processor -- start 2024-02-26T10:38:32.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.459+0000 7f772f200640 1 -- start start 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.459+0000 7f772f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006e2d0 0x7f7730144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.459+0000 7f772f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 0x7f77301472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.459+0000 7f772f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730147800 0x7f77301555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.459+0000 7f772f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300c0f80 con 0x7f77301503a0 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.459+0000 7f772f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773006ddb0 con 0x7f7730147800 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77300c3520 con 0x7f773006e2d0 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 0x7f77301472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 0x7f77301472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37546/0 (socket says 172.21.15.67:37546) 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 -- 172.21.15.67:0/3477622688 learned_addr learned my addr 172.21.15.67:0/3477622688 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772e800640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730147800 0x7f77301555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772de00640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006e2d0 0x7f7730144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 -- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006e2d0 msgr2=0x7f7730144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006e2d0 0x7f7730144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 -- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730147800 msgr2=0x7f77301555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730147800 0x7f77301555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.460+0000 7f772d400640 1 -- 172.21.15.67:0/3477622688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77301452a0 con 0x7f77301503a0 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.461+0000 7f772d400640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 0x7f77301472c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f771c0097d0 tx=0x7f771c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.461+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c037040 con 0x7f77301503a0 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.461+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730144fc0 con 0x7f77301503a0 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.461+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c004250 con 0x7f77301503a0 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.461+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c03c670 con 0x7f77301503a0 2024-02-26T10:38:32.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.461+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7730150d10 con 0x7f77301503a0 2024-02-26T10:38:32.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.462+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7730074a30 con 0x7f77301503a0 2024-02-26T10:38:32.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.463+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f771c03a070 con 0x7f77301503a0 2024-02-26T10:38:32.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.464+0000 7f7726000640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f771006b470 0x7f771006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:32.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.464+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f771c0ab740 con 0x7f77301503a0 2024-02-26T10:38:32.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.465+0000 7f772de00640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f771006b470 0x7f771006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:32.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.466+0000 7f772de00640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f771006b470 0x7f771006d930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77300c1dc0 tx=0x7f7720009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:32.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.469+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f771c041020 con 0x7f77301503a0 2024-02-26T10:38:32.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:32.665+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f77300c3ad0 con 0x7f77301503a0 2024-02-26T10:38:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.130+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f771c0768d0 con 0x7f77301503a0 2024-02-26T10:38:33.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.135+0000 7f7726000640 1 -- 172.21.15.67:0/3477622688 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f771c07a960 con 0x7f77301503a0 2024-02-26T10:38:33.139 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f771006b470 msgr2=0x7f771006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f771006b470 0x7f771006d930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77300c1dc0 tx=0x7f7720009040 comp rx=0 tx=0).stop 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 msgr2=0x7f77301472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 0x7f77301472c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f771c0097d0 tx=0x7f771c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 shutdown_connections 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f771006b470 0x7f771006d930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730147800 0x7f77301555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77301503a0 0x7f77301472c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 --2- 172.21.15.67:0/3477622688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006e2d0 0x7f7730144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.139+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 >> 172.21.15.67:0/3477622688 conn(0x7f77300c6380 msgr2=0x7f77300c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.140+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 shutdown_connections 2024-02-26T10:38:33.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.140+0000 7f772f200640 1 -- 172.21.15.67:0/3477622688 wait complete. 2024-02-26T10:38:33.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:38:33.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.382+0000 7f3003200640 1 Processor -- start 2024-02-26T10:38:33.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.383+0000 7f3003200640 1 -- start start 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3003200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3004152390 0x7f30040ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3003200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30040bdcb0 0x7f30040c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3003200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3004155c30 con 0x7f30040c4fa0 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3003200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3004155700 con 0x7f30040bdcb0 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3003200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3004155950 con 0x7f3004152390 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3001400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30040bdcb0 0x7f30040c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3001400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30040bdcb0 0x7f30040c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37550/0 (socket says 172.21.15.67:37550) 2024-02-26T10:38:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3001400640 1 -- 172.21.15.67:0/18957274 learned_addr learned my addr 172.21.15.67:0/18957274 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:33.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.384+0000 7f3001e00640 1 --2- 172.21.15.67:0/18957274 >> v2:172.21.15.82:3300/0 conn(0x7f3004152390 0x7f30040ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:33.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.385+0000 7f3001400640 1 -- 172.21.15.67:0/18957274 >> v2:172.21.15.82:3300/0 conn(0x7f3004152390 msgr2=0x7f30040ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:33.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.385+0000 7f3001400640 1 --2- 172.21.15.67:0/18957274 >> v2:172.21.15.82:3300/0 conn(0x7f3004152390 0x7f30040ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.385+0000 7f3001400640 1 -- 172.21.15.67:0/18957274 >> v1:172.21.15.82:6789/0 conn(0x7f30040c4fa0 legacy=0x7f30040c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:33.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.385+0000 7f3001400640 1 -- 172.21.15.67:0/18957274 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300406bc70 con 0x7f30040bdcb0 2024-02-26T10:38:33.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.387+0000 7f3001400640 1 --2- 172.21.15.67:0/18957274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30040bdcb0 0x7f30040c85b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f300408bc20 tx=0x7f2ff002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91605cd4559a78a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:33.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.387+0000 7f3000a00640 1 -- 172.21.15.67:0/18957274 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff002e070 con 0x7f30040bdcb0 2024-02-26T10:38:33.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.387+0000 7f3000a00640 1 -- 172.21.15.67:0/18957274 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff0037070 con 0x7f30040bdcb0 2024-02-26T10:38:33.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.387+0000 7f3000a00640 1 -- 172.21.15.67:0/18957274 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0031a00 con 0x7f30040bdcb0 2024-02-26T10:38:33.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 -- 172.21.15.67:0/18957274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30040bdcb0 msgr2=0x7f30040c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:33.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 --2- 172.21.15.67:0/18957274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30040bdcb0 0x7f30040c85b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f300408bc20 tx=0x7f2ff002b880 comp rx=0 tx=0).stop 2024-02-26T10:38:33.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 -- 172.21.15.67:0/18957274 shutdown_connections 2024-02-26T10:38:33.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 --2- 172.21.15.67:0/18957274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30040bdcb0 0x7f30040c85b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 --2- 172.21.15.67:0/18957274 >> v2:172.21.15.82:3300/0 conn(0x7f3004152390 0x7f30040ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 -- 172.21.15.67:0/18957274 >> 172.21.15.67:0/18957274 conn(0x7f30040ca620 msgr2=0x7f30040caa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:33.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 -- 172.21.15.67:0/18957274 shutdown_connections 2024-02-26T10:38:33.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.388+0000 7f3003200640 1 -- 172.21.15.67:0/18957274 wait complete. 2024-02-26T10:38:33.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.390+0000 7f3003200640 1 Processor -- start 2024-02-26T10:38:33.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.390+0000 7f3003200640 1 -- start start 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3003200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30040bdcb0 0x7f30040abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3003200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30040c4fa0 0x7f30041eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3003200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 0x7f30041eb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3003200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040c9200 con 0x7f3004152390 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3003200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300406c230 con 0x7f30040bdcb0 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3003200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30040ca2f0 con 0x7f30040c4fa0 2024-02-26T10:38:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3002800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 0x7f30041eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3002800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 0x7f30041eb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37556/0 (socket says 172.21.15.67:37556) 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3002800640 1 -- 172.21.15.67:0/2794396173 learned_addr learned my addr 172.21.15.67:0/2794396173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3001e00640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30040bdcb0 0x7f30040abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.391+0000 7f3001400640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30040c4fa0 0x7f30041eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3002800640 1 -- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30040c4fa0 msgr2=0x7f30041eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3002800640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30040c4fa0 0x7f30041eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3002800640 1 -- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30040bdcb0 msgr2=0x7f30040abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3002800640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30040bdcb0 0x7f30040abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3002800640 1 -- 172.21.15.67:0/2794396173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040c0160 con 0x7f3004152390 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3002800640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 0x7f30041eb020 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c7a0 tx=0x7f2ffc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc003b70 con 0x7f3004152390 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc003d30 con 0x7f3004152390 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00aa90 con 0x7f3004152390 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.392+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30040c0440 con 0x7f3004152390 2024-02-26T10:38:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.393+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30040c0a00 con 0x7f3004152390 2024-02-26T10:38:33.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.394+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3004074a30 con 0x7f3004152390 2024-02-26T10:38:33.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.395+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ffc00ac30 con 0x7f3004152390 2024-02-26T10:38:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.396+0000 7f2ffa000640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2fe406b4d0 0x7f2fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.396+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2ffc089dc0 con 0x7f3004152390 2024-02-26T10:38:33.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.396+0000 7f3001e00640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2fe406b4d0 0x7f2fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:33.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.397+0000 7f3001e00640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2fe406b4d0 0x7f2fe406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3004150580 tx=0x7f2ff4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.402+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ffc058840 con 0x7f3004152390 2024-02-26T10:38:33.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:33.592+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f30040c9ff0 con 0x7f3004152390 2024-02-26T10:38:34.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.141+0000 7f2ffa000640 1 -- 172.21.15.67:0/2794396173 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2ffc05c8e0 con 0x7f3004152390 2024-02-26T10:38:34.143 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2fe406b4d0 msgr2=0x7f2fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2fe406b4d0 0x7f2fe406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3004150580 tx=0x7f2ff4009190 comp rx=0 tx=0).stop 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 msgr2=0x7f30041eb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 0x7f30041eb020 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c7a0 tx=0x7f2ffc00cc60 comp rx=0 tx=0).stop 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 shutdown_connections 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3004152390 0x7f30041eb020 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30040c4fa0 0x7f30041eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2fe406b4d0 0x7f2fe406d990 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3004150580 tx=0x7f2ff4009190 comp rx=0 tx=0).stop 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 --2- 172.21.15.67:0/2794396173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30040bdcb0 0x7f30040abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.147+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 >> 172.21.15.67:0/2794396173 conn(0x7f30040ca620 msgr2=0x7f3004150ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.149+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 shutdown_connections 2024-02-26T10:38:34.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.150+0000 7f3003200640 1 -- 172.21.15.67:0/2794396173 wait complete. 2024-02-26T10:38:34.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T10:38:34.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.371+0000 7fa17ee00640 1 Processor -- start 2024-02-26T10:38:34.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.372+0000 7fa17ee00640 1 -- start start 2024-02-26T10:38:34.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.373+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa178158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.373+0000 7fa17ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa178072ce0 con 0x7fa1780c1400 2024-02-26T10:38:34.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.373+0000 7fa17ee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa178155bf0 con 0x7fa1780c0ef0 2024-02-26T10:38:34.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.373+0000 7fa17ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa178073010 con 0x7fa1781591b0 2024-02-26T10:38:34.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.373+0000 7fa17ce00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa1780c1400 0x7fa17806de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40560/0 (socket says 172.21.15.67:40560) 2024-02-26T10:38:34.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.373+0000 7fa17ce00640 1 -- 172.21.15.67:0/1867596299 learned_addr learned my addr 172.21.15.67:0/1867596299 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177e00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa178158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853216515 0 0) 0x7fa178072ce0 con 0x7fa1780c1400 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa15c003520 con 0x7fa1780c1400 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2410169159 0 0) 0x7fa15c003520 con 0x7fa1780c1400 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 msgr2=0x7fa178158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa178158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 >> v1:172.21.15.82:6789/0 conn(0x7fa1780c0ef0 legacy=0x7fa178070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178155ed0 con 0x7fa1780c1400 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fa170001a10 con 0x7fa1780c1400 2024-02-26T10:38:34.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 >> v1:172.21.15.67:6789/0 conn(0x7fa1780c1400 legacy=0x7fa17806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.374+0000 7fa177400640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa15c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177400640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa15c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177400640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c004d40 0x7fa15c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa178073010 con 0x7fa15c003f60 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00bb50 con 0x7fa15c004d40 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00bd50 con 0x7fa1781591b0 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa16fe00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa15c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa17ce00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c004d40 0x7fa15c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177e00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa15c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa16fe00640 1 -- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 msgr2=0x7fa15c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa16fe00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa15c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa16fe00640 1 -- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c004d40 msgr2=0x7fa15c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa16fe00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c004d40 0x7fa15c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa16fe00640 1 -- 172.21.15.67:0/1867596299 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c003520 con 0x7fa15c003f60 2024-02-26T10:38:34.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa177e00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781591b0 0x7fa15c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:34.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.375+0000 7fa17ce00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa15c004d40 0x7fa15c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.376+0000 7fa16fe00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa15c004800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa160001720 tx=0x7fa15c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce3fd9124aa36da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160001370 con 0x7fa15c003f60 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1600052d0 con 0x7fa15c003f60 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa177400640 1 -- 172.21.15.67:0/1867596299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160003830 con 0x7fa15c003f60 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ee00640 1 -- 172.21.15.67:0/1867596299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 msgr2=0x7fa15c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ee00640 1 --2- 172.21.15.67:0/1867596299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa15c004800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa160001720 tx=0x7fa15c005830 comp rx=0 tx=0).stop 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ce00640 1 -- 172.21.15.67:0/1867596299 reap_dead start 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ee00640 1 -- 172.21.15.67:0/1867596299 shutdown_connections 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ee00640 1 -- 172.21.15.67:0/1867596299 >> 172.21.15.67:0/1867596299 conn(0x7fa1780c6da0 msgr2=0x7fa1780c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ee00640 1 -- 172.21.15.67:0/1867596299 shutdown_connections 2024-02-26T10:38:34.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.377+0000 7fa17ee00640 1 -- 172.21.15.67:0/1867596299 wait complete. 2024-02-26T10:38:34.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.379+0000 7fa17ee00640 1 Processor -- start 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.379+0000 7fa17ee00640 1 -- start start 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa1781509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa17814bd90 0x7fa1781511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa17ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781520f0 0x7fa1780a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa17ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17806e560 con 0x7fa15c003f60 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa17ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17814f490 con 0x7fa17814bd90 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa17ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17806e6e0 con 0x7fa1781520f0 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa177e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa1781509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa177e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa1781509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37582/0 (socket says 172.21.15.67:37582) 2024-02-26T10:38:34.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.380+0000 7fa177e00640 1 -- 172.21.15.67:0/3559823731 learned_addr learned my addr 172.21.15.67:0/3559823731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa16fe00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa17814bd90 0x7fa1781511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa177e00640 1 -- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781520f0 msgr2=0x7fa1780a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa17ce00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781520f0 0x7fa1780a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa177e00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781520f0 0x7fa1780a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa177e00640 1 -- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa17814bd90 msgr2=0x7fa1781511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa177e00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa17814bd90 0x7fa1781511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa177e00640 1 -- 172.21.15.67:0/3559823731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17814bb20 con 0x7fa15c003f60 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa16fe00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa17814bd90 0x7fa1781511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa17ce00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781520f0 0x7fa1780a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:34.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa177e00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa1781509c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1680108e0 tx=0x7fa168010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800f040 con 0x7fa15c003f60 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16800ec20 con 0x7fa15c003f60 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16801c400 con 0x7fa15c003f60 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.381+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17814b840 con 0x7fa15c003f60 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.382+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780abce0 con 0x7fa15c003f60 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.383+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa178074a30 con 0x7fa15c003f60 2024-02-26T10:38:34.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.384+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa16801c5a0 con 0x7fa15c003f60 2024-02-26T10:38:34.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.385+0000 7fa174c00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:34.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.385+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa16808a8c0 con 0x7fa15c003f60 2024-02-26T10:38:34.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.385+0000 7fa16fe00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:34.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.386+0000 7fa16fe00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa160001720 tx=0x7fa160009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:34.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.390+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa168018030 con 0x7fa15c003f60 2024-02-26T10:38:34.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:34.629+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa17806eae0 con 0x7fa15c003f60 2024-02-26T10:38:35.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.161+0000 7fa174c00640 1 -- 172.21.15.67:0/3559823731 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa168059830 con 0x7fa15c003f60 2024-02-26T10:38:35.170 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-02-26T10:38:35.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.167+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa15406b4d0 msgr2=0x7fa15406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.167+0000 7fa17ee00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa160001720 tx=0x7fa160009030 comp rx=0 tx=0).stop 2024-02-26T10:38:35.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.167+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 msgr2=0x7fa1781509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.167+0000 7fa17ee00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa1781509c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1680108e0 tx=0x7fa168010da0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.170+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 shutdown_connections 2024-02-26T10:38:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.170+0000 7fa17ee00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa1781520f0 0x7fa1780a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.170+0000 7fa17ee00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa160001720 tx=0x7fa160009030 comp rx=0 tx=0).stop 2024-02-26T10:38:35.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.170+0000 7fa17ee00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa17814bd90 0x7fa1781511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.170+0000 7fa17ee00640 1 --2- 172.21.15.67:0/3559823731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa15c003f60 0x7fa1781509c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.170+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 >> 172.21.15.67:0/3559823731 conn(0x7fa1780c6da0 msgr2=0x7fa1780bf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:35.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.171+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 shutdown_connections 2024-02-26T10:38:35.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.171+0000 7fa17ee00640 1 -- 172.21.15.67:0/3559823731 wait complete. 2024-02-26T10:38:35.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:35.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.377+0000 7ff359a00640 1 Processor -- start 2024-02-26T10:38:35.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.378+0000 7ff359a00640 1 -- start start 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff359a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff3540c2870 0x7ff354155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff359a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3541556a0 0x7ff354071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff359a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff354155c00 con 0x7ff35414f3d0 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff359a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35406dd80 con 0x7ff3540c2870 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff359a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff354155ee0 con 0x7ff3541556a0 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff34be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff3540c2870 0x7ff354155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff34be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff3540c2870 0x7ff354155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37590/0 (socket says 172.21.15.67:37590) 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.379+0000 7ff34be00640 1 -- 172.21.15.67:0/2578120640 learned_addr learned my addr 172.21.15.67:0/2578120640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.380+0000 7ff34be00640 1 -- 172.21.15.67:0/2578120640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3541556a0 msgr2=0x7ff354071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.380+0000 7ff34be00640 1 --2- 172.21.15.67:0/2578120640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3541556a0 0x7ff354071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.380+0000 7ff34be00640 1 -- 172.21.15.67:0/2578120640 >> v1:172.21.15.82:6789/0 conn(0x7ff35414f3d0 legacy=0x7ff35406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.380+0000 7ff34be00640 1 -- 172.21.15.67:0/2578120640 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35406dfd0 con 0x7ff3540c2870 2024-02-26T10:38:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.381+0000 7ff34be00640 1 --2- 172.21.15.67:0/2578120640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3540c2870 0x7ff354155160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009870 tx=0x7ff33c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dd290455ec835f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.381+0000 7ff352000640 1 -- 172.21.15.67:0/2578120640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0317c0 con 0x7ff3540c2870 2024-02-26T10:38:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.381+0000 7ff352000640 1 -- 172.21.15.67:0/2578120640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c037070 con 0x7ff3540c2870 2024-02-26T10:38:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.381+0000 7ff352000640 1 -- 172.21.15.67:0/2578120640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c031b80 con 0x7ff3540c2870 2024-02-26T10:38:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.381+0000 7ff359a00640 1 -- 172.21.15.67:0/2578120640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3540c2870 msgr2=0x7ff354155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.381+0000 7ff359a00640 1 --2- 172.21.15.67:0/2578120640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3540c2870 0x7ff354155160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009870 tx=0x7ff33c02b860 comp rx=0 tx=0).stop 2024-02-26T10:38:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.382+0000 7ff359a00640 1 -- 172.21.15.67:0/2578120640 shutdown_connections 2024-02-26T10:38:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.382+0000 7ff359a00640 1 --2- 172.21.15.67:0/2578120640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3541556a0 0x7ff354071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.382+0000 7ff359a00640 1 --2- 172.21.15.67:0/2578120640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3540c2870 0x7ff354155160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.382+0000 7ff359a00640 1 -- 172.21.15.67:0/2578120640 >> 172.21.15.67:0/2578120640 conn(0x7ff3540a84a0 msgr2=0x7ff3540a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.382+0000 7ff359a00640 1 -- 172.21.15.67:0/2578120640 shutdown_connections 2024-02-26T10:38:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.382+0000 7ff359a00640 1 -- 172.21.15.67:0/2578120640 wait complete. 2024-02-26T10:38:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.383+0000 7ff359a00640 1 Processor -- start 2024-02-26T10:38:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.383+0000 7ff359a00640 1 -- start start 2024-02-26T10:38:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff359a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3540c2870 0x7ff354150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff359a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff35414f3d0 0x7ff3540c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff359a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 0x7ff3541ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff359a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540c2cb0 con 0x7ff3541556a0 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff359a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540c1810 con 0x7ff3540c2870 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff359a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540c2e30 con 0x7ff35414f3d0 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.384+0000 7ff353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 0x7ff3541ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 0x7ff3541ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37596/0 (socket says 172.21.15.67:37596) 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 -- 172.21.15.67:0/668368511 learned_addr learned my addr 172.21.15.67:0/668368511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff34be00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff35414f3d0 0x7ff3540c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff352a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3540c2870 0x7ff354150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 -- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff35414f3d0 msgr2=0x7ff3540c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff35414f3d0 0x7ff3540c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 -- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3540c2870 msgr2=0x7ff354150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3540c2870 0x7ff354150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 -- 172.21.15.67:0/668368511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff354152980 con 0x7ff3541556a0 2024-02-26T10:38:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff353400640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 0x7ff3541ee2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00b970 tx=0x7ff34c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.385+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c019070 con 0x7ff3541556a0 2024-02-26T10:38:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.386+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00cb60 con 0x7ff3541556a0 2024-02-26T10:38:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.386+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c014440 con 0x7ff3541556a0 2024-02-26T10:38:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.386+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35414b2b0 con 0x7ff3541556a0 2024-02-26T10:38:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.386+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3540c7310 con 0x7ff3541556a0 2024-02-26T10:38:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.387+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354074a30 con 0x7ff3541556a0 2024-02-26T10:38:35.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.388+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff34c01c070 con 0x7ff3541556a0 2024-02-26T10:38:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.389+0000 7ff34aa00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff33006b4d0 0x7ff33006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.389+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff34c089f30 con 0x7ff3541556a0 2024-02-26T10:38:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.389+0000 7ff352a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff33006b4d0 0x7ff33006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.390+0000 7ff352a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff33006b4d0 0x7ff33006d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff344006fd0 tx=0x7ff344008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:35.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.393+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff34c058da0 con 0x7ff3541556a0 2024-02-26T10:38:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.585+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3540c13f0 con 0x7ff3541556a0 2024-02-26T10:38:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.586+0000 7ff34aa00640 1 -- 172.21.15.67:0/668368511 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16546 (secure 0 0 0) 0x7ff34c0586a0 con 0x7ff3541556a0 2024-02-26T10:38:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:35.589 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":20,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:34.630979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff33006b4d0 msgr2=0x7ff33006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff33006b4d0 0x7ff33006d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff344006fd0 tx=0x7ff344008040 comp rx=0 tx=0).stop 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 msgr2=0x7ff3541ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 0x7ff3541ee2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00b970 tx=0x7ff34c00be30 comp rx=0 tx=0).stop 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 shutdown_connections 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3541556a0 0x7ff3541ee2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff35414f3d0 0x7ff3540c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff33006b4d0 0x7ff33006d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 --2- 172.21.15.67:0/668368511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3540c2870 0x7ff354150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 >> 172.21.15.67:0/668368511 conn(0x7ff3540a84a0 msgr2=0x7ff35406e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 shutdown_connections 2024-02-26T10:38:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.589+0000 7ff359a00640 1 -- 172.21.15.67:0/668368511 wait complete. 2024-02-26T10:38:35.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.827+0000 7f1876800640 1 Processor -- start 2024-02-26T10:38:35.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.827+0000 7f1876800640 1 -- start start 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.828+0000 7f1876800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f187006ac20 0x7f18700a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.828+0000 7f1876800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 0x7f187006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.828+0000 7f1876800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187006b460 con 0x7f1870153610 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.828+0000 7f1876800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18700a9a80 con 0x7f187006ac20 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.828+0000 7f1876800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18700a9d60 con 0x7f187006b000 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.829+0000 7f1875e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1870153610 0x7f18700abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40562/0 (socket says 172.21.15.67:40562) 2024-02-26T10:38:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.829+0000 7f1875e00640 1 -- 172.21.15.67:0/2232847820 learned_addr learned my addr 172.21.15.67:0/2232847820 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.829+0000 7f1874a00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 0x7f187006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.829+0000 7f1875400640 1 --2- 172.21.15.67:0/2232847820 >> v2:172.21.15.82:3300/0 conn(0x7f187006ac20 0x7f18700a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.829+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667929831 0 0) 0x7f187006b460 con 0x7f1870153610 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.829+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1858003520 con 0x7f1870153610 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1243644613 0 0) 0x7f1858003520 con 0x7f1870153610 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 msgr2=0x7f187006db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 0x7f187006db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 >> v2:172.21.15.82:3300/0 conn(0x7f187006ac20 msgr2=0x7f18700a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 --2- 172.21.15.67:0/2232847820 >> v2:172.21.15.82:3300/0 conn(0x7f187006ac20 0x7f18700a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18700ab750 con 0x7f1870153610 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f18680019f0 con 0x7f1870153610 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 >> v1:172.21.15.67:6789/0 conn(0x7f1870153610 legacy=0x7f18700abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f1858003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 0x7f1858004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1858004d40 0x7f18580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18700a9a80 con 0x7f18580039b0 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18700a9d60 con 0x7f1858004d40 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185800b9c0 con 0x7f187006b000 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875400640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f1858003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875e00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1858004d40 0x7f18580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1874a00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 0x7f1858004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875400640 1 -- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 msgr2=0x7f1858004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875400640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f187006b000 0x7f1858004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875400640 1 -- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1858004d40 msgr2=0x7f18580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875400640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1858004d40 0x7f18580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875400640 1 -- 172.21.15.67:0/2232847820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1858003520 con 0x7f18580039b0 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.830+0000 7f1875e00640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1858004d40 0x7f18580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.831+0000 7f1875400640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f1858003dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ac0 tx=0x7f1858005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec32ccadc56fb711 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864039070 con 0x7f18580039b0 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186402c710 con 0x7f18580039b0 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f186fe00640 1 -- 172.21.15.67:0/2232847820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864009690 con 0x7f18580039b0 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f1876800640 1 -- 172.21.15.67:0/2232847820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 msgr2=0x7f1858003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f1876800640 1 --2- 172.21.15.67:0/2232847820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f1858003dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ac0 tx=0x7f1858005830 comp rx=0 tx=0).stop 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f1875e00640 1 -- 172.21.15.67:0/2232847820 reap_dead start 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f1876800640 1 -- 172.21.15.67:0/2232847820 shutdown_connections 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.832+0000 7f1876800640 1 -- 172.21.15.67:0/2232847820 >> 172.21.15.67:0/2232847820 conn(0x7f18700c63f0 msgr2=0x7f18700c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.833+0000 7f1876800640 1 -- 172.21.15.67:0/2232847820 shutdown_connections 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.833+0000 7f1876800640 1 -- 172.21.15.67:0/2232847820 wait complete. 2024-02-26T10:38:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.834+0000 7f1876800640 1 Processor -- start 2024-02-26T10:38:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.834+0000 7f1876800640 1 -- start start 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.835+0000 7f1876800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f187014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.835+0000 7f1876800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1870070c50 0x7f187014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.835+0000 7f1876800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187014d130 0x7f1870148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.835+0000 7f1876800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870150410 con 0x7f18580039b0 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.835+0000 7f1876800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870153aa0 con 0x7f187014d130 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.835+0000 7f1876800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1870150590 con 0x7f1870070c50 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f187014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f187014dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37606/0 (socket says 172.21.15.67:37606) 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 -- 172.21.15.67:0/3785465806 learned_addr learned my addr 172.21.15.67:0/3785465806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875e00640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187014d130 0x7f1870148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1874a00640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1870070c50 0x7f187014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 -- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1870070c50 msgr2=0x7f187014f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1870070c50 0x7f187014f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 -- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187014d130 msgr2=0x7f1870148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187014d130 0x7f1870148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 -- 172.21.15.67:0/3785465806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18701ee8c0 con 0x7f18580039b0 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875e00640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187014d130 0x7f1870148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.836+0000 7f1875400640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f187014dc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f186402c380 tx=0x7f1864004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.837+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864042020 con 0x7f18580039b0 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.837+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186403a040 con 0x7f18580039b0 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.837+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18701eeda0 con 0x7f18580039b0 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.837+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864039070 con 0x7f18580039b0 2024-02-26T10:38:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.837+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1870149cb0 con 0x7f18580039b0 2024-02-26T10:38:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.838+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1870149ef0 con 0x7f18580039b0 2024-02-26T10:38:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.839+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1864045020 con 0x7f18580039b0 2024-02-26T10:38:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.840+0000 7f186d600640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f185006b500 0x7f185006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.840+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18640ab270 con 0x7f18580039b0 2024-02-26T10:38:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.840+0000 7f1874a00640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f185006b500 0x7f185006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:35.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.841+0000 7f1874a00640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f185006b500 0x7f185006d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1870150140 tx=0x7f185c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:35.844+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f186404c020 con 0x7f18580039b0 2024-02-26T10:38:36.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.047+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1870149aa0 con 0x7f18580039b0 2024-02-26T10:38:36.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.050+0000 7f186d600640 1 -- 172.21.15.67:0/3785465806 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16546 (secure 0 0 0) 0x7f186407a180 con 0x7f18580039b0 2024-02-26T10:38:36.053 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:36.053 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":20,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:34.630979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f185006b500 msgr2=0x7f185006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f185006b500 0x7f185006d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1870150140 tx=0x7f185c008040 comp rx=0 tx=0).stop 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 msgr2=0x7f187014dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f187014dc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f186402c380 tx=0x7f1864004730 comp rx=0 tx=0).stop 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 shutdown_connections 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f185006b500 0x7f185006d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187014d130 0x7f1870148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1870070c50 0x7f187014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 --2- 172.21.15.67:0/3785465806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18580039b0 0x7f187014dc30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 >> 172.21.15.67:0/3785465806 conn(0x7f18700c63f0 msgr2=0x7f18700bf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:36.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 shutdown_connections 2024-02-26T10:38:36.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.053+0000 7f1876800640 1 -- 172.21.15.67:0/3785465806 wait complete. 2024-02-26T10:38:36.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T10:38:36.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.298+0000 7fbe3d600640 1 Processor -- start 2024-02-26T10:38:36.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.298+0000 7fbe3d600640 1 -- start start 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe3d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbe381591b0 0x7fbe38158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe38070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe3d600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe38071410 0x7fbe3806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe3d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe38073010 con 0x7fbe381591b0 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe3d600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3814f490 con 0x7fbe38071410 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe3d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe38072ce0 con 0x7fbe380c0ef0 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.299+0000 7fbe36a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbe381591b0 0x7fbe38158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbe381591b0 0x7fbe38158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37620/0 (socket says 172.21.15.67:37620) 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 -- 172.21.15.67:0/311953563 learned_addr learned my addr 172.21.15.67:0/311953563 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36000640 1 --2- 172.21.15.67:0/311953563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe38070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe37400640 1 --2- 172.21.15.67:0/311953563 >> v2:172.21.15.82:3300/0 conn(0x7fbe38071410 0x7fbe3806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 -- 172.21.15.67:0/311953563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 msgr2=0x7fbe38070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 --2- 172.21.15.67:0/311953563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe38070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 -- 172.21.15.67:0/311953563 >> v2:172.21.15.82:3300/0 conn(0x7fbe38071410 msgr2=0x7fbe3806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 --2- 172.21.15.67:0/311953563 >> v2:172.21.15.82:3300/0 conn(0x7fbe38071410 0x7fbe3806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe36a00640 1 -- 172.21.15.67:0/311953563 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe38155bf0 con 0x7fbe381591b0 2024-02-26T10:38:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.300+0000 7fbe37400640 1 --2- 172.21.15.67:0/311953563 >> v2:172.21.15.82:3300/0 conn(0x7fbe38071410 0x7fbe3806de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.301+0000 7fbe36a00640 1 --2- 172.21.15.67:0/311953563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe381591b0 0x7fbe38158580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe380c3ff0 tx=0x7fbe2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1512f794cc6af4a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.301+0000 7fbe35600640 1 -- 172.21.15.67:0/311953563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2802e070 con 0x7fbe381591b0 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe35600640 1 -- 172.21.15.67:0/311953563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28037070 con 0x7fbe381591b0 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe35600640 1 -- 172.21.15.67:0/311953563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28031a00 con 0x7fbe381591b0 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 -- 172.21.15.67:0/311953563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe381591b0 msgr2=0x7fbe38158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 --2- 172.21.15.67:0/311953563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe381591b0 0x7fbe38158580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe380c3ff0 tx=0x7fbe2802b880 comp rx=0 tx=0).stop 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 -- 172.21.15.67:0/311953563 shutdown_connections 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 --2- 172.21.15.67:0/311953563 >> v2:172.21.15.82:3300/0 conn(0x7fbe38071410 0x7fbe3806de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 --2- 172.21.15.67:0/311953563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe38070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 --2- 172.21.15.67:0/311953563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe381591b0 0x7fbe38158580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 -- 172.21.15.67:0/311953563 >> 172.21.15.67:0/311953563 conn(0x7fbe380c6da0 msgr2=0x7fbe380c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:36.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.302+0000 7fbe3d600640 1 -- 172.21.15.67:0/311953563 shutdown_connections 2024-02-26T10:38:36.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.303+0000 7fbe3d600640 1 -- 172.21.15.67:0/311953563 wait complete. 2024-02-26T10:38:36.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.304+0000 7fbe3d600640 1 Processor -- start 2024-02-26T10:38:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.304+0000 7fbe3d600640 1 -- start start 2024-02-26T10:38:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.305+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 0x7fbe3814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.305+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe381573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.305+0000 7fbe3d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe381591b0 0x7fbe380a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.305+0000 7fbe3d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380a8d20 con 0x7fbe38071410 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.305+0000 7fbe3d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380a8ba0 con 0x7fbe381591b0 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.305+0000 7fbe3d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3806e390 con 0x7fbe380c0ef0 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 0x7fbe3814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 0x7fbe3814f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37626/0 (socket says 172.21.15.67:37626) 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 -- 172.21.15.67:0/3656041488 learned_addr learned my addr 172.21.15.67:0/3656041488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe37400640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe381591b0 0x7fbe380a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36000640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe381573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 -- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 msgr2=0x7fbe381573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe381573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 -- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe381591b0 msgr2=0x7fbe380a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe381591b0 0x7fbe380a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36a00640 1 -- 172.21.15.67:0/3656041488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe381516e0 con 0x7fbe38071410 2024-02-26T10:38:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.306+0000 7fbe36000640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe381573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.307+0000 7fbe36a00640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 0x7fbe3814f8d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe28000c00 tx=0x7fbe280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.307+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28032880 con 0x7fbe38071410 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.307+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28032a40 con 0x7fbe38071410 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.307+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28032e30 con 0x7fbe38071410 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.307+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe381519c0 con 0x7fbe38071410 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.307+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe38151f80 con 0x7fbe38071410 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.309+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe38074a30 con 0x7fbe38071410 2024-02-26T10:38:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.309+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe280453f0 con 0x7fbe38071410 2024-02-26T10:38:36.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.310+0000 7fbe2ea00640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe1406b4d0 0x7fbe1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:36.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.310+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe280abfe0 con 0x7fbe38071410 2024-02-26T10:38:36.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.311+0000 7fbe36000640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe1406b4d0 0x7fbe1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:36.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.312+0000 7fbe36000640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe1406b4d0 0x7fbe1406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe380ac740 tx=0x7fbe20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:36.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.315+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe28040070 con 0x7fbe38071410 2024-02-26T10:38:36.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:36.551+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbe3806eb10 con 0x7fbe38071410 2024-02-26T10:38:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.179+0000 7fbe2ea00640 1 -- 172.21.15.67:0/3656041488 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbe2807aeb0 con 0x7fbe38071410 2024-02-26T10:38:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.182+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe1406b4d0 msgr2=0x7fbe1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.182+0000 7fbe3d600640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe1406b4d0 0x7fbe1406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe380ac740 tx=0x7fbe20008040 comp rx=0 tx=0).stop 2024-02-26T10:38:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.182+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 msgr2=0x7fbe3814f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.182+0000 7fbe3d600640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 0x7fbe3814f8d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe28000c00 tx=0x7fbe280046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 shutdown_connections 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe1406b4d0 0x7fbe1406d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe381591b0 0x7fbe380a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe380c0ef0 0x7fbe381573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 --2- 172.21.15.67:0/3656041488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe38071410 0x7fbe3814f8d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 >> 172.21.15.67:0/3656041488 conn(0x7fbe380c6da0 msgr2=0x7fbe380c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 shutdown_connections 2024-02-26T10:38:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.183+0000 7fbe3d600640 1 -- 172.21.15.67:0/3656041488 wait complete. 2024-02-26T10:38:37.209 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:38:37.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-26T10:38:37.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.431+0000 7fbd8c600640 1 Processor -- start 2024-02-26T10:38:37.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.431+0000 7fbd8c600640 1 -- start start 2024-02-26T10:38:37.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:37.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8c600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd840c1400 0x7fbd8406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:37.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8c600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd84072ce0 con 0x7fbd841591b0 2024-02-26T10:38:37.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8c600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd84073010 con 0x7fbd840c1400 2024-02-26T10:38:37.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8414f490 con 0x7fbd840c0ef0 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd840c1400 0x7fbd8406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd840c1400 0x7fbd8406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37636/0 (socket says 172.21.15.67:37636) 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd8a600640 1 -- 172.21.15.67:0/3612110182 learned_addr learned my addr 172.21.15.67:0/3612110182 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.433+0000 7fbd89200640 1 --2- 172.21.15.67:0/3612110182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.434+0000 7fbd8a600640 1 -- 172.21.15.67:0/3612110182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 msgr2=0x7fbd84158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.434+0000 7fbd8a600640 1 --2- 172.21.15.67:0/3612110182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.434+0000 7fbd8a600640 1 -- 172.21.15.67:0/3612110182 >> v1:172.21.15.82:6789/0 conn(0x7fbd841591b0 legacy=0x7fbd84070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.434+0000 7fbd8a600640 1 -- 172.21.15.67:0/3612110182 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84155bf0 con 0x7fbd840c1400 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.435+0000 7fbd8a600640 1 --2- 172.21.15.67:0/3612110182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd840c1400 0x7fbd8406de50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd84073190 tx=0x7fbd7c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a33090c5063fa0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd83e00640 1 -- 172.21.15.67:0/3612110182 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c035020 con 0x7fbd840c1400 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd83e00640 1 -- 172.21.15.67:0/3612110182 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c031640 con 0x7fbd840c1400 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd83e00640 1 -- 172.21.15.67:0/3612110182 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c02d040 con 0x7fbd840c1400 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 -- 172.21.15.67:0/3612110182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd840c1400 msgr2=0x7fbd8406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 --2- 172.21.15.67:0/3612110182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd840c1400 0x7fbd8406de50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd84073190 tx=0x7fbd7c02b8f0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 -- 172.21.15.67:0/3612110182 shutdown_connections 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 --2- 172.21.15.67:0/3612110182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd840c1400 0x7fbd8406de50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 --2- 172.21.15.67:0/3612110182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 -- 172.21.15.67:0/3612110182 >> 172.21.15.67:0/3612110182 conn(0x7fbd840c6da0 msgr2=0x7fbd840c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.436+0000 7fbd8c600640 1 -- 172.21.15.67:0/3612110182 shutdown_connections 2024-02-26T10:38:37.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.437+0000 7fbd8c600640 1 -- 172.21.15.67:0/3612110182 wait complete. 2024-02-26T10:38:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.438+0000 7fbd8c600640 1 Processor -- start 2024-02-26T10:38:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.438+0000 7fbd8c600640 1 -- start start 2024-02-26T10:38:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd840c1400 0x7fbd841487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 0x7fbd840a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8c600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840a8ba0 con 0x7fbd841591b0 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840715e0 con 0x7fbd840c1400 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840a8d20 con 0x7fbd840c0ef0 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 0x7fbd840a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 0x7fbd840a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37644/0 (socket says 172.21.15.67:37644) 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.439+0000 7fbd8a600640 1 -- 172.21.15.67:0/1159620303 learned_addr learned my addr 172.21.15.67:0/1159620303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd89c00640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd89200640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd840c1400 0x7fbd841487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8a600640 1 -- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 msgr2=0x7fbd84148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8a600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8a600640 1 -- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd840c1400 msgr2=0x7fbd841487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8a600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd840c1400 0x7fbd841487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8a600640 1 -- 172.21.15.67:0/1159620303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8414c250 con 0x7fbd841591b0 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8a600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 0x7fbd840a9910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02b930 tx=0x7fbd7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c040070 con 0x7fbd841591b0 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c0399f0 con 0x7fbd841591b0 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd841eb690 con 0x7fbd841591b0 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.440+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c035040 con 0x7fbd841591b0 2024-02-26T10:38:37.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.441+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd841e2260 con 0x7fbd841591b0 2024-02-26T10:38:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.442+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd841e2430 con 0x7fbd841591b0 2024-02-26T10:38:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.442+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd7c038020 con 0x7fbd841591b0 2024-02-26T10:38:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.443+0000 7fbd81600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbd6406b4d0 0x7fbd6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.443+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd7c0abac0 con 0x7fbd841591b0 2024-02-26T10:38:37.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.444+0000 7fbd89c00640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbd6406b4d0 0x7fbd6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.445+0000 7fbd89c00640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbd6406b4d0 0x7fbd6406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbd84157d10 tx=0x7fbd78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.447+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd8414c250 con 0x7fbd841591b0 2024-02-26T10:38:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:37.687+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbd84074a30 con 0x7fbd841591b0 2024-02-26T10:38:38.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.182+0000 7fbd81600640 1 -- 172.21.15.67:0/1159620303 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd7c07a930 con 0x7fbd841591b0 2024-02-26T10:38:38.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbd6406b4d0 msgr2=0x7fbd6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbd6406b4d0 0x7fbd6406d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbd84157d10 tx=0x7fbd78009040 comp rx=0 tx=0).stop 2024-02-26T10:38:38.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 msgr2=0x7fbd840a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 0x7fbd840a9910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02b930 tx=0x7fbd7c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 shutdown_connections 2024-02-26T10:38:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd841591b0 0x7fbd840a9910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbd6406b4d0 0x7fbd6406d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd840c1400 0x7fbd841487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 --2- 172.21.15.67:0/1159620303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd840c0ef0 0x7fbd84148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 >> 172.21.15.67:0/1159620303 conn(0x7fbd840c6da0 msgr2=0x7fbd8406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 shutdown_connections 2024-02-26T10:38:38.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.186+0000 7fbd8c600640 1 -- 172.21.15.67:0/1159620303 wait complete. 2024-02-26T10:38:38.210 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-26T10:38:38.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:38:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.426+0000 7f9816200640 1 Processor -- start 2024-02-26T10:38:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.427+0000 7f9816200640 1 -- start start 2024-02-26T10:38:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.428+0000 7f9816200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98100c17e0 0x7f98100c3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.428+0000 7f9816200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f98100c7f10 0x7f98100c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.428+0000 7f9816200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9810155970 con 0x7f98100c2660 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.428+0000 7f9816200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98100c2e50 con 0x7f98100c7f10 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.428+0000 7f9816200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98100c3130 con 0x7f98100c17e0 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9815800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f98100c2660 0x7f98100c6330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40578/0 (socket says 172.21.15.67:40578) 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9815800640 1 -- 172.21.15.67:0/4282971529 learned_addr learned my addr 172.21.15.67:0/4282971529 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9807e00640 1 --2- 172.21.15.67:0/4282971529 >> v2:172.21.15.67:3300/0 conn(0x7f98100c7f10 0x7f98100c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9814e00640 1 --2- 172.21.15.67:0/4282971529 >> v2:172.21.15.82:3300/0 conn(0x7f98100c17e0 0x7f98100c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f980fe00640 1 -- 172.21.15.67:0/4282971529 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134777177 0 0) 0x7f9810155970 con 0x7f98100c2660 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f980fe00640 1 -- 172.21.15.67:0/4282971529 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f4003520 con 0x7f98100c2660 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9807e00640 1 -- 172.21.15.67:0/4282971529 >> v2:172.21.15.82:3300/0 conn(0x7f98100c17e0 msgr2=0x7f98100c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9807e00640 1 --2- 172.21.15.67:0/4282971529 >> v2:172.21.15.82:3300/0 conn(0x7f98100c17e0 0x7f98100c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9807e00640 1 -- 172.21.15.67:0/4282971529 >> v1:172.21.15.67:6789/0 conn(0x7f98100c2660 legacy=0x7f98100c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.429+0000 7f9807e00640 1 -- 172.21.15.67:0/4282971529 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98100c3410 con 0x7f98100c7f10 2024-02-26T10:38:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.430+0000 7f9807e00640 1 --2- 172.21.15.67:0/4282971529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c7f10 0x7f98100c1bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f981004f920 tx=0x7f97f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193348f30727cabe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f980fe00640 1 -- 172.21.15.67:0/4282971529 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f80315a0 con 0x7f98100c7f10 2024-02-26T10:38:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f980fe00640 1 -- 172.21.15.67:0/4282971529 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f8037070 con 0x7f98100c7f10 2024-02-26T10:38:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f980fe00640 1 -- 172.21.15.67:0/4282971529 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8031960 con 0x7f98100c7f10 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 -- 172.21.15.67:0/4282971529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c7f10 msgr2=0x7f98100c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 --2- 172.21.15.67:0/4282971529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c7f10 0x7f98100c1bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f981004f920 tx=0x7f97f802b840 comp rx=0 tx=0).stop 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 -- 172.21.15.67:0/4282971529 shutdown_connections 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 --2- 172.21.15.67:0/4282971529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c7f10 0x7f98100c1bc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 --2- 172.21.15.67:0/4282971529 >> v2:172.21.15.82:3300/0 conn(0x7f98100c17e0 0x7f98100c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 -- 172.21.15.67:0/4282971529 >> 172.21.15.67:0/4282971529 conn(0x7f98100c8430 msgr2=0x7f98100c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 -- 172.21.15.67:0/4282971529 shutdown_connections 2024-02-26T10:38:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.431+0000 7f9816200640 1 -- 172.21.15.67:0/4282971529 wait complete. 2024-02-26T10:38:38.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.433+0000 7f9816200640 1 Processor -- start 2024-02-26T10:38:38.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.433+0000 7f9816200640 1 -- start start 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9816200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98100c17e0 0x7f9810157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9816200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 0x7f9810151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9816200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98100c7f10 0x7f98100ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9816200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810152730 con 0x7f98100c2660 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9816200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810155c50 con 0x7f98100c7f10 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9816200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981015f660 con 0x7f98100c17e0 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9807e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 0x7f9810151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9807e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 0x7f9810151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37668/0 (socket says 172.21.15.67:37668) 2024-02-26T10:38:38.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.434+0000 7f9807e00640 1 -- 172.21.15.67:0/3839956885 learned_addr learned my addr 172.21.15.67:0/3839956885 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9815800640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98100c7f10 0x7f98100ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9814e00640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98100c17e0 0x7f9810157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9807e00640 1 -- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98100c17e0 msgr2=0x7f9810157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9807e00640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98100c17e0 0x7f9810157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9807e00640 1 -- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98100c7f10 msgr2=0x7f98100ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9807e00640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98100c7f10 0x7f98100ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9807e00640 1 -- 172.21.15.67:0/3839956885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981014fb30 con 0x7f98100c2660 2024-02-26T10:38:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9807e00640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 0x7f9810151900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009890 tx=0x7f97f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8032630 con 0x7f98100c2660 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f80327f0 con 0x7f98100c2660 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8032be0 con 0x7f98100c2660 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9810157070 con 0x7f98100c2660 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.435+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9810157630 con 0x7f98100c2660 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.437+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98100c3930 con 0x7f98100c2660 2024-02-26T10:38:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.437+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97f804c020 con 0x7f98100c2660 2024-02-26T10:38:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.438+0000 7f980d600640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f97ec06b500 0x7f97ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.438+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97f802f020 con 0x7f98100c2660 2024-02-26T10:38:38.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.439+0000 7f9814e00640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f97ec06b500 0x7f97ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.440+0000 7f9814e00640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f97ec06b500 0x7f97ec06d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9800007380 tx=0x7f980000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.443+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98100c3930 con 0x7f98100c2660 2024-02-26T10:38:38.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.677+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9810074a30 con 0x7f98100c2660 2024-02-26T10:38:38.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.678+0000 7f980d600640 1 -- 172.21.15.67:0/3839956885 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f97f8040070 con 0x7f98100c2660 2024-02-26T10:38:38.681 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:38.681 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:38:38.681 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f97ec06b500 msgr2=0x7f97ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f97ec06b500 0x7f97ec06d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9800007380 tx=0x7f980000b040 comp rx=0 tx=0).stop 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 msgr2=0x7f9810151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 0x7f9810151900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009890 tx=0x7f97f80046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 shutdown_connections 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f97ec06b500 0x7f97ec06d9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98100c7f10 0x7f98100ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98100c2660 0x7f9810151900 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 --2- 172.21.15.67:0/3839956885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98100c17e0 0x7f9810157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 >> 172.21.15.67:0/3839956885 conn(0x7f98100c8430 msgr2=0x7f98100a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 shutdown_connections 2024-02-26T10:38:38.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.681+0000 7f9816200640 1 -- 172.21.15.67:0/3839956885 wait complete. 2024-02-26T10:38:38.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.910+0000 7fbab9000640 1 Processor -- start 2024-02-26T10:38:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.911+0000 7fbab9000640 1 -- start start 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.912+0000 7fbab9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406ac20 0x7fbab40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.912+0000 7fbab9000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbab406b000 0x7fbab406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.912+0000 7fbab9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbab4153610 0x7fbab40abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.912+0000 7fbab9000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab406b460 con 0x7fbab4153610 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.912+0000 7fbab9000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40a9a80 con 0x7fbab406b000 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.912+0000 7fbab9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40a9d60 con 0x7fbab406ac20 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbab4153610 0x7fbab40abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbab4153610 0x7fbab40abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37670/0 (socket says 172.21.15.67:37670) 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 -- 172.21.15.67:0/957071273 learned_addr learned my addr 172.21.15.67:0/957071273 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbaaaa00640 1 --2- 172.21.15.67:0/957071273 >> v2:172.21.15.82:3300/0 conn(0x7fbab406b000 0x7fbab406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3400640 1 --2- 172.21.15.67:0/957071273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406ac20 0x7fbab40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 -- 172.21.15.67:0/957071273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406ac20 msgr2=0x7fbab40a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:38.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 --2- 172.21.15.67:0/957071273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406ac20 0x7fbab40a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 -- 172.21.15.67:0/957071273 >> v2:172.21.15.82:3300/0 conn(0x7fbab406b000 msgr2=0x7fbab406db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:38.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 --2- 172.21.15.67:0/957071273 >> v2:172.21.15.82:3300/0 conn(0x7fbab406b000 0x7fbab406db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.913+0000 7fbab3e00640 1 -- 172.21.15.67:0/957071273 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40ab750 con 0x7fbab4153610 2024-02-26T10:38:38.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.914+0000 7fbab3e00640 1 --2- 172.21.15.67:0/957071273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab4153610 0x7fbab40abf50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbab406ed30 tx=0x7fbaac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c73515a5c10a62 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:38.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab2a00640 1 -- 172.21.15.67:0/957071273 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac02ca70 con 0x7fbab4153610 2024-02-26T10:38:38.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab2a00640 1 -- 172.21.15.67:0/957071273 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac02cc10 con 0x7fbab4153610 2024-02-26T10:38:38.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab2a00640 1 -- 172.21.15.67:0/957071273 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0343f0 con 0x7fbab4153610 2024-02-26T10:38:38.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 -- 172.21.15.67:0/957071273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab4153610 msgr2=0x7fbab40abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 --2- 172.21.15.67:0/957071273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab4153610 0x7fbab40abf50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbab406ed30 tx=0x7fbaac02bed0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 -- 172.21.15.67:0/957071273 shutdown_connections 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 --2- 172.21.15.67:0/957071273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab4153610 0x7fbab40abf50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 --2- 172.21.15.67:0/957071273 >> v2:172.21.15.82:3300/0 conn(0x7fbab406b000 0x7fbab406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 --2- 172.21.15.67:0/957071273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406ac20 0x7fbab40a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 -- 172.21.15.67:0/957071273 >> 172.21.15.67:0/957071273 conn(0x7fbab40c63f0 msgr2=0x7fbab40c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 -- 172.21.15.67:0/957071273 shutdown_connections 2024-02-26T10:38:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.915+0000 7fbab9000640 1 -- 172.21.15.67:0/957071273 wait complete. 2024-02-26T10:38:38.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.917+0000 7fbab9000640 1 Processor -- start 2024-02-26T10:38:38.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.917+0000 7fbab9000640 1 -- start start 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 0x7fbab41f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406b000 0x7fbab4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4153610 0x7fbab41efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab9000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40ac490 con 0x7fbab406ac20 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4153aa0 con 0x7fbab4153610 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4150410 con 0x7fbab406b000 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 0x7fbab41f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 0x7fbab41f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37680/0 (socket says 172.21.15.67:37680) 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.918+0000 7fbab3400640 1 -- 172.21.15.67:0/422590296 learned_addr learned my addr 172.21.15.67:0/422590296 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:38.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3400640 1 -- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406b000 msgr2=0x7fbab4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbaaaa00640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406b000 0x7fbab4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3e00640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4153610 0x7fbab41efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3400640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406b000 0x7fbab4071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3400640 1 -- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4153610 msgr2=0x7fbab41efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3400640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4153610 0x7fbab41efc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3400640 1 -- 172.21.15.67:0/422590296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab414c580 con 0x7fbab406ac20 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab3400640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 0x7fbab41f2910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400d800 tx=0x7fbaa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4018020 con 0x7fbab406ac20 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa40096a0 con 0x7fbab406ac20 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab414c2a0 con 0x7fbab406ac20 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.919+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4009a70 con 0x7fbab406ac20 2024-02-26T10:38:38.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.920+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab41f2110 con 0x7fbab406ac20 2024-02-26T10:38:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.921+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab4074a30 con 0x7fbab406ac20 2024-02-26T10:38:38.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.922+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaa401b020 con 0x7fbab406ac20 2024-02-26T10:38:38.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.922+0000 7fbaabe00640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba9006b4d0 0x7fba9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:38.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.923+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbaa408a7d0 con 0x7fbab406ac20 2024-02-26T10:38:38.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.923+0000 7fbaaaa00640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba9006b4d0 0x7fba9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:38.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.924+0000 7fbaaaa00640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba9006b4d0 0x7fba9006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbab41f2e70 tx=0x7fba9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:38.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:38.929+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaa4022020 con 0x7fbab406ac20 2024-02-26T10:38:39.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.127+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab406b9b0 con 0x7fbab406ac20 2024-02-26T10:38:39.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.129+0000 7fbaabe00640 1 -- 172.21.15.67:0/422590296 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fbaa4058f60 con 0x7fbab406ac20 2024-02-26T10:38:39.132 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:39.132 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.132+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba9006b4d0 msgr2=0x7fba9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.132+0000 7fbab9000640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba9006b4d0 0x7fba9006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbab41f2e70 tx=0x7fba9c008040 comp rx=0 tx=0).stop 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.132+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 msgr2=0x7fbab41f2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.132+0000 7fbab9000640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 0x7fbab41f2910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400d800 tx=0x7fbaa400dcc0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 shutdown_connections 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba9006b4d0 0x7fba9006d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4153610 0x7fbab41efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbab406b000 0x7fbab4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 --2- 172.21.15.67:0/422590296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbab406ac20 0x7fbab41f2910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 >> 172.21.15.67:0/422590296 conn(0x7fbab40c63f0 msgr2=0x7fbab40c1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 shutdown_connections 2024-02-26T10:38:39.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.133+0000 7fbab9000640 1 -- 172.21.15.67:0/422590296 wait complete. 2024-02-26T10:38:39.153 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:38:39.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:39.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.369+0000 7f7f0f200640 1 Processor -- start 2024-02-26T10:38:39.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.369+0000 7f7f0f200640 1 -- start start 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f0806ac20 0x7f7f080a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7f0806b000 0x7f7f0806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0806b460 con 0x7f7f08153610 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080a9a80 con 0x7f7f0806b000 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080a9d60 con 0x7f7f0806ac20 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7f0806b000 0x7f7f0806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7f0806b000 0x7f7f0806db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37696/0 (socket says 172.21.15.67:37696) 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0d400640 1 -- 172.21.15.67:0/1401875445 learned_addr learned my addr 172.21.15.67:0/1401875445 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0de00640 1 --2- 172.21.15.67:0/1401875445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f0806ac20 0x7f7f080a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0d400640 1 -- 172.21.15.67:0/1401875445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f0806ac20 msgr2=0x7f7f080a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.371+0000 7f7f0d400640 1 --2- 172.21.15.67:0/1401875445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f0806ac20 0x7f7f080a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.372+0000 7f7f0d400640 1 -- 172.21.15.67:0/1401875445 >> v1:172.21.15.82:6789/0 conn(0x7f7f08153610 legacy=0x7f7f080abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.372+0000 7f7f0d400640 1 -- 172.21.15.67:0/1401875445 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080ab750 con 0x7f7f0806b000 2024-02-26T10:38:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.373+0000 7f7f0d400640 1 --2- 172.21.15.67:0/1401875445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f0806db00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009870 tx=0x7f7ef802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146ab93b03a312a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.373+0000 7f7f0ca00640 1 -- 172.21.15.67:0/1401875445 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8035020 con 0x7f7f0806b000 2024-02-26T10:38:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.373+0000 7f7f0ca00640 1 -- 172.21.15.67:0/1401875445 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8031640 con 0x7f7f0806b000 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.373+0000 7f7f0ca00640 1 -- 172.21.15.67:0/1401875445 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef802d040 con 0x7f7f0806b000 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.373+0000 7f7f0f200640 1 -- 172.21.15.67:0/1401875445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 msgr2=0x7f7f0806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.373+0000 7f7f0f200640 1 --2- 172.21.15.67:0/1401875445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f0806db00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009870 tx=0x7f7ef802b8f0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.374+0000 7f7f0f200640 1 -- 172.21.15.67:0/1401875445 shutdown_connections 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.374+0000 7f7f0f200640 1 --2- 172.21.15.67:0/1401875445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f0806db00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.374+0000 7f7f0f200640 1 --2- 172.21.15.67:0/1401875445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f0806ac20 0x7f7f080a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.374+0000 7f7f0f200640 1 -- 172.21.15.67:0/1401875445 >> 172.21.15.67:0/1401875445 conn(0x7f7f080c63f0 msgr2=0x7f7f080c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.374+0000 7f7f0f200640 1 -- 172.21.15.67:0/1401875445 shutdown_connections 2024-02-26T10:38:39.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.374+0000 7f7f0f200640 1 -- 172.21.15.67:0/1401875445 wait complete. 2024-02-26T10:38:39.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.375+0000 7f7f0f200640 1 Processor -- start 2024-02-26T10:38:39.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.375+0000 7f7f0f200640 1 -- start start 2024-02-26T10:38:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.376+0000 7f7f0f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f0806ac20 0x7f7f080c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.376+0000 7f7f0f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f081eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.376+0000 7f7f0f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f08153610 0x7f7f08071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.376+0000 7f7f0f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08150410 con 0x7f7f0806b000 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.376+0000 7f7f0f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08153aa0 con 0x7f7f0806ac20 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.376+0000 7f7f0f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08150590 con 0x7f7f08153610 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f081eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f081eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33418/0 (socket says 172.21.15.67:33418) 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 -- 172.21.15.67:0/63811653 learned_addr learned my addr 172.21.15.67:0/63811653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 -- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f08153610 msgr2=0x7f7f08071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0de00640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f0806ac20 0x7f7f080c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0e800640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f08153610 0x7f7f08071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f08153610 0x7f7f08071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 -- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f0806ac20 msgr2=0x7f7f080c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f0806ac20 0x7f7f080c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 -- 172.21.15.67:0/63811653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080c3db0 con 0x7f7f0806b000 2024-02-26T10:38:39.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0de00640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f0806ac20 0x7f7f080c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:39.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.377+0000 7f7f0d400640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f081eb7d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ef802b930 tx=0x7f7ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.378+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef803aa60 con 0x7f7f0806b000 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.378+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef803ac20 con 0x7f7f0806b000 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.378+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80443f0 con 0x7f7f0806b000 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.378+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f080c4050 con 0x7f7f0806b000 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.378+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f080c4610 con 0x7f7f0806b000 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.379+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f08074a30 con 0x7f7f0806b000 2024-02-26T10:38:39.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.380+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ef804b060 con 0x7f7f0806b000 2024-02-26T10:38:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.381+0000 7f7f02000640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eec06b4d0 0x7f7eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.381+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ef80abfa0 con 0x7f7f0806b000 2024-02-26T10:38:39.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.381+0000 7f7f0de00640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eec06b4d0 0x7f7eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.382+0000 7f7f0de00640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eec06b4d0 0x7f7eec06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7efc006fd0 tx=0x7f7efc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:39.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.385+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ef803f070 con 0x7f7f0806b000 2024-02-26T10:38:39.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.580+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f0806b9b0 con 0x7f7eec06b4d0 2024-02-26T10:38:39.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.601+0000 7f7f02000640 1 -- 172.21.15.67:0/63811653 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240572 (secure 0 0 0) 0x7f7f0806b9b0 con 0x7f7eec06b4d0 2024-02-26T10:38:39.607 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:39.608 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:39.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.605+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eec06b4d0 msgr2=0x7f7eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.605+0000 7f7f0f200640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eec06b4d0 0x7f7eec06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7efc006fd0 tx=0x7f7efc008040 comp rx=0 tx=0).stop 2024-02-26T10:38:39.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 msgr2=0x7f7f081eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f081eb7d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ef802b930 tx=0x7f7ef80046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 shutdown_connections 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f08153610 0x7f7f08071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f0806b000 0x7f7f081eb7d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eec06b4d0 0x7f7eec06d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 --2- 172.21.15.67:0/63811653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f0806ac20 0x7f7f080c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 >> 172.21.15.67:0/63811653 conn(0x7f7f080c63f0 msgr2=0x7f7f080bf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 shutdown_connections 2024-02-26T10:38:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.606+0000 7f7f0f200640 1 -- 172.21.15.67:0/63811653 wait complete. 2024-02-26T10:38:39.611 INFO:tasks.ceph.mds.b.smithi082.stderr:2024-02-26T10:38:39.610+0000 7f7f62e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:38:39.627 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-02-26T10:38:38.320875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":160,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219564,"kb_used_data":2536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755156,"statfs":{"total":773094113280,"available":772869279744,"internally_reserved":0,"allocated":2596864,"data_stored":1621731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.176662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.142734+0000","last_change":"2024-02-26T10:38:33.142734+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.142734+0000","last_undegraded":"2024-02-26T10:38:33.142734+0000","last_fullsized":"2024-02-26T10:38:33.142734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150197+0000","last_change":"2024-02-26T10:38:33.150197+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150197+0000","last_undegraded":"2024-02-26T10:38:33.150197+0000","last_fullsized":"2024-02-26T10:38:33.150197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.142536+0000","last_change":"2024-02-26T10:38:33.142536+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.142536+0000","last_undegraded":"2024-02-26T10:38:33.142536+0000","last_fullsized":"2024-02-26T10:38:33.142536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.155358+0000","last_change":"2024-02-26T10:38:33.155358+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.155358+0000","last_undegraded":"2024-02-26T10:38:33.155358+0000","last_fullsized":"2024-02-26T10:38:33.155358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.151268+0000","last_change":"2024-02-26T10:38:33.151268+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.151268+0000","last_undegraded":"2024-02-26T10:38:33.151268+0000","last_fullsized":"2024-02-26T10:38:33.151268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.158213+0000","last_change":"2024-02-26T10:38:33.158213+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.158213+0000","last_undegraded":"2024-02-26T10:38:33.158213+0000","last_fullsized":"2024-02-26T10:38:33.158213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.154185+0000","last_change":"2024-02-26T10:38:33.154185+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.154185+0000","last_undegraded":"2024-02-26T10:38:33.154185+0000","last_fullsized":"2024-02-26T10:38:33.154185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.149009+0000","last_change":"2024-02-26T10:38:33.149009+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.149009+0000","last_undegraded":"2024-02-26T10:38:33.149009+0000","last_fullsized":"2024-02-26T10:38:33.149009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:32.943866+0000","last_change":"2024-02-26T10:38:32.943866+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:32.943866+0000","last_undegraded":"2024-02-26T10:38:32.943866+0000","last_fullsized":"2024-02-26T10:38:32.943866+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.158946+0000","last_change":"2024-02-26T10:38:33.158946+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.158946+0000","last_undegraded":"2024-02-26T10:38:33.158946+0000","last_fullsized":"2024-02-26T10:38:33.158946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150044+0000","last_change":"2024-02-26T10:38:33.150044+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150044+0000","last_undegraded":"2024-02-26T10:38:33.150044+0000","last_fullsized":"2024-02-26T10:38:33.150044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.156822+0000","last_change":"2024-02-26T10:38:33.156822+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.156822+0000","last_undegraded":"2024-02-26T10:38:33.156822+0000","last_fullsized":"2024-02-26T10:38:33.156822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.151427+0000","last_change":"2024-02-26T10:38:33.151427+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.151427+0000","last_undegraded":"2024-02-26T10:38:33.151427+0000","last_fullsized":"2024-02-26T10:38:33.151427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'105","reported_seq":149,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-02-26T10:38:33.153449+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:33.153449+0000","last_peered":"2024-02-26T10:38:33.153449+0000","last_clean":"2024-02-26T10:38:33.153449+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:33.153449+0000","last_undegraded":"2024-02-26T10:38:33.153449+0000","last_fullsized":"2024-02-26T10:38:33.153449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.155202+0000","last_change":"2024-02-26T10:38:33.155202+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.155202+0000","last_undegraded":"2024-02-26T10:38:33.155202+0000","last_fullsized":"2024-02-26T10:38:33.155202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.146206+0000","last_change":"2024-02-26T10:38:33.146206+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.146206+0000","last_undegraded":"2024-02-26T10:38:33.146206+0000","last_fullsized":"2024-02-26T10:38:33.146206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.155122+0000","last_change":"2024-02-26T10:38:33.155122+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.155122+0000","last_undegraded":"2024-02-26T10:38:33.155122+0000","last_fullsized":"2024-02-26T10:38:33.155122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150292+0000","last_change":"2024-02-26T10:38:33.150292+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150292+0000","last_undegraded":"2024-02-26T10:38:33.150292+0000","last_fullsized":"2024-02-26T10:38:33.150292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.147417+0000","last_change":"2024-02-26T10:38:33.147417+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.147417+0000","last_undegraded":"2024-02-26T10:38:33.147417+0000","last_fullsized":"2024-02-26T10:38:33.147417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150742+0000","last_change":"2024-02-26T10:38:33.150742+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150742+0000","last_undegraded":"2024-02-26T10:38:33.150742+0000","last_fullsized":"2024-02-26T10:38:33.150742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.149890+0000","last_change":"2024-02-26T10:38:33.149890+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.149890+0000","last_undegraded":"2024-02-26T10:38:33.149890+0000","last_fullsized":"2024-02-26T10:38:33.149890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.147647+0000","last_change":"2024-02-26T10:38:33.147647+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.147647+0000","last_undegraded":"2024-02-26T10:38:33.147647+0000","last_fullsized":"2024-02-26T10:38:33.147647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150494+0000","last_change":"2024-02-26T10:38:33.150494+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150494+0000","last_undegraded":"2024-02-26T10:38:33.150494+0000","last_fullsized":"2024-02-26T10:38:33.150494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150337+0000","last_change":"2024-02-26T10:38:33.150337+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150337+0000","last_undegraded":"2024-02-26T10:38:33.150337+0000","last_fullsized":"2024-02-26T10:38:33.150337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.146695+0000","last_change":"2024-02-26T10:38:33.146695+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.146695+0000","last_undegraded":"2024-02-26T10:38:33.146695+0000","last_fullsized":"2024-02-26T10:38:33.146695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.148283+0000","last_change":"2024-02-26T10:38:33.148283+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.148283+0000","last_undegraded":"2024-02-26T10:38:33.148283+0000","last_fullsized":"2024-02-26T10:38:33.148283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.148552+0000","last_change":"2024-02-26T10:38:33.148552+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.148552+0000","last_undegraded":"2024-02-26T10:38:33.148552+0000","last_fullsized":"2024-02-26T10:38:33.148552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.142502+0000","last_change":"2024-02-26T10:38:33.142502+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.142502+0000","last_undegraded":"2024-02-26T10:38:33.142502+0000","last_fullsized":"2024-02-26T10:38:33.142502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:32.943866+0000","last_change":"2024-02-26T10:38:32.943866+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:32.943866+0000","last_undegraded":"2024-02-26T10:38:32.943866+0000","last_fullsized":"2024-02-26T10:38:32.943866+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":757760,"data_stored":639438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":49070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":49070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":770048,"data_stored":645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:39.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:39.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.888+0000 7f2ce1c00640 1 Processor -- start 2024-02-26T10:38:39.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.888+0000 7f2ce1c00640 1 -- start start 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2ce1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cdc06ae90 0x7f2cdc1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2ce1c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0c0a40 con 0x7f2cdc072a30 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2ce1c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0c7010 con 0x7f2cdc072ea0 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2ce1c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc073330 con 0x7f2cdc06ae90 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2cdaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cdc06ae90 0x7f2cdc1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2cdaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cdc06ae90 0x7f2cdc1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33420/0 (socket says 172.21.15.67:33420) 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2cdaa00640 1 -- 172.21.15.67:0/2032268965 learned_addr learned my addr 172.21.15.67:0/2032268965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.890+0000 7f2cdb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2cdc072a30 0x7f2cdc155580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54490/0 (socket says 172.21.15.67:54490) 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.891+0000 7f2cdaa00640 1 -- 172.21.15.67:0/2032268965 >> v1:172.21.15.82:6789/0 conn(0x7f2cdc072ea0 legacy=0x7f2cdc154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.891+0000 7f2cdaa00640 1 -- 172.21.15.67:0/2032268965 >> v1:172.21.15.67:6789/0 conn(0x7f2cdc072a30 legacy=0x7f2cdc155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.891+0000 7f2cdaa00640 1 -- 172.21.15.67:0/2032268965 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0c72f0 con 0x7f2cdc06ae90 2024-02-26T10:38:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.891+0000 7f2cd9600640 1 -- 172.21.15.67:0/2032268965 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051330666 0 0) 0x7f2cdc0c0a40 con 0x7f2cdc072a30 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.892+0000 7f2cdaa00640 1 --2- 172.21.15.67:0/2032268965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1521b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2728e3d83e6a0fe7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.892+0000 7f2cd9600640 1 -- 172.21.15.67:0/2032268965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc02cbb0 con 0x7f2cdc06ae90 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.892+0000 7f2cd9600640 1 -- 172.21.15.67:0/2032268965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc02cd50 con 0x7f2cdc06ae90 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.892+0000 7f2cd9600640 1 -- 172.21.15.67:0/2032268965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc034610 con 0x7f2cdc06ae90 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 -- 172.21.15.67:0/2032268965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 msgr2=0x7f2cdc1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/2032268965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1521b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc02c010 comp rx=0 tx=0).stop 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 -- 172.21.15.67:0/2032268965 shutdown_connections 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/2032268965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1521b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 -- 172.21.15.67:0/2032268965 >> 172.21.15.67:0/2032268965 conn(0x7f2cdc0c7c40 msgr2=0x7f2cdc06bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 -- 172.21.15.67:0/2032268965 shutdown_connections 2024-02-26T10:38:39.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.893+0000 7f2ce1c00640 1 -- 172.21.15.67:0/2032268965 wait complete. 2024-02-26T10:38:39.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.894+0000 7f2ce1c00640 1 Processor -- start 2024-02-26T10:38:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 -- start start 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cdc072a30 0x7f2cdc0abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cdc153960 0x7f2cdc14c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c5510 con 0x7f2cdc06ae90 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c0d70 con 0x7f2cdc072a30 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.895+0000 7f2ce1c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c5690 con 0x7f2cdc153960 2024-02-26T10:38:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33426/0 (socket says 172.21.15.67:33426) 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 -- 172.21.15.67:0/3381387094 learned_addr learned my addr 172.21.15.67:0/3381387094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 -- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cdc153960 msgr2=0x7f2cdc14c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cdc153960 0x7f2cdc14c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cda000640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cdc072a30 0x7f2cdc0abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 -- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cdc072a30 msgr2=0x7f2cdc0abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cdc072a30 0x7f2cdc0abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 -- 172.21.15.67:0/3381387094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0aaa40 con 0x7f2cdc06ae90 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cda000640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cdc072a30 0x7f2cdc0abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.896+0000 7f2cdaa00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1ee170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.897+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc043020 con 0x7f2cdc06ae90 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.897+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc03e450 con 0x7f2cdc06ae90 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.897+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc009510 con 0x7f2cdc06ae90 2024-02-26T10:38:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.897+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1dbce0 con 0x7f2cdc06ae90 2024-02-26T10:38:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.897+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc1eec90 con 0x7f2cdc06ae90 2024-02-26T10:38:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.898+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc0aa2a0 con 0x7f2cdc06ae90 2024-02-26T10:38:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.899+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ccc046020 con 0x7f2cdc06ae90 2024-02-26T10:38:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.900+0000 7f2cd2a00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2cb806b4d0 0x7f2cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.900+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ccc0acb40 con 0x7f2cdc06ae90 2024-02-26T10:38:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.900+0000 7f2cda000640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2cb806b4d0 0x7f2cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:39.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.901+0000 7f2cda000640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2cb806b4d0 0x7f2cb806d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc155470 tx=0x7f2cc4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:39.904+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cdc0aa2a0 con 0x7f2cdc06ae90 2024-02-26T10:38:40.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.099+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2cdc074a30 con 0x7f2cb806b4d0 2024-02-26T10:38:40.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.117+0000 7f2cd2a00640 1 -- 172.21.15.67:0/3381387094 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240572 (secure 0 0 0) 0x7f2cdc074a30 con 0x7f2cb806b4d0 2024-02-26T10:38:40.123 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.121+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2cb806b4d0 msgr2=0x7f2cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.121+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2cb806b4d0 0x7f2cb806d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc155470 tx=0x7f2cc4009040 comp rx=0 tx=0).stop 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.121+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 msgr2=0x7f2cdc1ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.121+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1ee170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 shutdown_connections 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cdc153960 0x7f2cdc14c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2cb806b4d0 0x7f2cb806d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cdc072a30 0x7f2cdc0abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 --2- 172.21.15.67:0/3381387094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cdc06ae90 0x7f2cdc1ee170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 >> 172.21.15.67:0/3381387094 conn(0x7f2cdc0c7c40 msgr2=0x7f2cdc14e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 shutdown_connections 2024-02-26T10:38:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.122+0000 7f2ce1c00640 1 -- 172.21.15.67:0/3381387094 wait complete. 2024-02-26T10:38:40.144 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-02-26T10:38:38.320875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":160,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219564,"kb_used_data":2536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755156,"statfs":{"total":773094113280,"available":772869279744,"internally_reserved":0,"allocated":2596864,"data_stored":1621731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.176662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.142734+0000","last_change":"2024-02-26T10:38:33.142734+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.142734+0000","last_undegraded":"2024-02-26T10:38:33.142734+0000","last_fullsized":"2024-02-26T10:38:33.142734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150197+0000","last_change":"2024-02-26T10:38:33.150197+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150197+0000","last_undegraded":"2024-02-26T10:38:33.150197+0000","last_fullsized":"2024-02-26T10:38:33.150197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.142536+0000","last_change":"2024-02-26T10:38:33.142536+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.142536+0000","last_undegraded":"2024-02-26T10:38:33.142536+0000","last_fullsized":"2024-02-26T10:38:33.142536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.155358+0000","last_change":"2024-02-26T10:38:33.155358+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.155358+0000","last_undegraded":"2024-02-26T10:38:33.155358+0000","last_fullsized":"2024-02-26T10:38:33.155358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.151268+0000","last_change":"2024-02-26T10:38:33.151268+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.151268+0000","last_undegraded":"2024-02-26T10:38:33.151268+0000","last_fullsized":"2024-02-26T10:38:33.151268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.158213+0000","last_change":"2024-02-26T10:38:33.158213+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.158213+0000","last_undegraded":"2024-02-26T10:38:33.158213+0000","last_fullsized":"2024-02-26T10:38:33.158213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.154185+0000","last_change":"2024-02-26T10:38:33.154185+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.154185+0000","last_undegraded":"2024-02-26T10:38:33.154185+0000","last_fullsized":"2024-02-26T10:38:33.154185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.149009+0000","last_change":"2024-02-26T10:38:33.149009+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.149009+0000","last_undegraded":"2024-02-26T10:38:33.149009+0000","last_fullsized":"2024-02-26T10:38:33.149009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:32.943866+0000","last_change":"2024-02-26T10:38:32.943866+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:32.943866+0000","last_undegraded":"2024-02-26T10:38:32.943866+0000","last_fullsized":"2024-02-26T10:38:32.943866+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.158946+0000","last_change":"2024-02-26T10:38:33.158946+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.158946+0000","last_undegraded":"2024-02-26T10:38:33.158946+0000","last_fullsized":"2024-02-26T10:38:33.158946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150044+0000","last_change":"2024-02-26T10:38:33.150044+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150044+0000","last_undegraded":"2024-02-26T10:38:33.150044+0000","last_fullsized":"2024-02-26T10:38:33.150044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.156822+0000","last_change":"2024-02-26T10:38:33.156822+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.156822+0000","last_undegraded":"2024-02-26T10:38:33.156822+0000","last_fullsized":"2024-02-26T10:38:33.156822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.151427+0000","last_change":"2024-02-26T10:38:33.151427+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.151427+0000","last_undegraded":"2024-02-26T10:38:33.151427+0000","last_fullsized":"2024-02-26T10:38:33.151427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'105","reported_seq":149,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-02-26T10:38:33.153449+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:33.153449+0000","last_peered":"2024-02-26T10:38:33.153449+0000","last_clean":"2024-02-26T10:38:33.153449+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:33.153449+0000","last_undegraded":"2024-02-26T10:38:33.153449+0000","last_fullsized":"2024-02-26T10:38:33.153449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.155202+0000","last_change":"2024-02-26T10:38:33.155202+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.155202+0000","last_undegraded":"2024-02-26T10:38:33.155202+0000","last_fullsized":"2024-02-26T10:38:33.155202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.146206+0000","last_change":"2024-02-26T10:38:33.146206+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.146206+0000","last_undegraded":"2024-02-26T10:38:33.146206+0000","last_fullsized":"2024-02-26T10:38:33.146206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.155122+0000","last_change":"2024-02-26T10:38:33.155122+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.155122+0000","last_undegraded":"2024-02-26T10:38:33.155122+0000","last_fullsized":"2024-02-26T10:38:33.155122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150292+0000","last_change":"2024-02-26T10:38:33.150292+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150292+0000","last_undegraded":"2024-02-26T10:38:33.150292+0000","last_fullsized":"2024-02-26T10:38:33.150292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.147417+0000","last_change":"2024-02-26T10:38:33.147417+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.147417+0000","last_undegraded":"2024-02-26T10:38:33.147417+0000","last_fullsized":"2024-02-26T10:38:33.147417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150742+0000","last_change":"2024-02-26T10:38:33.150742+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150742+0000","last_undegraded":"2024-02-26T10:38:33.150742+0000","last_fullsized":"2024-02-26T10:38:33.150742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.149890+0000","last_change":"2024-02-26T10:38:33.149890+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.149890+0000","last_undegraded":"2024-02-26T10:38:33.149890+0000","last_fullsized":"2024-02-26T10:38:33.149890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.147647+0000","last_change":"2024-02-26T10:38:33.147647+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.147647+0000","last_undegraded":"2024-02-26T10:38:33.147647+0000","last_fullsized":"2024-02-26T10:38:33.147647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150494+0000","last_change":"2024-02-26T10:38:33.150494+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150494+0000","last_undegraded":"2024-02-26T10:38:33.150494+0000","last_fullsized":"2024-02-26T10:38:33.150494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.150337+0000","last_change":"2024-02-26T10:38:33.150337+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.150337+0000","last_undegraded":"2024-02-26T10:38:33.150337+0000","last_fullsized":"2024-02-26T10:38:33.150337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.146695+0000","last_change":"2024-02-26T10:38:33.146695+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.146695+0000","last_undegraded":"2024-02-26T10:38:33.146695+0000","last_fullsized":"2024-02-26T10:38:33.146695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.148283+0000","last_change":"2024-02-26T10:38:33.148283+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.148283+0000","last_undegraded":"2024-02-26T10:38:33.148283+0000","last_fullsized":"2024-02-26T10:38:33.148283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.148552+0000","last_change":"2024-02-26T10:38:33.148552+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.148552+0000","last_undegraded":"2024-02-26T10:38:33.148552+0000","last_fullsized":"2024-02-26T10:38:33.148552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-02-26T10:38:33.142502+0000","last_change":"2024-02-26T10:38:33.142502+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:33.142502+0000","last_undegraded":"2024-02-26T10:38:33.142502+0000","last_fullsized":"2024-02-26T10:38:33.142502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:34.135370+0000","last_change":"2024-02-26T10:38:34.135370+0000","last_active":"2024-02-26T10:38:34.135370+0000","last_peered":"2024-02-26T10:38:34.135370+0000","last_clean":"2024-02-26T10:38:34.135370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:34.135370+0000","last_undegraded":"2024-02-26T10:38:34.135370+0000","last_fullsized":"2024-02-26T10:38:34.135370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:38:32.943866+0000","last_change":"2024-02-26T10:38:32.943866+0000","last_active":"2024-02-26T10:38:32.943866+0000","last_peered":"2024-02-26T10:38:32.943866+0000","last_clean":"2024-02-26T10:38:32.943866+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:38:32.943866+0000","last_undegraded":"2024-02-26T10:38:32.943866+0000","last_fullsized":"2024-02-26T10:38:32.943866+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":757760,"data_stored":639438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":49070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":163840,"data_stored":49070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":54887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":770048,"data_stored":645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:40.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.409+0000 7f67c7400640 1 Processor -- start 2024-02-26T10:38:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.409+0000 7f67c7400640 1 -- start start 2024-02-26T10:38:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.410+0000 7f67c7400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67c006ea30 0x7f67c006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c00c1070 0x7f67c0072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c7400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c006b100 con 0x7f67c006ac20 2024-02-26T10:38:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c7400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c0151670 con 0x7f67c006ea30 2024-02-26T10:38:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c7400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c006b3e0 con 0x7f67c00c1070 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67c006ea30 0x7f67c006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67bfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c00c1070 0x7f67c0072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67c006ea30 0x7f67c006dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38698/0 (socket says 172.21.15.67:38698) 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.411+0000 7f67c4a00640 1 -- 172.21.15.67:0/534142124 learned_addr learned my addr 172.21.15.67:0/534142124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.412+0000 7f67c4a00640 1 -- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c00c1070 msgr2=0x7f67c0072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.412+0000 7f67c4a00640 1 --2- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c00c1070 0x7f67c0072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.412+0000 7f67c4a00640 1 -- 172.21.15.67:0/534142124 >> v1:172.21.15.82:6789/0 conn(0x7f67c006ac20 legacy=0x7f67c0151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.412+0000 7f67c4a00640 1 -- 172.21.15.67:0/534142124 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c006b6c0 con 0x7f67c006ea30 2024-02-26T10:38:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.413+0000 7f67c4a00640 1 --2- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ea30 0x7f67c006dd80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009870 tx=0x7f67b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ced67619888d0b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.414+0000 7f67bf400640 1 -- 172.21.15.67:0/534142124 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0034070 con 0x7f67c006ea30 2024-02-26T10:38:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.414+0000 7f67bf400640 1 -- 172.21.15.67:0/534142124 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0030840 con 0x7f67c006ea30 2024-02-26T10:38:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.414+0000 7f67bf400640 1 -- 172.21.15.67:0/534142124 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0030c10 con 0x7f67c006ea30 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 -- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ea30 msgr2=0x7f67c006dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 --2- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ea30 0x7f67c006dd80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009870 tx=0x7f67b002b980 comp rx=0 tx=0).stop 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 -- 172.21.15.67:0/534142124 shutdown_connections 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 --2- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c00c1070 0x7f67c0072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 --2- 172.21.15.67:0/534142124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ea30 0x7f67c006dd80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 -- 172.21.15.67:0/534142124 >> 172.21.15.67:0/534142124 conn(0x7f67c00a84a0 msgr2=0x7f67c00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 -- 172.21.15.67:0/534142124 shutdown_connections 2024-02-26T10:38:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.415+0000 7f67c7400640 1 -- 172.21.15.67:0/534142124 wait complete. 2024-02-26T10:38:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.416+0000 7f67c7400640 1 Processor -- start 2024-02-26T10:38:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.417+0000 7f67c7400640 1 -- start start 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.417+0000 7f67c7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ac20 0x7f67c01550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.417+0000 7f67c7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c006ea30 0x7f67c00c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.417+0000 7f67c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 0x7f67c00c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c7400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00c3100 con 0x7f67c00bb830 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c7400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0151290 con 0x7f67c006ac20 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c7400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00c3280 con 0x7f67c006ea30 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 0x7f67c00c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 0x7f67c00c08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33440/0 (socket says 172.21.15.67:33440) 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 -- 172.21.15.67:0/550144716 learned_addr learned my addr 172.21.15.67:0/550144716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c4a00640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ac20 0x7f67c01550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67bfe00640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c006ea30 0x7f67c00c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 -- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c006ea30 msgr2=0x7f67c00c03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c006ea30 0x7f67c00c03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 -- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ac20 msgr2=0x7f67c01550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ac20 0x7f67c01550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c5400640 1 -- 172.21.15.67:0/550144716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00bc9e0 con 0x7f67c00bb830 2024-02-26T10:38:40.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.418+0000 7f67c4a00640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ac20 0x7f67c01550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:40.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.419+0000 7f67c5400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 0x7f67c00c08f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c3d0 tx=0x7f67b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:40.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.419+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8017070 con 0x7f67c00bb830 2024-02-26T10:38:40.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.419+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80052d0 con 0x7f67c00bb830 2024-02-26T10:38:40.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.419+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8003830 con 0x7f67c00bb830 2024-02-26T10:38:40.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.419+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c0155630 con 0x7f67c00bb830 2024-02-26T10:38:40.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.419+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c00bc390 con 0x7f67c00bb830 2024-02-26T10:38:40.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.420+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c0074a30 con 0x7f67c00bb830 2024-02-26T10:38:40.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.421+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67b801a070 con 0x7f67c00bb830 2024-02-26T10:38:40.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.422+0000 7f67bcc00640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f679c06b500 0x7f679c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.422+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67b8089540 con 0x7f67c00bb830 2024-02-26T10:38:40.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.422+0000 7f67c4a00640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f679c06b500 0x7f679c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.424+0000 7f67c4a00640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f679c06b500 0x7f679c06d9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67b0036000 tx=0x7f67b003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:40.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.427+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67b8058470 con 0x7f67c00bb830 2024-02-26T10:38:40.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.624+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67c0154e50 con 0x7f679c06b500 2024-02-26T10:38:40.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.643+0000 7f67bcc00640 1 -- 172.21.15.67:0/550144716 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251969 (secure 0 0 0) 0x7f67c0154e50 con 0x7f679c06b500 2024-02-26T10:38:40.647 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.649+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f679c06b500 msgr2=0x7f679c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.649+0000 7f67c7400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f679c06b500 0x7f679c06d9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67b0036000 tx=0x7f67b003b040 comp rx=0 tx=0).stop 2024-02-26T10:38:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.649+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 msgr2=0x7f67c00c08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.649+0000 7f67c7400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 0x7f67c00c08f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c3d0 tx=0x7f67b800c890 comp rx=0 tx=0).stop 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 shutdown_connections 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67c00bb830 0x7f67c00c08f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c006ea30 0x7f67c00c03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f679c06b500 0x7f679c06d9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 --2- 172.21.15.67:0/550144716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c006ac20 0x7f67c01550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 >> 172.21.15.67:0/550144716 conn(0x7f67c00a84a0 msgr2=0x7f67c00a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 shutdown_connections 2024-02-26T10:38:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.650+0000 7f67c7400640 1 -- 172.21.15.67:0/550144716 wait complete. 2024-02-26T10:38:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-02-26T10:38:40.321642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219204,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755516,"statfs":{"total":773094113280,"available":772869648384,"internally_reserved":0,"allocated":2957312,"data_stored":1779452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.177429"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237702+0000","last_change":"2024-02-26T10:38:35.196643+0000","last_active":"2024-02-26T10:38:37.237702+0000","last_peered":"2024-02-26T10:38:37.237702+0000","last_clean":"2024-02-26T10:38:37.237702+0000","last_became_active":"2024-02-26T10:38:35.195966+0000","last_became_peered":"2024-02-26T10:38:35.195966+0000","last_unstale":"2024-02-26T10:38:37.237702+0000","last_undegraded":"2024-02-26T10:38:37.237702+0000","last_fullsized":"2024-02-26T10:38:37.237702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174389+0000","last_change":"2024-02-26T10:38:35.189849+0000","last_active":"2024-02-26T10:38:36.174389+0000","last_peered":"2024-02-26T10:38:36.174389+0000","last_clean":"2024-02-26T10:38:36.174389+0000","last_became_active":"2024-02-26T10:38:35.189290+0000","last_became_peered":"2024-02-26T10:38:35.189290+0000","last_unstale":"2024-02-26T10:38:36.174389+0000","last_undegraded":"2024-02-26T10:38:36.174389+0000","last_fullsized":"2024-02-26T10:38:36.174389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178614+0000","last_change":"2024-02-26T10:38:35.184041+0000","last_active":"2024-02-26T10:38:36.178614+0000","last_peered":"2024-02-26T10:38:36.178614+0000","last_clean":"2024-02-26T10:38:36.178614+0000","last_became_active":"2024-02-26T10:38:35.181755+0000","last_became_peered":"2024-02-26T10:38:35.181755+0000","last_unstale":"2024-02-26T10:38:36.178614+0000","last_undegraded":"2024-02-26T10:38:36.178614+0000","last_fullsized":"2024-02-26T10:38:36.178614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235190+0000","last_change":"2024-02-26T10:38:35.199057+0000","last_active":"2024-02-26T10:38:37.235190+0000","last_peered":"2024-02-26T10:38:37.235190+0000","last_clean":"2024-02-26T10:38:37.235190+0000","last_became_active":"2024-02-26T10:38:35.198578+0000","last_became_peered":"2024-02-26T10:38:35.198578+0000","last_unstale":"2024-02-26T10:38:37.235190+0000","last_undegraded":"2024-02-26T10:38:37.235190+0000","last_fullsized":"2024-02-26T10:38:37.235190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235490+0000","last_change":"2024-02-26T10:38:35.188780+0000","last_active":"2024-02-26T10:38:37.235490+0000","last_peered":"2024-02-26T10:38:37.235490+0000","last_clean":"2024-02-26T10:38:37.235490+0000","last_became_active":"2024-02-26T10:38:35.187459+0000","last_became_peered":"2024-02-26T10:38:35.187459+0000","last_unstale":"2024-02-26T10:38:37.235490+0000","last_undegraded":"2024-02-26T10:38:37.235490+0000","last_fullsized":"2024-02-26T10:38:37.235490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235076+0000","last_change":"2024-02-26T10:38:35.204704+0000","last_active":"2024-02-26T10:38:37.235076+0000","last_peered":"2024-02-26T10:38:37.235076+0000","last_clean":"2024-02-26T10:38:37.235076+0000","last_became_active":"2024-02-26T10:38:35.204329+0000","last_became_peered":"2024-02-26T10:38:35.204329+0000","last_unstale":"2024-02-26T10:38:37.235076+0000","last_undegraded":"2024-02-26T10:38:37.235076+0000","last_fullsized":"2024-02-26T10:38:37.235076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237988+0000","last_change":"2024-02-26T10:38:35.193948+0000","last_active":"2024-02-26T10:38:37.237988+0000","last_peered":"2024-02-26T10:38:37.237988+0000","last_clean":"2024-02-26T10:38:37.237988+0000","last_became_active":"2024-02-26T10:38:35.192130+0000","last_became_peered":"2024-02-26T10:38:35.192130+0000","last_unstale":"2024-02-26T10:38:37.237988+0000","last_undegraded":"2024-02-26T10:38:37.237988+0000","last_fullsized":"2024-02-26T10:38:37.237988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:56:35.875738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234899+0000","last_change":"2024-02-26T10:38:35.205622+0000","last_active":"2024-02-26T10:38:37.234899+0000","last_peered":"2024-02-26T10:38:37.234899+0000","last_clean":"2024-02-26T10:38:37.234899+0000","last_became_active":"2024-02-26T10:38:35.205359+0000","last_became_peered":"2024-02-26T10:38:35.205359+0000","last_unstale":"2024-02-26T10:38:37.234899+0000","last_undegraded":"2024-02-26T10:38:37.234899+0000","last_fullsized":"2024-02-26T10:38:37.234899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237849+0000","last_change":"2024-02-26T10:38:35.202549+0000","last_active":"2024-02-26T10:38:37.237849+0000","last_peered":"2024-02-26T10:38:37.237849+0000","last_clean":"2024-02-26T10:38:37.237849+0000","last_became_active":"2024-02-26T10:38:35.201568+0000","last_became_peered":"2024-02-26T10:38:35.201568+0000","last_unstale":"2024-02-26T10:38:37.237849+0000","last_undegraded":"2024-02-26T10:38:37.237849+0000","last_fullsized":"2024-02-26T10:38:37.237849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182530+0000","last_change":"2024-02-26T10:38:35.206569+0000","last_active":"2024-02-26T10:38:36.182530+0000","last_peered":"2024-02-26T10:38:36.182530+0000","last_clean":"2024-02-26T10:38:36.182530+0000","last_became_active":"2024-02-26T10:38:35.206285+0000","last_became_peered":"2024-02-26T10:38:35.206285+0000","last_unstale":"2024-02-26T10:38:36.182530+0000","last_undegraded":"2024-02-26T10:38:36.182530+0000","last_fullsized":"2024-02-26T10:38:36.182530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182446+0000","last_change":"2024-02-26T10:38:35.209104+0000","last_active":"2024-02-26T10:38:36.182446+0000","last_peered":"2024-02-26T10:38:36.182446+0000","last_clean":"2024-02-26T10:38:36.182446+0000","last_became_active":"2024-02-26T10:38:35.208841+0000","last_became_peered":"2024-02-26T10:38:35.208841+0000","last_unstale":"2024-02-26T10:38:36.182446+0000","last_undegraded":"2024-02-26T10:38:36.182446+0000","last_fullsized":"2024-02-26T10:38:36.182446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175483+0000","last_change":"2024-02-26T10:38:35.198390+0000","last_active":"2024-02-26T10:38:36.175483+0000","last_peered":"2024-02-26T10:38:36.175483+0000","last_clean":"2024-02-26T10:38:36.175483+0000","last_became_active":"2024-02-26T10:38:35.197895+0000","last_became_peered":"2024-02-26T10:38:35.197895+0000","last_unstale":"2024-02-26T10:38:36.175483+0000","last_undegraded":"2024-02-26T10:38:36.175483+0000","last_fullsized":"2024-02-26T10:38:36.175483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:01.842347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.192330+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.192046+0000","last_became_peered":"2024-02-26T10:38:35.192046+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177330+0000","last_change":"2024-02-26T10:38:35.181873+0000","last_active":"2024-02-26T10:38:36.177330+0000","last_peered":"2024-02-26T10:38:36.177330+0000","last_clean":"2024-02-26T10:38:36.177330+0000","last_became_active":"2024-02-26T10:38:35.180543+0000","last_became_peered":"2024-02-26T10:38:35.180543+0000","last_unstale":"2024-02-26T10:38:36.177330+0000","last_undegraded":"2024-02-26T10:38:36.177330+0000","last_fullsized":"2024-02-26T10:38:36.177330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:04:47.351870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173628+0000","last_change":"2024-02-26T10:38:35.186385+0000","last_active":"2024-02-26T10:38:36.173628+0000","last_peered":"2024-02-26T10:38:36.173628+0000","last_clean":"2024-02-26T10:38:36.173628+0000","last_became_active":"2024-02-26T10:38:35.186103+0000","last_became_peered":"2024-02-26T10:38:35.186103+0000","last_unstale":"2024-02-26T10:38:36.173628+0000","last_undegraded":"2024-02-26T10:38:36.173628+0000","last_fullsized":"2024-02-26T10:38:36.173628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234734+0000","last_change":"2024-02-26T10:38:35.203847+0000","last_active":"2024-02-26T10:38:37.234734+0000","last_peered":"2024-02-26T10:38:37.234734+0000","last_clean":"2024-02-26T10:38:37.234734+0000","last_became_active":"2024-02-26T10:38:35.201819+0000","last_became_peered":"2024-02-26T10:38:35.201819+0000","last_unstale":"2024-02-26T10:38:37.234734+0000","last_undegraded":"2024-02-26T10:38:37.234734+0000","last_fullsized":"2024-02-26T10:38:37.234734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237593+0000","last_change":"2024-02-26T10:38:35.205600+0000","last_active":"2024-02-26T10:38:37.237593+0000","last_peered":"2024-02-26T10:38:37.237593+0000","last_clean":"2024-02-26T10:38:37.237593+0000","last_became_active":"2024-02-26T10:38:35.205272+0000","last_became_peered":"2024-02-26T10:38:35.205272+0000","last_unstale":"2024-02-26T10:38:37.237593+0000","last_undegraded":"2024-02-26T10:38:37.237593+0000","last_fullsized":"2024-02-26T10:38:37.237593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237554+0000","last_change":"2024-02-26T10:38:35.191539+0000","last_active":"2024-02-26T10:38:37.237554+0000","last_peered":"2024-02-26T10:38:37.237554+0000","last_clean":"2024-02-26T10:38:37.237554+0000","last_became_active":"2024-02-26T10:38:35.191258+0000","last_became_peered":"2024-02-26T10:38:35.191258+0000","last_unstale":"2024-02-26T10:38:37.237554+0000","last_undegraded":"2024-02-26T10:38:37.237554+0000","last_fullsized":"2024-02-26T10:38:37.237554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182184+0000","last_change":"2024-02-26T10:38:35.197155+0000","last_active":"2024-02-26T10:38:36.182184+0000","last_peered":"2024-02-26T10:38:36.182184+0000","last_clean":"2024-02-26T10:38:36.182184+0000","last_became_active":"2024-02-26T10:38:35.192752+0000","last_became_peered":"2024-02-26T10:38:35.192752+0000","last_unstale":"2024-02-26T10:38:36.182184+0000","last_undegraded":"2024-02-26T10:38:36.182184+0000","last_fullsized":"2024-02-26T10:38:36.182184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178148+0000","last_change":"2024-02-26T10:38:35.192588+0000","last_active":"2024-02-26T10:38:36.178148+0000","last_peered":"2024-02-26T10:38:36.178148+0000","last_clean":"2024-02-26T10:38:36.178148+0000","last_became_active":"2024-02-26T10:38:35.192244+0000","last_became_peered":"2024-02-26T10:38:35.192244+0000","last_unstale":"2024-02-26T10:38:36.178148+0000","last_undegraded":"2024-02-26T10:38:36.178148+0000","last_fullsized":"2024-02-26T10:38:36.178148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235257+0000","last_change":"2024-02-26T10:38:35.193557+0000","last_active":"2024-02-26T10:38:37.235257+0000","last_peered":"2024-02-26T10:38:37.235257+0000","last_clean":"2024-02-26T10:38:37.235257+0000","last_became_active":"2024-02-26T10:38:35.192814+0000","last_became_peered":"2024-02-26T10:38:35.192814+0000","last_unstale":"2024-02-26T10:38:37.235257+0000","last_undegraded":"2024-02-26T10:38:37.235257+0000","last_fullsized":"2024-02-26T10:38:37.235257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235534+0000","last_change":"2024-02-26T10:38:35.188741+0000","last_active":"2024-02-26T10:38:37.235534+0000","last_peered":"2024-02-26T10:38:37.235534+0000","last_clean":"2024-02-26T10:38:37.235534+0000","last_became_active":"2024-02-26T10:38:35.188296+0000","last_became_peered":"2024-02-26T10:38:35.188296+0000","last_unstale":"2024-02-26T10:38:37.235534+0000","last_undegraded":"2024-02-26T10:38:37.235534+0000","last_fullsized":"2024-02-26T10:38:37.235534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235999+0000","last_change":"2024-02-26T10:38:35.193665+0000","last_active":"2024-02-26T10:38:37.235999+0000","last_peered":"2024-02-26T10:38:37.235999+0000","last_clean":"2024-02-26T10:38:37.235999+0000","last_became_active":"2024-02-26T10:38:35.193314+0000","last_became_peered":"2024-02-26T10:38:35.193314+0000","last_unstale":"2024-02-26T10:38:37.235999+0000","last_undegraded":"2024-02-26T10:38:37.235999+0000","last_fullsized":"2024-02-26T10:38:37.235999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236084+0000","last_change":"2024-02-26T10:38:35.186570+0000","last_active":"2024-02-26T10:38:37.236084+0000","last_peered":"2024-02-26T10:38:37.236084+0000","last_clean":"2024-02-26T10:38:37.236084+0000","last_became_active":"2024-02-26T10:38:35.186248+0000","last_became_peered":"2024-02-26T10:38:35.186248+0000","last_unstale":"2024-02-26T10:38:37.236084+0000","last_undegraded":"2024-02-26T10:38:37.236084+0000","last_fullsized":"2024-02-26T10:38:37.236084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182707+0000","last_change":"2024-02-26T10:38:35.197545+0000","last_active":"2024-02-26T10:38:36.182707+0000","last_peered":"2024-02-26T10:38:36.182707+0000","last_clean":"2024-02-26T10:38:36.182707+0000","last_became_active":"2024-02-26T10:38:35.197220+0000","last_became_peered":"2024-02-26T10:38:35.197220+0000","last_unstale":"2024-02-26T10:38:36.182707+0000","last_undegraded":"2024-02-26T10:38:36.182707+0000","last_fullsized":"2024-02-26T10:38:36.182707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173665+0000","last_change":"2024-02-26T10:38:35.174943+0000","last_active":"2024-02-26T10:38:36.173665+0000","last_peered":"2024-02-26T10:38:36.173665+0000","last_clean":"2024-02-26T10:38:36.173665+0000","last_became_active":"2024-02-26T10:38:35.174618+0000","last_became_peered":"2024-02-26T10:38:35.174618+0000","last_unstale":"2024-02-26T10:38:36.173665+0000","last_undegraded":"2024-02-26T10:38:36.173665+0000","last_fullsized":"2024-02-26T10:38:36.173665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235860+0000","last_change":"2024-02-26T10:38:35.188806+0000","last_active":"2024-02-26T10:38:37.235860+0000","last_peered":"2024-02-26T10:38:37.235860+0000","last_clean":"2024-02-26T10:38:37.235860+0000","last_became_active":"2024-02-26T10:38:35.188539+0000","last_became_peered":"2024-02-26T10:38:35.188539+0000","last_unstale":"2024-02-26T10:38:37.235860+0000","last_undegraded":"2024-02-26T10:38:37.235860+0000","last_fullsized":"2024-02-26T10:38:37.235860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178458+0000","last_change":"2024-02-26T10:38:35.183491+0000","last_active":"2024-02-26T10:38:36.178458+0000","last_peered":"2024-02-26T10:38:36.178458+0000","last_clean":"2024-02-26T10:38:36.178458+0000","last_became_active":"2024-02-26T10:38:35.183153+0000","last_became_peered":"2024-02-26T10:38:35.183153+0000","last_unstale":"2024-02-26T10:38:36.178458+0000","last_undegraded":"2024-02-26T10:38:36.178458+0000","last_fullsized":"2024-02-26T10:38:36.178458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235539+0000","last_change":"2024-02-26T10:38:35.185342+0000","last_active":"2024-02-26T10:38:37.235539+0000","last_peered":"2024-02-26T10:38:37.235539+0000","last_clean":"2024-02-26T10:38:37.235539+0000","last_became_active":"2024-02-26T10:38:35.185067+0000","last_became_peered":"2024-02-26T10:38:35.185067+0000","last_unstale":"2024-02-26T10:38:37.235539+0000","last_undegraded":"2024-02-26T10:38:37.235539+0000","last_fullsized":"2024-02-26T10:38:37.235539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181690+0000","last_change":"2024-02-26T10:38:35.191702+0000","last_active":"2024-02-26T10:38:36.181690+0000","last_peered":"2024-02-26T10:38:36.181690+0000","last_clean":"2024-02-26T10:38:36.181690+0000","last_became_active":"2024-02-26T10:38:35.191416+0000","last_became_peered":"2024-02-26T10:38:35.191416+0000","last_unstale":"2024-02-26T10:38:36.181690+0000","last_undegraded":"2024-02-26T10:38:36.181690+0000","last_fullsized":"2024-02-26T10:38:36.181690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175512+0000","last_change":"2024-02-26T10:38:35.169056+0000","last_active":"2024-02-26T10:38:36.175512+0000","last_peered":"2024-02-26T10:38:36.175512+0000","last_clean":"2024-02-26T10:38:36.175512+0000","last_became_active":"2024-02-26T10:38:35.168640+0000","last_became_peered":"2024-02-26T10:38:35.168640+0000","last_unstale":"2024-02-26T10:38:36.175512+0000","last_undegraded":"2024-02-26T10:38:36.175512+0000","last_fullsized":"2024-02-26T10:38:36.175512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:44.237255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235514+0000","last_change":"2024-02-26T10:38:34.169112+0000","last_active":"2024-02-26T10:38:37.235514+0000","last_peered":"2024-02-26T10:38:37.235514+0000","last_clean":"2024-02-26T10:38:37.235514+0000","last_became_active":"2024-02-26T10:38:34.168796+0000","last_became_peered":"2024-02-26T10:38:34.168796+0000","last_unstale":"2024-02-26T10:38:37.235514+0000","last_undegraded":"2024-02-26T10:38:37.235514+0000","last_fullsized":"2024-02-26T10:38:37.235514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:29:24.719556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178388+0000","last_change":"2024-02-26T10:38:35.187182+0000","last_active":"2024-02-26T10:38:36.178388+0000","last_peered":"2024-02-26T10:38:36.178388+0000","last_clean":"2024-02-26T10:38:36.178388+0000","last_became_active":"2024-02-26T10:38:35.185661+0000","last_became_peered":"2024-02-26T10:38:35.185661+0000","last_unstale":"2024-02-26T10:38:36.178388+0000","last_undegraded":"2024-02-26T10:38:36.178388+0000","last_fullsized":"2024-02-26T10:38:36.178388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181759+0000","last_change":"2024-02-26T10:38:34.179842+0000","last_active":"2024-02-26T10:38:36.181759+0000","last_peered":"2024-02-26T10:38:36.181759+0000","last_clean":"2024-02-26T10:38:36.181759+0000","last_became_active":"2024-02-26T10:38:34.179535+0000","last_became_peered":"2024-02-26T10:38:34.179535+0000","last_unstale":"2024-02-26T10:38:36.181759+0000","last_undegraded":"2024-02-26T10:38:36.181759+0000","last_fullsized":"2024-02-26T10:38:36.181759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235171+0000","last_change":"2024-02-26T10:38:34.167613+0000","last_active":"2024-02-26T10:38:37.235171+0000","last_peered":"2024-02-26T10:38:37.235171+0000","last_clean":"2024-02-26T10:38:37.235171+0000","last_became_active":"2024-02-26T10:38:34.167252+0000","last_became_peered":"2024-02-26T10:38:34.167252+0000","last_unstale":"2024-02-26T10:38:37.235171+0000","last_undegraded":"2024-02-26T10:38:37.235171+0000","last_fullsized":"2024-02-26T10:38:37.235171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235040+0000","last_change":"2024-02-26T10:38:35.185939+0000","last_active":"2024-02-26T10:38:37.235040+0000","last_peered":"2024-02-26T10:38:37.235040+0000","last_clean":"2024-02-26T10:38:37.235040+0000","last_became_active":"2024-02-26T10:38:35.183226+0000","last_became_peered":"2024-02-26T10:38:35.183226+0000","last_unstale":"2024-02-26T10:38:37.235040+0000","last_undegraded":"2024-02-26T10:38:37.235040+0000","last_fullsized":"2024-02-26T10:38:37.235040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236134+0000","last_change":"2024-02-26T10:38:34.186454+0000","last_active":"2024-02-26T10:38:37.236134+0000","last_peered":"2024-02-26T10:38:37.236134+0000","last_clean":"2024-02-26T10:38:37.236134+0000","last_became_active":"2024-02-26T10:38:34.185676+0000","last_became_peered":"2024-02-26T10:38:34.185676+0000","last_unstale":"2024-02-26T10:38:37.236134+0000","last_undegraded":"2024-02-26T10:38:37.236134+0000","last_fullsized":"2024-02-26T10:38:37.236134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173408+0000","last_change":"2024-02-26T10:38:35.176977+0000","last_active":"2024-02-26T10:38:36.173408+0000","last_peered":"2024-02-26T10:38:36.173408+0000","last_clean":"2024-02-26T10:38:36.173408+0000","last_became_active":"2024-02-26T10:38:35.175624+0000","last_became_peered":"2024-02-26T10:38:35.175624+0000","last_unstale":"2024-02-26T10:38:36.173408+0000","last_undegraded":"2024-02-26T10:38:36.173408+0000","last_fullsized":"2024-02-26T10:38:36.173408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182463+0000","last_change":"2024-02-26T10:38:34.188391+0000","last_active":"2024-02-26T10:38:36.182463+0000","last_peered":"2024-02-26T10:38:36.182463+0000","last_clean":"2024-02-26T10:38:36.182463+0000","last_became_active":"2024-02-26T10:38:34.188077+0000","last_became_peered":"2024-02-26T10:38:34.188077+0000","last_unstale":"2024-02-26T10:38:36.182463+0000","last_undegraded":"2024-02-26T10:38:36.182463+0000","last_fullsized":"2024-02-26T10:38:36.182463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236526+0000","last_change":"2024-02-26T10:38:35.199765+0000","last_active":"2024-02-26T10:38:37.236526+0000","last_peered":"2024-02-26T10:38:37.236526+0000","last_clean":"2024-02-26T10:38:37.236526+0000","last_became_active":"2024-02-26T10:38:35.199478+0000","last_became_peered":"2024-02-26T10:38:35.199478+0000","last_unstale":"2024-02-26T10:38:37.236526+0000","last_undegraded":"2024-02-26T10:38:37.236526+0000","last_fullsized":"2024-02-26T10:38:37.236526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236673+0000","last_change":"2024-02-26T10:38:34.189159+0000","last_active":"2024-02-26T10:38:37.236673+0000","last_peered":"2024-02-26T10:38:37.236673+0000","last_clean":"2024-02-26T10:38:37.236673+0000","last_became_active":"2024-02-26T10:38:34.187774+0000","last_became_peered":"2024-02-26T10:38:34.187774+0000","last_unstale":"2024-02-26T10:38:37.236673+0000","last_undegraded":"2024-02-26T10:38:37.236673+0000","last_fullsized":"2024-02-26T10:38:37.236673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178929+0000","last_change":"2024-02-26T10:38:35.198495+0000","last_active":"2024-02-26T10:38:36.178929+0000","last_peered":"2024-02-26T10:38:36.178929+0000","last_clean":"2024-02-26T10:38:36.178929+0000","last_became_active":"2024-02-26T10:38:35.198049+0000","last_became_peered":"2024-02-26T10:38:35.198049+0000","last_unstale":"2024-02-26T10:38:36.178929+0000","last_undegraded":"2024-02-26T10:38:36.178929+0000","last_fullsized":"2024-02-26T10:38:36.178929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173743+0000","last_change":"2024-02-26T10:38:34.182673+0000","last_active":"2024-02-26T10:38:36.173743+0000","last_peered":"2024-02-26T10:38:36.173743+0000","last_clean":"2024-02-26T10:38:36.173743+0000","last_became_active":"2024-02-26T10:38:34.182369+0000","last_became_peered":"2024-02-26T10:38:34.182369+0000","last_unstale":"2024-02-26T10:38:36.173743+0000","last_undegraded":"2024-02-26T10:38:36.173743+0000","last_fullsized":"2024-02-26T10:38:36.173743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175617+0000","last_change":"2024-02-26T10:38:35.193526+0000","last_active":"2024-02-26T10:38:36.175617+0000","last_peered":"2024-02-26T10:38:36.175617+0000","last_clean":"2024-02-26T10:38:36.175617+0000","last_became_active":"2024-02-26T10:38:35.193192+0000","last_became_peered":"2024-02-26T10:38:35.193192+0000","last_unstale":"2024-02-26T10:38:36.175617+0000","last_undegraded":"2024-02-26T10:38:36.175617+0000","last_fullsized":"2024-02-26T10:38:36.175617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:08.969223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182341+0000","last_change":"2024-02-26T10:38:34.178005+0000","last_active":"2024-02-26T10:38:36.182341+0000","last_peered":"2024-02-26T10:38:36.182341+0000","last_clean":"2024-02-26T10:38:36.182341+0000","last_became_active":"2024-02-26T10:38:34.177684+0000","last_became_peered":"2024-02-26T10:38:34.177684+0000","last_unstale":"2024-02-26T10:38:36.182341+0000","last_undegraded":"2024-02-26T10:38:36.182341+0000","last_fullsized":"2024-02-26T10:38:36.182341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.196333+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.191241+0000","last_became_peered":"2024-02-26T10:38:35.191241+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178541+0000","last_change":"2024-02-26T10:38:34.178253+0000","last_active":"2024-02-26T10:38:36.178541+0000","last_peered":"2024-02-26T10:38:36.178541+0000","last_clean":"2024-02-26T10:38:36.178541+0000","last_became_active":"2024-02-26T10:38:34.177880+0000","last_became_peered":"2024-02-26T10:38:34.177880+0000","last_unstale":"2024-02-26T10:38:36.178541+0000","last_undegraded":"2024-02-26T10:38:36.178541+0000","last_fullsized":"2024-02-26T10:38:36.178541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177760+0000","last_change":"2024-02-26T10:38:35.196164+0000","last_active":"2024-02-26T10:38:36.177760+0000","last_peered":"2024-02-26T10:38:36.177760+0000","last_clean":"2024-02-26T10:38:36.177760+0000","last_became_active":"2024-02-26T10:38:35.195846+0000","last_became_peered":"2024-02-26T10:38:35.195846+0000","last_unstale":"2024-02-26T10:38:36.177760+0000","last_undegraded":"2024-02-26T10:38:36.177760+0000","last_fullsized":"2024-02-26T10:38:36.177760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:36.847842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235392+0000","last_change":"2024-02-26T10:38:34.180900+0000","last_active":"2024-02-26T10:38:37.235392+0000","last_peered":"2024-02-26T10:38:37.235392+0000","last_clean":"2024-02-26T10:38:37.235392+0000","last_became_active":"2024-02-26T10:38:34.180316+0000","last_became_peered":"2024-02-26T10:38:34.180316+0000","last_unstale":"2024-02-26T10:38:37.235392+0000","last_undegraded":"2024-02-26T10:38:37.235392+0000","last_fullsized":"2024-02-26T10:38:37.235392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235901+0000","last_change":"2024-02-26T10:38:35.188962+0000","last_active":"2024-02-26T10:38:37.235901+0000","last_peered":"2024-02-26T10:38:37.235901+0000","last_clean":"2024-02-26T10:38:37.235901+0000","last_became_active":"2024-02-26T10:38:35.188506+0000","last_became_peered":"2024-02-26T10:38:35.188506+0000","last_unstale":"2024-02-26T10:38:37.235901+0000","last_undegraded":"2024-02-26T10:38:37.235901+0000","last_fullsized":"2024-02-26T10:38:37.235901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181772+0000","last_change":"2024-02-26T10:38:34.180239+0000","last_active":"2024-02-26T10:38:36.181772+0000","last_peered":"2024-02-26T10:38:36.181772+0000","last_clean":"2024-02-26T10:38:36.181772+0000","last_became_active":"2024-02-26T10:38:34.179109+0000","last_became_peered":"2024-02-26T10:38:34.179109+0000","last_unstale":"2024-02-26T10:38:36.181772+0000","last_undegraded":"2024-02-26T10:38:36.181772+0000","last_fullsized":"2024-02-26T10:38:36.181772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181887+0000","last_change":"2024-02-26T10:38:35.196600+0000","last_active":"2024-02-26T10:38:36.181887+0000","last_peered":"2024-02-26T10:38:36.181887+0000","last_clean":"2024-02-26T10:38:36.181887+0000","last_became_active":"2024-02-26T10:38:35.192063+0000","last_became_peered":"2024-02-26T10:38:35.192063+0000","last_unstale":"2024-02-26T10:38:36.181887+0000","last_undegraded":"2024-02-26T10:38:36.181887+0000","last_fullsized":"2024-02-26T10:38:36.181887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237658+0000","last_change":"2024-02-26T10:38:35.190544+0000","last_active":"2024-02-26T10:38:37.237658+0000","last_peered":"2024-02-26T10:38:37.237658+0000","last_clean":"2024-02-26T10:38:37.237658+0000","last_became_active":"2024-02-26T10:38:35.190066+0000","last_became_peered":"2024-02-26T10:38:35.190066+0000","last_unstale":"2024-02-26T10:38:37.237658+0000","last_undegraded":"2024-02-26T10:38:37.237658+0000","last_fullsized":"2024-02-26T10:38:37.237658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175919+0000","last_change":"2024-02-26T10:38:34.165622+0000","last_active":"2024-02-26T10:38:36.175919+0000","last_peered":"2024-02-26T10:38:36.175919+0000","last_clean":"2024-02-26T10:38:36.175919+0000","last_became_active":"2024-02-26T10:38:34.165165+0000","last_became_peered":"2024-02-26T10:38:34.165165+0000","last_unstale":"2024-02-26T10:38:36.175919+0000","last_undegraded":"2024-02-26T10:38:36.175919+0000","last_fullsized":"2024-02-26T10:38:36.175919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:02.556621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236130+0000","last_change":"2024-02-26T10:38:35.191334+0000","last_active":"2024-02-26T10:38:37.236130+0000","last_peered":"2024-02-26T10:38:37.236130+0000","last_clean":"2024-02-26T10:38:37.236130+0000","last_became_active":"2024-02-26T10:38:35.187187+0000","last_became_peered":"2024-02-26T10:38:35.187187+0000","last_unstale":"2024-02-26T10:38:37.236130+0000","last_undegraded":"2024-02-26T10:38:37.236130+0000","last_fullsized":"2024-02-26T10:38:37.236130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.238041+0000","last_change":"2024-02-26T10:38:35.189775+0000","last_active":"2024-02-26T10:38:37.238041+0000","last_peered":"2024-02-26T10:38:37.238041+0000","last_clean":"2024-02-26T10:38:37.238041+0000","last_became_active":"2024-02-26T10:38:35.189470+0000","last_became_peered":"2024-02-26T10:38:35.189470+0000","last_unstale":"2024-02-26T10:38:37.238041+0000","last_undegraded":"2024-02-26T10:38:37.238041+0000","last_fullsized":"2024-02-26T10:38:37.238041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:56:01.655512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174043+0000","last_change":"2024-02-26T10:38:34.180005+0000","last_active":"2024-02-26T10:38:36.174043+0000","last_peered":"2024-02-26T10:38:36.174043+0000","last_clean":"2024-02-26T10:38:36.174043+0000","last_became_active":"2024-02-26T10:38:34.179694+0000","last_became_peered":"2024-02-26T10:38:34.179694+0000","last_unstale":"2024-02-26T10:38:36.174043+0000","last_undegraded":"2024-02-26T10:38:36.174043+0000","last_fullsized":"2024-02-26T10:38:36.174043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236746+0000","last_change":"2024-02-26T10:38:35.190450+0000","last_active":"2024-02-26T10:38:37.236746+0000","last_peered":"2024-02-26T10:38:37.236746+0000","last_clean":"2024-02-26T10:38:37.236746+0000","last_became_active":"2024-02-26T10:38:35.187303+0000","last_became_peered":"2024-02-26T10:38:35.187303+0000","last_unstale":"2024-02-26T10:38:37.236746+0000","last_undegraded":"2024-02-26T10:38:37.236746+0000","last_fullsized":"2024-02-26T10:38:37.236746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235843+0000","last_change":"2024-02-26T10:38:35.196564+0000","last_active":"2024-02-26T10:38:37.235843+0000","last_peered":"2024-02-26T10:38:37.235843+0000","last_clean":"2024-02-26T10:38:37.235843+0000","last_became_active":"2024-02-26T10:38:35.196286+0000","last_became_peered":"2024-02-26T10:38:35.196286+0000","last_unstale":"2024-02-26T10:38:37.235843+0000","last_undegraded":"2024-02-26T10:38:37.235843+0000","last_fullsized":"2024-02-26T10:38:37.235843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235369+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:37.235369+0000","last_peered":"2024-02-26T10:38:37.235369+0000","last_clean":"2024-02-26T10:38:37.235369+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:37.235369+0000","last_undegraded":"2024-02-26T10:38:37.235369+0000","last_fullsized":"2024-02-26T10:38:37.235369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234740+0000","last_change":"2024-02-26T10:38:34.181165+0000","last_active":"2024-02-26T10:38:37.234740+0000","last_peered":"2024-02-26T10:38:37.234740+0000","last_clean":"2024-02-26T10:38:37.234740+0000","last_became_active":"2024-02-26T10:38:34.180374+0000","last_became_peered":"2024-02-26T10:38:34.180374+0000","last_unstale":"2024-02-26T10:38:37.234740+0000","last_undegraded":"2024-02-26T10:38:37.234740+0000","last_fullsized":"2024-02-26T10:38:37.234740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235370+0000","last_change":"2024-02-26T10:38:35.205573+0000","last_active":"2024-02-26T10:38:37.235370+0000","last_peered":"2024-02-26T10:38:37.235370+0000","last_clean":"2024-02-26T10:38:37.235370+0000","last_became_active":"2024-02-26T10:38:35.205295+0000","last_became_peered":"2024-02-26T10:38:35.205295+0000","last_unstale":"2024-02-26T10:38:37.235370+0000","last_undegraded":"2024-02-26T10:38:37.235370+0000","last_fullsized":"2024-02-26T10:38:37.235370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234753+0000","last_change":"2024-02-26T10:38:34.177423+0000","last_active":"2024-02-26T10:38:37.234753+0000","last_peered":"2024-02-26T10:38:37.234753+0000","last_clean":"2024-02-26T10:38:37.234753+0000","last_became_active":"2024-02-26T10:38:34.175531+0000","last_became_peered":"2024-02-26T10:38:34.175531+0000","last_unstale":"2024-02-26T10:38:37.234753+0000","last_undegraded":"2024-02-26T10:38:37.234753+0000","last_fullsized":"2024-02-26T10:38:37.234753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.176766+0000","last_change":"2024-02-26T10:38:35.182828+0000","last_active":"2024-02-26T10:38:36.176766+0000","last_peered":"2024-02-26T10:38:36.176766+0000","last_clean":"2024-02-26T10:38:36.176766+0000","last_became_active":"2024-02-26T10:38:35.182453+0000","last_became_peered":"2024-02-26T10:38:35.182453+0000","last_unstale":"2024-02-26T10:38:36.176766+0000","last_undegraded":"2024-02-26T10:38:36.176766+0000","last_fullsized":"2024-02-26T10:38:36.176766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:15.325827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173393+0000","last_change":"2024-02-26T10:38:35.180693+0000","last_active":"2024-02-26T10:38:36.173393+0000","last_peered":"2024-02-26T10:38:36.173393+0000","last_clean":"2024-02-26T10:38:36.173393+0000","last_became_active":"2024-02-26T10:38:35.178787+0000","last_became_peered":"2024-02-26T10:38:35.178787+0000","last_unstale":"2024-02-26T10:38:36.173393+0000","last_undegraded":"2024-02-26T10:38:36.173393+0000","last_fullsized":"2024-02-26T10:38:36.173393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235683+0000","last_change":"2024-02-26T10:38:34.184572+0000","last_active":"2024-02-26T10:38:37.235683+0000","last_peered":"2024-02-26T10:38:37.235683+0000","last_clean":"2024-02-26T10:38:37.235683+0000","last_became_active":"2024-02-26T10:38:34.183650+0000","last_became_peered":"2024-02-26T10:38:34.183650+0000","last_unstale":"2024-02-26T10:38:37.235683+0000","last_undegraded":"2024-02-26T10:38:37.235683+0000","last_fullsized":"2024-02-26T10:38:37.235683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175481+0000","last_change":"2024-02-26T10:38:35.186511+0000","last_active":"2024-02-26T10:38:36.175481+0000","last_peered":"2024-02-26T10:38:36.175481+0000","last_clean":"2024-02-26T10:38:36.175481+0000","last_became_active":"2024-02-26T10:38:35.185321+0000","last_became_peered":"2024-02-26T10:38:35.185321+0000","last_unstale":"2024-02-26T10:38:36.175481+0000","last_undegraded":"2024-02-26T10:38:36.175481+0000","last_fullsized":"2024-02-26T10:38:36.175481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:18.018830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174062+0000","last_change":"2024-02-26T10:38:34.165105+0000","last_active":"2024-02-26T10:38:36.174062+0000","last_peered":"2024-02-26T10:38:36.174062+0000","last_clean":"2024-02-26T10:38:36.174062+0000","last_became_active":"2024-02-26T10:38:34.164756+0000","last_became_peered":"2024-02-26T10:38:34.164756+0000","last_unstale":"2024-02-26T10:38:36.174062+0000","last_undegraded":"2024-02-26T10:38:36.174062+0000","last_fullsized":"2024-02-26T10:38:36.174062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237188+0000","last_change":"2024-02-26T10:38:35.191275+0000","last_active":"2024-02-26T10:38:37.237188+0000","last_peered":"2024-02-26T10:38:37.237188+0000","last_clean":"2024-02-26T10:38:37.237188+0000","last_became_active":"2024-02-26T10:38:35.190875+0000","last_became_peered":"2024-02-26T10:38:35.190875+0000","last_unstale":"2024-02-26T10:38:37.237188+0000","last_undegraded":"2024-02-26T10:38:37.237188+0000","last_fullsized":"2024-02-26T10:38:37.237188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237699+0000","last_change":"2024-02-26T10:38:34.176727+0000","last_active":"2024-02-26T10:38:37.237699+0000","last_peered":"2024-02-26T10:38:37.237699+0000","last_clean":"2024-02-26T10:38:37.237699+0000","last_became_active":"2024-02-26T10:38:34.176422+0000","last_became_peered":"2024-02-26T10:38:34.176422+0000","last_unstale":"2024-02-26T10:38:37.237699+0000","last_undegraded":"2024-02-26T10:38:37.237699+0000","last_fullsized":"2024-02-26T10:38:37.237699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174128+0000","last_change":"2024-02-26T10:38:35.174940+0000","last_active":"2024-02-26T10:38:36.174128+0000","last_peered":"2024-02-26T10:38:36.174128+0000","last_clean":"2024-02-26T10:38:36.174128+0000","last_became_active":"2024-02-26T10:38:35.174609+0000","last_became_peered":"2024-02-26T10:38:35.174609+0000","last_unstale":"2024-02-26T10:38:36.174128+0000","last_undegraded":"2024-02-26T10:38:36.174128+0000","last_fullsized":"2024-02-26T10:38:36.174128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174557+0000","last_change":"2024-02-26T10:38:34.176149+0000","last_active":"2024-02-26T10:38:36.174557+0000","last_peered":"2024-02-26T10:38:36.174557+0000","last_clean":"2024-02-26T10:38:36.174557+0000","last_became_active":"2024-02-26T10:38:34.175800+0000","last_became_peered":"2024-02-26T10:38:34.175800+0000","last_unstale":"2024-02-26T10:38:36.174557+0000","last_undegraded":"2024-02-26T10:38:36.174557+0000","last_fullsized":"2024-02-26T10:38:36.174557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236113+0000","last_change":"2024-02-26T10:38:35.191410+0000","last_active":"2024-02-26T10:38:37.236113+0000","last_peered":"2024-02-26T10:38:37.236113+0000","last_clean":"2024-02-26T10:38:37.236113+0000","last_became_active":"2024-02-26T10:38:35.190954+0000","last_became_peered":"2024-02-26T10:38:35.190954+0000","last_unstale":"2024-02-26T10:38:37.236113+0000","last_undegraded":"2024-02-26T10:38:37.236113+0000","last_fullsized":"2024-02-26T10:38:37.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182629+0000","last_change":"2024-02-26T10:38:34.185447+0000","last_active":"2024-02-26T10:38:36.182629+0000","last_peered":"2024-02-26T10:38:36.182629+0000","last_clean":"2024-02-26T10:38:36.182629+0000","last_became_active":"2024-02-26T10:38:34.185054+0000","last_became_peered":"2024-02-26T10:38:34.185054+0000","last_unstale":"2024-02-26T10:38:36.182629+0000","last_undegraded":"2024-02-26T10:38:36.182629+0000","last_fullsized":"2024-02-26T10:38:36.182629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179198+0000","last_change":"2024-02-26T10:38:35.181510+0000","last_active":"2024-02-26T10:38:36.179198+0000","last_peered":"2024-02-26T10:38:36.179198+0000","last_clean":"2024-02-26T10:38:36.179198+0000","last_became_active":"2024-02-26T10:38:35.180830+0000","last_became_peered":"2024-02-26T10:38:35.180830+0000","last_unstale":"2024-02-26T10:38:36.179198+0000","last_undegraded":"2024-02-26T10:38:36.179198+0000","last_fullsized":"2024-02-26T10:38:36.179198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234630+0000","last_change":"2024-02-26T10:38:34.174303+0000","last_active":"2024-02-26T10:38:37.234630+0000","last_peered":"2024-02-26T10:38:37.234630+0000","last_clean":"2024-02-26T10:38:37.234630+0000","last_became_active":"2024-02-26T10:38:34.174006+0000","last_became_peered":"2024-02-26T10:38:34.174006+0000","last_unstale":"2024-02-26T10:38:37.234630+0000","last_undegraded":"2024-02-26T10:38:37.234630+0000","last_fullsized":"2024-02-26T10:38:37.234630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175416+0000","last_change":"2024-02-26T10:38:35.176832+0000","last_active":"2024-02-26T10:38:36.175416+0000","last_peered":"2024-02-26T10:38:36.175416+0000","last_clean":"2024-02-26T10:38:36.175416+0000","last_became_active":"2024-02-26T10:38:35.176305+0000","last_became_peered":"2024-02-26T10:38:35.176305+0000","last_unstale":"2024-02-26T10:38:36.175416+0000","last_undegraded":"2024-02-26T10:38:36.175416+0000","last_fullsized":"2024-02-26T10:38:36.175416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:06.286416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173704+0000","last_change":"2024-02-26T10:38:34.169425+0000","last_active":"2024-02-26T10:38:36.173704+0000","last_peered":"2024-02-26T10:38:36.173704+0000","last_clean":"2024-02-26T10:38:36.173704+0000","last_became_active":"2024-02-26T10:38:34.168217+0000","last_became_peered":"2024-02-26T10:38:34.168217+0000","last_unstale":"2024-02-26T10:38:36.173704+0000","last_undegraded":"2024-02-26T10:38:36.173704+0000","last_fullsized":"2024-02-26T10:38:36.173704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236029+0000","last_change":"2024-02-26T10:38:35.172836+0000","last_active":"2024-02-26T10:38:37.236029+0000","last_peered":"2024-02-26T10:38:37.236029+0000","last_clean":"2024-02-26T10:38:37.236029+0000","last_became_active":"2024-02-26T10:38:35.172554+0000","last_became_peered":"2024-02-26T10:38:35.172554+0000","last_unstale":"2024-02-26T10:38:37.236029+0000","last_undegraded":"2024-02-26T10:38:37.236029+0000","last_fullsized":"2024-02-26T10:38:37.236029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173693+0000","last_change":"2024-02-26T10:38:34.166377+0000","last_active":"2024-02-26T10:38:36.173693+0000","last_peered":"2024-02-26T10:38:36.173693+0000","last_clean":"2024-02-26T10:38:36.173693+0000","last_became_active":"2024-02-26T10:38:34.165963+0000","last_became_peered":"2024-02-26T10:38:34.165963+0000","last_unstale":"2024-02-26T10:38:36.173693+0000","last_undegraded":"2024-02-26T10:38:36.173693+0000","last_fullsized":"2024-02-26T10:38:36.173693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235575+0000","last_change":"2024-02-26T10:38:35.185120+0000","last_active":"2024-02-26T10:38:37.235575+0000","last_peered":"2024-02-26T10:38:37.235575+0000","last_clean":"2024-02-26T10:38:37.235575+0000","last_became_active":"2024-02-26T10:38:35.184832+0000","last_became_peered":"2024-02-26T10:38:35.184832+0000","last_unstale":"2024-02-26T10:38:37.235575+0000","last_undegraded":"2024-02-26T10:38:37.235575+0000","last_fullsized":"2024-02-26T10:38:37.235575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236461+0000","last_change":"2024-02-26T10:38:34.180225+0000","last_active":"2024-02-26T10:38:37.236461+0000","last_peered":"2024-02-26T10:38:37.236461+0000","last_clean":"2024-02-26T10:38:37.236461+0000","last_became_active":"2024-02-26T10:38:34.179909+0000","last_became_peered":"2024-02-26T10:38:34.179909+0000","last_unstale":"2024-02-26T10:38:37.236461+0000","last_undegraded":"2024-02-26T10:38:37.236461+0000","last_fullsized":"2024-02-26T10:38:37.236461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:05:58.590708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182262+0000","last_change":"2024-02-26T10:38:35.188975+0000","last_active":"2024-02-26T10:38:36.182262+0000","last_peered":"2024-02-26T10:38:36.182262+0000","last_clean":"2024-02-26T10:38:36.182262+0000","last_became_active":"2024-02-26T10:38:35.188608+0000","last_became_peered":"2024-02-26T10:38:35.188608+0000","last_unstale":"2024-02-26T10:38:36.182262+0000","last_undegraded":"2024-02-26T10:38:36.182262+0000","last_fullsized":"2024-02-26T10:38:36.182262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236117+0000","last_change":"2024-02-26T10:38:35.196289+0000","last_active":"2024-02-26T10:38:37.236117+0000","last_peered":"2024-02-26T10:38:37.236117+0000","last_clean":"2024-02-26T10:38:37.236117+0000","last_became_active":"2024-02-26T10:38:35.194751+0000","last_became_peered":"2024-02-26T10:38:35.194751+0000","last_unstale":"2024-02-26T10:38:37.236117+0000","last_undegraded":"2024-02-26T10:38:37.236117+0000","last_fullsized":"2024-02-26T10:38:37.236117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236713+0000","last_change":"2024-02-26T10:38:34.186760+0000","last_active":"2024-02-26T10:38:37.236713+0000","last_peered":"2024-02-26T10:38:37.236713+0000","last_clean":"2024-02-26T10:38:37.236713+0000","last_became_active":"2024-02-26T10:38:34.186428+0000","last_became_peered":"2024-02-26T10:38:34.186428+0000","last_unstale":"2024-02-26T10:38:37.236713+0000","last_undegraded":"2024-02-26T10:38:37.236713+0000","last_fullsized":"2024-02-26T10:38:37.236713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236518+0000","last_change":"2024-02-26T10:38:35.202616+0000","last_active":"2024-02-26T10:38:37.236518+0000","last_peered":"2024-02-26T10:38:37.236518+0000","last_clean":"2024-02-26T10:38:37.236518+0000","last_became_active":"2024-02-26T10:38:35.201498+0000","last_became_peered":"2024-02-26T10:38:35.201498+0000","last_unstale":"2024-02-26T10:38:37.236518+0000","last_undegraded":"2024-02-26T10:38:37.236518+0000","last_fullsized":"2024-02-26T10:38:37.236518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235996+0000","last_change":"2024-02-26T10:38:34.186745+0000","last_active":"2024-02-26T10:38:37.235996+0000","last_peered":"2024-02-26T10:38:37.235996+0000","last_clean":"2024-02-26T10:38:37.235996+0000","last_became_active":"2024-02-26T10:38:34.186470+0000","last_became_peered":"2024-02-26T10:38:34.186470+0000","last_unstale":"2024-02-26T10:38:37.235996+0000","last_undegraded":"2024-02-26T10:38:37.235996+0000","last_fullsized":"2024-02-26T10:38:37.235996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178984+0000","last_change":"2024-02-26T10:38:35.206276+0000","last_active":"2024-02-26T10:38:36.178984+0000","last_peered":"2024-02-26T10:38:36.178984+0000","last_clean":"2024-02-26T10:38:36.178984+0000","last_became_active":"2024-02-26T10:38:35.205932+0000","last_became_peered":"2024-02-26T10:38:35.205932+0000","last_unstale":"2024-02-26T10:38:36.178984+0000","last_undegraded":"2024-02-26T10:38:36.178984+0000","last_fullsized":"2024-02-26T10:38:36.178984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235907+0000","last_change":"2024-02-26T10:38:34.165440+0000","last_active":"2024-02-26T10:38:37.235907+0000","last_peered":"2024-02-26T10:38:37.235907+0000","last_clean":"2024-02-26T10:38:37.235907+0000","last_became_active":"2024-02-26T10:38:34.165164+0000","last_became_peered":"2024-02-26T10:38:34.165164+0000","last_unstale":"2024-02-26T10:38:37.235907+0000","last_undegraded":"2024-02-26T10:38:37.235907+0000","last_fullsized":"2024-02-26T10:38:37.235907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235740+0000","last_change":"2024-02-26T10:38:35.199753+0000","last_active":"2024-02-26T10:38:37.235740+0000","last_peered":"2024-02-26T10:38:37.235740+0000","last_clean":"2024-02-26T10:38:37.235740+0000","last_became_active":"2024-02-26T10:38:35.199470+0000","last_became_peered":"2024-02-26T10:38:35.199470+0000","last_unstale":"2024-02-26T10:38:37.235740+0000","last_undegraded":"2024-02-26T10:38:37.235740+0000","last_fullsized":"2024-02-26T10:38:37.235740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179016+0000","last_change":"2024-02-26T10:38:34.184020+0000","last_active":"2024-02-26T10:38:36.179016+0000","last_peered":"2024-02-26T10:38:36.179016+0000","last_clean":"2024-02-26T10:38:36.179016+0000","last_became_active":"2024-02-26T10:38:34.183497+0000","last_became_peered":"2024-02-26T10:38:34.183497+0000","last_unstale":"2024-02-26T10:38:36.179016+0000","last_undegraded":"2024-02-26T10:38:36.179016+0000","last_fullsized":"2024-02-26T10:38:36.179016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:25:08.410476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":74327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":819200,"data_stored":665243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:40.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-02-26T10:38:40.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.927+0000 7fbea7200640 1 Processor -- start 2024-02-26T10:38:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.927+0000 7fbea7200640 1 -- start start 2024-02-26T10:38:40.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea8072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbea81561a0 0x7fbea8156630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea8156b70 con 0x7fbea806e580 2024-02-26T10:38:40.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea806eac0 con 0x7fbea81561a0 2024-02-26T10:38:40.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea8152f00 con 0x7fbea81500d0 2024-02-26T10:38:40.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbea81561a0 0x7fbea8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbea81561a0 0x7fbea8156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33454/0 (socket says 172.21.15.67:33454) 2024-02-26T10:38:40.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea6800640 1 -- 172.21.15.67:0/1027986747 learned_addr learned my addr 172.21.15.67:0/1027986747 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.929+0000 7fbea5e00640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea8072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.930+0000 7fbea6800640 1 -- 172.21.15.67:0/1027986747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 msgr2=0x7fbea8072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.930+0000 7fbea6800640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea8072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.930+0000 7fbea6800640 1 -- 172.21.15.67:0/1027986747 >> v1:172.21.15.82:6789/0 conn(0x7fbea806e580 legacy=0x7fbea80c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.930+0000 7fbea6800640 1 -- 172.21.15.67:0/1027986747 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea81531e0 con 0x7fbea81561a0 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.930+0000 7fbea5e00640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea8072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.931+0000 7fbea6800640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea8156630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009e50 tx=0x7fbea002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5624dbc3c7033b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.931+0000 7fbea5400640 1 -- 172.21.15.67:0/1027986747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea002c9f0 con 0x7fbea81561a0 2024-02-26T10:38:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.931+0000 7fbea5400640 1 -- 172.21.15.67:0/1027986747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea002cb90 con 0x7fbea81561a0 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.931+0000 7fbea5400640 1 -- 172.21.15.67:0/1027986747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea00343f0 con 0x7fbea81561a0 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 -- 172.21.15.67:0/1027986747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 msgr2=0x7fbea8156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea8156630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009e50 tx=0x7fbea002be50 comp rx=0 tx=0).stop 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 -- 172.21.15.67:0/1027986747 shutdown_connections 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea8156630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 --2- 172.21.15.67:0/1027986747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea8072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 -- 172.21.15.67:0/1027986747 >> 172.21.15.67:0/1027986747 conn(0x7fbea80a84a0 msgr2=0x7fbea80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 -- 172.21.15.67:0/1027986747 shutdown_connections 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.932+0000 7fbea7200640 1 -- 172.21.15.67:0/1027986747 wait complete. 2024-02-26T10:38:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.933+0000 7fbea7200640 1 Processor -- start 2024-02-26T10:38:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.934+0000 7fbea7200640 1 -- start start 2024-02-26T10:38:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea806e580 0x7fbea80bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea813b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea813b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80c37d0 con 0x7fbea81561a0 2024-02-26T10:38:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea8152bb0 con 0x7fbea806e580 2024-02-26T10:38:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80c3950 con 0x7fbea81500d0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea813b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea813b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33466/0 (socket says 172.21.15.67:33466) 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 -- 172.21.15.67:0/1289960353 learned_addr learned my addr 172.21.15.67:0/1289960353 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea813b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea5e00640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea806e580 0x7fbea80bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 -- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 msgr2=0x7fbea813b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea813b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 -- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea806e580 msgr2=0x7fbea80bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea806e580 0x7fbea80bfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea6800640 1 -- 172.21.15.67:0/1289960353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80c2ae0 con 0x7fbea81561a0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbe9fe00640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea813b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.935+0000 7fbea5e00640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea806e580 0x7fbea80bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.936+0000 7fbea6800640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea813b8c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbea0036040 tx=0x7fbea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.936+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0043020 con 0x7fbea81561a0 2024-02-26T10:38:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.936+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea003e450 con 0x7fbea81561a0 2024-02-26T10:38:40.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.936+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea003e820 con 0x7fbea81561a0 2024-02-26T10:38:40.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.936+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea8154b20 con 0x7fbea81561a0 2024-02-26T10:38:40.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.936+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea80c2800 con 0x7fbea81561a0 2024-02-26T10:38:40.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.937+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea8074a30 con 0x7fbea81561a0 2024-02-26T10:38:40.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.938+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbea0005d10 con 0x7fbea81561a0 2024-02-26T10:38:40.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.939+0000 7fbe9e000640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe8406b500 0x7fbe8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:40.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.939+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea00ac700 con 0x7fbea81561a0 2024-02-26T10:38:40.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.940+0000 7fbea5e00640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe8406b500 0x7fbe8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:40.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.940+0000 7fbea5e00640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe8406b500 0x7fbe8406d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:40.944+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbea004d020 con 0x7fbea81561a0 2024-02-26T10:38:41.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.208+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbea80c1be0 con 0x7fbea81561a0 2024-02-26T10:38:41.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.213+0000 7fbe9e000640 1 -- 172.21.15.67:0/1289960353 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1167 (secure 0 0 0) 0x7fbea007b610 con 0x7fbea81561a0 2024-02-26T10:38:41.216 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:41.216 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":26,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1708943907,"num_in_osds":8,"osd_in_since":1708943898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":26},{"state_name":"active+clean","count":7}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":590368,"bytes_used":224833536,"bytes_avail":772869279744,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.2680412232875824,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4264}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-02-26T10:38:15.048545+0000","services":{}},"progress_events":{}} 2024-02-26T10:38:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.215+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe8406b500 msgr2=0x7fbe8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.215+0000 7fbea7200640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe8406b500 0x7fbe8406d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98008040 comp rx=0 tx=0).stop 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.215+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 msgr2=0x7fbea813b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.215+0000 7fbea7200640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea813b8c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbea0036040 tx=0x7fbea0004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 shutdown_connections 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbea81561a0 0x7fbea813b8c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbea81500d0 0x7fbea813b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbe8406b500 0x7fbe8406d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 --2- 172.21.15.67:0/1289960353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbea806e580 0x7fbea80bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 >> 172.21.15.67:0/1289960353 conn(0x7fbea80a84a0 msgr2=0x7fbea80a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 shutdown_connections 2024-02-26T10:38:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.216+0000 7fbea7200640 1 -- 172.21.15.67:0/1289960353 wait complete. 2024-02-26T10:38:41.238 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-02-26T10:38:41.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.450+0000 7fae9d800640 1 Processor -- start 2024-02-26T10:38:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.450+0000 7fae9d800640 1 -- start start 2024-02-26T10:38:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.451+0000 7fae9d800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae98150300 0x7fae980c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.451+0000 7fae9d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae980c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.451+0000 7fae9d800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae98152e50 con 0x7fae9806e410 2024-02-26T10:38:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.451+0000 7fae9d800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae981506e0 con 0x7fae98150300 2024-02-26T10:38:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.451+0000 7fae9d800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae98153a60 con 0x7fae98156230 2024-02-26T10:38:41.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.452+0000 7fae96a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae98150300 0x7fae980c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.452+0000 7fae97400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae980c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.452+0000 7fae97400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae980c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39328/0 (socket says 172.21.15.67:39328) 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.452+0000 7fae97400640 1 -- 172.21.15.67:0/1353554328 learned_addr learned my addr 172.21.15.67:0/1353554328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.453+0000 7fae96a00640 1 -- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 msgr2=0x7fae980c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.453+0000 7fae96a00640 1 --2- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae980c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.453+0000 7fae96a00640 1 -- 172.21.15.67:0/1353554328 >> v1:172.21.15.82:6789/0 conn(0x7fae9806e410 legacy=0x7fae980aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.453+0000 7fae96a00640 1 -- 172.21.15.67:0/1353554328 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98153130 con 0x7fae98150300 2024-02-26T10:38:41.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.453+0000 7fae97400640 1 --2- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae980c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:41.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.454+0000 7fae96a00640 1 --2- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98150300 0x7fae980c2ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae880097a0 tx=0x7fae8802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8988d4bac6a2aa2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:41.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae96000640 1 -- 172.21.15.67:0/1353554328 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88039070 con 0x7fae98150300 2024-02-26T10:38:41.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae96000640 1 -- 172.21.15.67:0/1353554328 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880316f0 con 0x7fae98150300 2024-02-26T10:38:41.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae96000640 1 -- 172.21.15.67:0/1353554328 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88031ac0 con 0x7fae98150300 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 -- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98150300 msgr2=0x7fae980c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 --2- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98150300 0x7fae980c2ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fae880097a0 tx=0x7fae8802b790 comp rx=0 tx=0).stop 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 -- 172.21.15.67:0/1353554328 shutdown_connections 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 --2- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae980c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 --2- 172.21.15.67:0/1353554328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98150300 0x7fae980c2ff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 -- 172.21.15.67:0/1353554328 >> 172.21.15.67:0/1353554328 conn(0x7fae980a84a0 msgr2=0x7fae980a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.455+0000 7fae9d800640 1 -- 172.21.15.67:0/1353554328 shutdown_connections 2024-02-26T10:38:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.456+0000 7fae9d800640 1 -- 172.21.15.67:0/1353554328 wait complete. 2024-02-26T10:38:41.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.457+0000 7fae9d800640 1 Processor -- start 2024-02-26T10:38:41.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.457+0000 7fae9d800640 1 -- start start 2024-02-26T10:38:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae9d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9806e410 0x7fae9814d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae9d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 0x7fae980bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae9d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae981eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae9d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9806dd80 con 0x7fae98150300 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae9d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae981536f0 con 0x7fae9806e410 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae9d800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae980ab2b0 con 0x7fae98156230 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae8e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 0x7fae980bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae8e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 0x7fae980bfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33470/0 (socket says 172.21.15.67:33470) 2024-02-26T10:38:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.458+0000 7fae8e000640 1 -- 172.21.15.67:0/2778956580 learned_addr learned my addr 172.21.15.67:0/2778956580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:41.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae97400640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae981eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:41.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae96a00640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9806e410 0x7fae9814d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8e000640 1 -- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 msgr2=0x7fae981eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8e000640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae981eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8e000640 1 -- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9806e410 msgr2=0x7fae9814d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8e000640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9806e410 0x7fae9814d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8e000640 1 -- 172.21.15.67:0/2778956580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae980c0680 con 0x7fae98150300 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae96a00640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9806e410 0x7fae9814d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae97400640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae981eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8e000640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 0x7fae980bfb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c470 tx=0x7fae80001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.459+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae800052d0 con 0x7fae98150300 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.460+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80005470 con 0x7fae98150300 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.460+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae981ee6e0 con 0x7fae98150300 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.460+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80007260 con 0x7fae98150300 2024-02-26T10:38:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.460+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae981ee980 con 0x7fae98150300 2024-02-26T10:38:41.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.461+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae980c3890 con 0x7fae98150300 2024-02-26T10:38:41.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.462+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae80007400 con 0x7fae98150300 2024-02-26T10:38:41.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.463+0000 7fae8f400640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae7406b500 0x7fae7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:41.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.463+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae80089720 con 0x7fae98150300 2024-02-26T10:38:41.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.463+0000 7fae96a00640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae7406b500 0x7fae7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:41.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.464+0000 7fae96a00640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae7406b500 0x7fae7406d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae88036000 tx=0x7fae88008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:41.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.467+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae980c3890 con 0x7fae98150300 2024-02-26T10:38:41.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.661+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae98074a30 con 0x7fae7406b500 2024-02-26T10:38:41.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.680+0000 7fae8f400640 1 -- 172.21.15.67:0/2778956580 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251969 (secure 0 0 0) 0x7fae98074a30 con 0x7fae7406b500 2024-02-26T10:38:41.683 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:41.687 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:41.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae7406b500 msgr2=0x7fae7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae7406b500 0x7fae7406d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae88036000 tx=0x7fae88008030 comp rx=0 tx=0).stop 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 msgr2=0x7fae980bfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 0x7fae980bfb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c470 tx=0x7fae80001e70 comp rx=0 tx=0).stop 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 shutdown_connections 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98156230 0x7fae981eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae98150300 0x7fae980bfb50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae7406b500 0x7fae7406d9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 --2- 172.21.15.67:0/2778956580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9806e410 0x7fae9814d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 >> 172.21.15.67:0/2778956580 conn(0x7fae980a84a0 msgr2=0x7fae980a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:41.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 shutdown_connections 2024-02-26T10:38:41.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:41.686+0000 7fae9d800640 1 -- 172.21.15.67:0/2778956580 wait complete. 2024-02-26T10:38:41.708 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-02-26T10:38:40.321642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219204,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755516,"statfs":{"total":773094113280,"available":772869648384,"internally_reserved":0,"allocated":2957312,"data_stored":1779452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.177429"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237702+0000","last_change":"2024-02-26T10:38:35.196643+0000","last_active":"2024-02-26T10:38:37.237702+0000","last_peered":"2024-02-26T10:38:37.237702+0000","last_clean":"2024-02-26T10:38:37.237702+0000","last_became_active":"2024-02-26T10:38:35.195966+0000","last_became_peered":"2024-02-26T10:38:35.195966+0000","last_unstale":"2024-02-26T10:38:37.237702+0000","last_undegraded":"2024-02-26T10:38:37.237702+0000","last_fullsized":"2024-02-26T10:38:37.237702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174389+0000","last_change":"2024-02-26T10:38:35.189849+0000","last_active":"2024-02-26T10:38:36.174389+0000","last_peered":"2024-02-26T10:38:36.174389+0000","last_clean":"2024-02-26T10:38:36.174389+0000","last_became_active":"2024-02-26T10:38:35.189290+0000","last_became_peered":"2024-02-26T10:38:35.189290+0000","last_unstale":"2024-02-26T10:38:36.174389+0000","last_undegraded":"2024-02-26T10:38:36.174389+0000","last_fullsized":"2024-02-26T10:38:36.174389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178614+0000","last_change":"2024-02-26T10:38:35.184041+0000","last_active":"2024-02-26T10:38:36.178614+0000","last_peered":"2024-02-26T10:38:36.178614+0000","last_clean":"2024-02-26T10:38:36.178614+0000","last_became_active":"2024-02-26T10:38:35.181755+0000","last_became_peered":"2024-02-26T10:38:35.181755+0000","last_unstale":"2024-02-26T10:38:36.178614+0000","last_undegraded":"2024-02-26T10:38:36.178614+0000","last_fullsized":"2024-02-26T10:38:36.178614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235190+0000","last_change":"2024-02-26T10:38:35.199057+0000","last_active":"2024-02-26T10:38:37.235190+0000","last_peered":"2024-02-26T10:38:37.235190+0000","last_clean":"2024-02-26T10:38:37.235190+0000","last_became_active":"2024-02-26T10:38:35.198578+0000","last_became_peered":"2024-02-26T10:38:35.198578+0000","last_unstale":"2024-02-26T10:38:37.235190+0000","last_undegraded":"2024-02-26T10:38:37.235190+0000","last_fullsized":"2024-02-26T10:38:37.235190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235490+0000","last_change":"2024-02-26T10:38:35.188780+0000","last_active":"2024-02-26T10:38:37.235490+0000","last_peered":"2024-02-26T10:38:37.235490+0000","last_clean":"2024-02-26T10:38:37.235490+0000","last_became_active":"2024-02-26T10:38:35.187459+0000","last_became_peered":"2024-02-26T10:38:35.187459+0000","last_unstale":"2024-02-26T10:38:37.235490+0000","last_undegraded":"2024-02-26T10:38:37.235490+0000","last_fullsized":"2024-02-26T10:38:37.235490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235076+0000","last_change":"2024-02-26T10:38:35.204704+0000","last_active":"2024-02-26T10:38:37.235076+0000","last_peered":"2024-02-26T10:38:37.235076+0000","last_clean":"2024-02-26T10:38:37.235076+0000","last_became_active":"2024-02-26T10:38:35.204329+0000","last_became_peered":"2024-02-26T10:38:35.204329+0000","last_unstale":"2024-02-26T10:38:37.235076+0000","last_undegraded":"2024-02-26T10:38:37.235076+0000","last_fullsized":"2024-02-26T10:38:37.235076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237988+0000","last_change":"2024-02-26T10:38:35.193948+0000","last_active":"2024-02-26T10:38:37.237988+0000","last_peered":"2024-02-26T10:38:37.237988+0000","last_clean":"2024-02-26T10:38:37.237988+0000","last_became_active":"2024-02-26T10:38:35.192130+0000","last_became_peered":"2024-02-26T10:38:35.192130+0000","last_unstale":"2024-02-26T10:38:37.237988+0000","last_undegraded":"2024-02-26T10:38:37.237988+0000","last_fullsized":"2024-02-26T10:38:37.237988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:56:35.875738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234899+0000","last_change":"2024-02-26T10:38:35.205622+0000","last_active":"2024-02-26T10:38:37.234899+0000","last_peered":"2024-02-26T10:38:37.234899+0000","last_clean":"2024-02-26T10:38:37.234899+0000","last_became_active":"2024-02-26T10:38:35.205359+0000","last_became_peered":"2024-02-26T10:38:35.205359+0000","last_unstale":"2024-02-26T10:38:37.234899+0000","last_undegraded":"2024-02-26T10:38:37.234899+0000","last_fullsized":"2024-02-26T10:38:37.234899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237849+0000","last_change":"2024-02-26T10:38:35.202549+0000","last_active":"2024-02-26T10:38:37.237849+0000","last_peered":"2024-02-26T10:38:37.237849+0000","last_clean":"2024-02-26T10:38:37.237849+0000","last_became_active":"2024-02-26T10:38:35.201568+0000","last_became_peered":"2024-02-26T10:38:35.201568+0000","last_unstale":"2024-02-26T10:38:37.237849+0000","last_undegraded":"2024-02-26T10:38:37.237849+0000","last_fullsized":"2024-02-26T10:38:37.237849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182530+0000","last_change":"2024-02-26T10:38:35.206569+0000","last_active":"2024-02-26T10:38:36.182530+0000","last_peered":"2024-02-26T10:38:36.182530+0000","last_clean":"2024-02-26T10:38:36.182530+0000","last_became_active":"2024-02-26T10:38:35.206285+0000","last_became_peered":"2024-02-26T10:38:35.206285+0000","last_unstale":"2024-02-26T10:38:36.182530+0000","last_undegraded":"2024-02-26T10:38:36.182530+0000","last_fullsized":"2024-02-26T10:38:36.182530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182446+0000","last_change":"2024-02-26T10:38:35.209104+0000","last_active":"2024-02-26T10:38:36.182446+0000","last_peered":"2024-02-26T10:38:36.182446+0000","last_clean":"2024-02-26T10:38:36.182446+0000","last_became_active":"2024-02-26T10:38:35.208841+0000","last_became_peered":"2024-02-26T10:38:35.208841+0000","last_unstale":"2024-02-26T10:38:36.182446+0000","last_undegraded":"2024-02-26T10:38:36.182446+0000","last_fullsized":"2024-02-26T10:38:36.182446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175483+0000","last_change":"2024-02-26T10:38:35.198390+0000","last_active":"2024-02-26T10:38:36.175483+0000","last_peered":"2024-02-26T10:38:36.175483+0000","last_clean":"2024-02-26T10:38:36.175483+0000","last_became_active":"2024-02-26T10:38:35.197895+0000","last_became_peered":"2024-02-26T10:38:35.197895+0000","last_unstale":"2024-02-26T10:38:36.175483+0000","last_undegraded":"2024-02-26T10:38:36.175483+0000","last_fullsized":"2024-02-26T10:38:36.175483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:01.842347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.192330+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.192046+0000","last_became_peered":"2024-02-26T10:38:35.192046+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177330+0000","last_change":"2024-02-26T10:38:35.181873+0000","last_active":"2024-02-26T10:38:36.177330+0000","last_peered":"2024-02-26T10:38:36.177330+0000","last_clean":"2024-02-26T10:38:36.177330+0000","last_became_active":"2024-02-26T10:38:35.180543+0000","last_became_peered":"2024-02-26T10:38:35.180543+0000","last_unstale":"2024-02-26T10:38:36.177330+0000","last_undegraded":"2024-02-26T10:38:36.177330+0000","last_fullsized":"2024-02-26T10:38:36.177330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:04:47.351870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173628+0000","last_change":"2024-02-26T10:38:35.186385+0000","last_active":"2024-02-26T10:38:36.173628+0000","last_peered":"2024-02-26T10:38:36.173628+0000","last_clean":"2024-02-26T10:38:36.173628+0000","last_became_active":"2024-02-26T10:38:35.186103+0000","last_became_peered":"2024-02-26T10:38:35.186103+0000","last_unstale":"2024-02-26T10:38:36.173628+0000","last_undegraded":"2024-02-26T10:38:36.173628+0000","last_fullsized":"2024-02-26T10:38:36.173628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234734+0000","last_change":"2024-02-26T10:38:35.203847+0000","last_active":"2024-02-26T10:38:37.234734+0000","last_peered":"2024-02-26T10:38:37.234734+0000","last_clean":"2024-02-26T10:38:37.234734+0000","last_became_active":"2024-02-26T10:38:35.201819+0000","last_became_peered":"2024-02-26T10:38:35.201819+0000","last_unstale":"2024-02-26T10:38:37.234734+0000","last_undegraded":"2024-02-26T10:38:37.234734+0000","last_fullsized":"2024-02-26T10:38:37.234734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237593+0000","last_change":"2024-02-26T10:38:35.205600+0000","last_active":"2024-02-26T10:38:37.237593+0000","last_peered":"2024-02-26T10:38:37.237593+0000","last_clean":"2024-02-26T10:38:37.237593+0000","last_became_active":"2024-02-26T10:38:35.205272+0000","last_became_peered":"2024-02-26T10:38:35.205272+0000","last_unstale":"2024-02-26T10:38:37.237593+0000","last_undegraded":"2024-02-26T10:38:37.237593+0000","last_fullsized":"2024-02-26T10:38:37.237593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237554+0000","last_change":"2024-02-26T10:38:35.191539+0000","last_active":"2024-02-26T10:38:37.237554+0000","last_peered":"2024-02-26T10:38:37.237554+0000","last_clean":"2024-02-26T10:38:37.237554+0000","last_became_active":"2024-02-26T10:38:35.191258+0000","last_became_peered":"2024-02-26T10:38:35.191258+0000","last_unstale":"2024-02-26T10:38:37.237554+0000","last_undegraded":"2024-02-26T10:38:37.237554+0000","last_fullsized":"2024-02-26T10:38:37.237554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182184+0000","last_change":"2024-02-26T10:38:35.197155+0000","last_active":"2024-02-26T10:38:36.182184+0000","last_peered":"2024-02-26T10:38:36.182184+0000","last_clean":"2024-02-26T10:38:36.182184+0000","last_became_active":"2024-02-26T10:38:35.192752+0000","last_became_peered":"2024-02-26T10:38:35.192752+0000","last_unstale":"2024-02-26T10:38:36.182184+0000","last_undegraded":"2024-02-26T10:38:36.182184+0000","last_fullsized":"2024-02-26T10:38:36.182184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178148+0000","last_change":"2024-02-26T10:38:35.192588+0000","last_active":"2024-02-26T10:38:36.178148+0000","last_peered":"2024-02-26T10:38:36.178148+0000","last_clean":"2024-02-26T10:38:36.178148+0000","last_became_active":"2024-02-26T10:38:35.192244+0000","last_became_peered":"2024-02-26T10:38:35.192244+0000","last_unstale":"2024-02-26T10:38:36.178148+0000","last_undegraded":"2024-02-26T10:38:36.178148+0000","last_fullsized":"2024-02-26T10:38:36.178148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235257+0000","last_change":"2024-02-26T10:38:35.193557+0000","last_active":"2024-02-26T10:38:37.235257+0000","last_peered":"2024-02-26T10:38:37.235257+0000","last_clean":"2024-02-26T10:38:37.235257+0000","last_became_active":"2024-02-26T10:38:35.192814+0000","last_became_peered":"2024-02-26T10:38:35.192814+0000","last_unstale":"2024-02-26T10:38:37.235257+0000","last_undegraded":"2024-02-26T10:38:37.235257+0000","last_fullsized":"2024-02-26T10:38:37.235257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235534+0000","last_change":"2024-02-26T10:38:35.188741+0000","last_active":"2024-02-26T10:38:37.235534+0000","last_peered":"2024-02-26T10:38:37.235534+0000","last_clean":"2024-02-26T10:38:37.235534+0000","last_became_active":"2024-02-26T10:38:35.188296+0000","last_became_peered":"2024-02-26T10:38:35.188296+0000","last_unstale":"2024-02-26T10:38:37.235534+0000","last_undegraded":"2024-02-26T10:38:37.235534+0000","last_fullsized":"2024-02-26T10:38:37.235534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235999+0000","last_change":"2024-02-26T10:38:35.193665+0000","last_active":"2024-02-26T10:38:37.235999+0000","last_peered":"2024-02-26T10:38:37.235999+0000","last_clean":"2024-02-26T10:38:37.235999+0000","last_became_active":"2024-02-26T10:38:35.193314+0000","last_became_peered":"2024-02-26T10:38:35.193314+0000","last_unstale":"2024-02-26T10:38:37.235999+0000","last_undegraded":"2024-02-26T10:38:37.235999+0000","last_fullsized":"2024-02-26T10:38:37.235999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236084+0000","last_change":"2024-02-26T10:38:35.186570+0000","last_active":"2024-02-26T10:38:37.236084+0000","last_peered":"2024-02-26T10:38:37.236084+0000","last_clean":"2024-02-26T10:38:37.236084+0000","last_became_active":"2024-02-26T10:38:35.186248+0000","last_became_peered":"2024-02-26T10:38:35.186248+0000","last_unstale":"2024-02-26T10:38:37.236084+0000","last_undegraded":"2024-02-26T10:38:37.236084+0000","last_fullsized":"2024-02-26T10:38:37.236084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182707+0000","last_change":"2024-02-26T10:38:35.197545+0000","last_active":"2024-02-26T10:38:36.182707+0000","last_peered":"2024-02-26T10:38:36.182707+0000","last_clean":"2024-02-26T10:38:36.182707+0000","last_became_active":"2024-02-26T10:38:35.197220+0000","last_became_peered":"2024-02-26T10:38:35.197220+0000","last_unstale":"2024-02-26T10:38:36.182707+0000","last_undegraded":"2024-02-26T10:38:36.182707+0000","last_fullsized":"2024-02-26T10:38:36.182707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173665+0000","last_change":"2024-02-26T10:38:35.174943+0000","last_active":"2024-02-26T10:38:36.173665+0000","last_peered":"2024-02-26T10:38:36.173665+0000","last_clean":"2024-02-26T10:38:36.173665+0000","last_became_active":"2024-02-26T10:38:35.174618+0000","last_became_peered":"2024-02-26T10:38:35.174618+0000","last_unstale":"2024-02-26T10:38:36.173665+0000","last_undegraded":"2024-02-26T10:38:36.173665+0000","last_fullsized":"2024-02-26T10:38:36.173665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235860+0000","last_change":"2024-02-26T10:38:35.188806+0000","last_active":"2024-02-26T10:38:37.235860+0000","last_peered":"2024-02-26T10:38:37.235860+0000","last_clean":"2024-02-26T10:38:37.235860+0000","last_became_active":"2024-02-26T10:38:35.188539+0000","last_became_peered":"2024-02-26T10:38:35.188539+0000","last_unstale":"2024-02-26T10:38:37.235860+0000","last_undegraded":"2024-02-26T10:38:37.235860+0000","last_fullsized":"2024-02-26T10:38:37.235860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178458+0000","last_change":"2024-02-26T10:38:35.183491+0000","last_active":"2024-02-26T10:38:36.178458+0000","last_peered":"2024-02-26T10:38:36.178458+0000","last_clean":"2024-02-26T10:38:36.178458+0000","last_became_active":"2024-02-26T10:38:35.183153+0000","last_became_peered":"2024-02-26T10:38:35.183153+0000","last_unstale":"2024-02-26T10:38:36.178458+0000","last_undegraded":"2024-02-26T10:38:36.178458+0000","last_fullsized":"2024-02-26T10:38:36.178458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235539+0000","last_change":"2024-02-26T10:38:35.185342+0000","last_active":"2024-02-26T10:38:37.235539+0000","last_peered":"2024-02-26T10:38:37.235539+0000","last_clean":"2024-02-26T10:38:37.235539+0000","last_became_active":"2024-02-26T10:38:35.185067+0000","last_became_peered":"2024-02-26T10:38:35.185067+0000","last_unstale":"2024-02-26T10:38:37.235539+0000","last_undegraded":"2024-02-26T10:38:37.235539+0000","last_fullsized":"2024-02-26T10:38:37.235539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181690+0000","last_change":"2024-02-26T10:38:35.191702+0000","last_active":"2024-02-26T10:38:36.181690+0000","last_peered":"2024-02-26T10:38:36.181690+0000","last_clean":"2024-02-26T10:38:36.181690+0000","last_became_active":"2024-02-26T10:38:35.191416+0000","last_became_peered":"2024-02-26T10:38:35.191416+0000","last_unstale":"2024-02-26T10:38:36.181690+0000","last_undegraded":"2024-02-26T10:38:36.181690+0000","last_fullsized":"2024-02-26T10:38:36.181690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175512+0000","last_change":"2024-02-26T10:38:35.169056+0000","last_active":"2024-02-26T10:38:36.175512+0000","last_peered":"2024-02-26T10:38:36.175512+0000","last_clean":"2024-02-26T10:38:36.175512+0000","last_became_active":"2024-02-26T10:38:35.168640+0000","last_became_peered":"2024-02-26T10:38:35.168640+0000","last_unstale":"2024-02-26T10:38:36.175512+0000","last_undegraded":"2024-02-26T10:38:36.175512+0000","last_fullsized":"2024-02-26T10:38:36.175512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:44.237255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235514+0000","last_change":"2024-02-26T10:38:34.169112+0000","last_active":"2024-02-26T10:38:37.235514+0000","last_peered":"2024-02-26T10:38:37.235514+0000","last_clean":"2024-02-26T10:38:37.235514+0000","last_became_active":"2024-02-26T10:38:34.168796+0000","last_became_peered":"2024-02-26T10:38:34.168796+0000","last_unstale":"2024-02-26T10:38:37.235514+0000","last_undegraded":"2024-02-26T10:38:37.235514+0000","last_fullsized":"2024-02-26T10:38:37.235514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:29:24.719556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178388+0000","last_change":"2024-02-26T10:38:35.187182+0000","last_active":"2024-02-26T10:38:36.178388+0000","last_peered":"2024-02-26T10:38:36.178388+0000","last_clean":"2024-02-26T10:38:36.178388+0000","last_became_active":"2024-02-26T10:38:35.185661+0000","last_became_peered":"2024-02-26T10:38:35.185661+0000","last_unstale":"2024-02-26T10:38:36.178388+0000","last_undegraded":"2024-02-26T10:38:36.178388+0000","last_fullsized":"2024-02-26T10:38:36.178388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181759+0000","last_change":"2024-02-26T10:38:34.179842+0000","last_active":"2024-02-26T10:38:36.181759+0000","last_peered":"2024-02-26T10:38:36.181759+0000","last_clean":"2024-02-26T10:38:36.181759+0000","last_became_active":"2024-02-26T10:38:34.179535+0000","last_became_peered":"2024-02-26T10:38:34.179535+0000","last_unstale":"2024-02-26T10:38:36.181759+0000","last_undegraded":"2024-02-26T10:38:36.181759+0000","last_fullsized":"2024-02-26T10:38:36.181759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235171+0000","last_change":"2024-02-26T10:38:34.167613+0000","last_active":"2024-02-26T10:38:37.235171+0000","last_peered":"2024-02-26T10:38:37.235171+0000","last_clean":"2024-02-26T10:38:37.235171+0000","last_became_active":"2024-02-26T10:38:34.167252+0000","last_became_peered":"2024-02-26T10:38:34.167252+0000","last_unstale":"2024-02-26T10:38:37.235171+0000","last_undegraded":"2024-02-26T10:38:37.235171+0000","last_fullsized":"2024-02-26T10:38:37.235171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235040+0000","last_change":"2024-02-26T10:38:35.185939+0000","last_active":"2024-02-26T10:38:37.235040+0000","last_peered":"2024-02-26T10:38:37.235040+0000","last_clean":"2024-02-26T10:38:37.235040+0000","last_became_active":"2024-02-26T10:38:35.183226+0000","last_became_peered":"2024-02-26T10:38:35.183226+0000","last_unstale":"2024-02-26T10:38:37.235040+0000","last_undegraded":"2024-02-26T10:38:37.235040+0000","last_fullsized":"2024-02-26T10:38:37.235040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236134+0000","last_change":"2024-02-26T10:38:34.186454+0000","last_active":"2024-02-26T10:38:37.236134+0000","last_peered":"2024-02-26T10:38:37.236134+0000","last_clean":"2024-02-26T10:38:37.236134+0000","last_became_active":"2024-02-26T10:38:34.185676+0000","last_became_peered":"2024-02-26T10:38:34.185676+0000","last_unstale":"2024-02-26T10:38:37.236134+0000","last_undegraded":"2024-02-26T10:38:37.236134+0000","last_fullsized":"2024-02-26T10:38:37.236134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173408+0000","last_change":"2024-02-26T10:38:35.176977+0000","last_active":"2024-02-26T10:38:36.173408+0000","last_peered":"2024-02-26T10:38:36.173408+0000","last_clean":"2024-02-26T10:38:36.173408+0000","last_became_active":"2024-02-26T10:38:35.175624+0000","last_became_peered":"2024-02-26T10:38:35.175624+0000","last_unstale":"2024-02-26T10:38:36.173408+0000","last_undegraded":"2024-02-26T10:38:36.173408+0000","last_fullsized":"2024-02-26T10:38:36.173408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182463+0000","last_change":"2024-02-26T10:38:34.188391+0000","last_active":"2024-02-26T10:38:36.182463+0000","last_peered":"2024-02-26T10:38:36.182463+0000","last_clean":"2024-02-26T10:38:36.182463+0000","last_became_active":"2024-02-26T10:38:34.188077+0000","last_became_peered":"2024-02-26T10:38:34.188077+0000","last_unstale":"2024-02-26T10:38:36.182463+0000","last_undegraded":"2024-02-26T10:38:36.182463+0000","last_fullsized":"2024-02-26T10:38:36.182463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236526+0000","last_change":"2024-02-26T10:38:35.199765+0000","last_active":"2024-02-26T10:38:37.236526+0000","last_peered":"2024-02-26T10:38:37.236526+0000","last_clean":"2024-02-26T10:38:37.236526+0000","last_became_active":"2024-02-26T10:38:35.199478+0000","last_became_peered":"2024-02-26T10:38:35.199478+0000","last_unstale":"2024-02-26T10:38:37.236526+0000","last_undegraded":"2024-02-26T10:38:37.236526+0000","last_fullsized":"2024-02-26T10:38:37.236526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236673+0000","last_change":"2024-02-26T10:38:34.189159+0000","last_active":"2024-02-26T10:38:37.236673+0000","last_peered":"2024-02-26T10:38:37.236673+0000","last_clean":"2024-02-26T10:38:37.236673+0000","last_became_active":"2024-02-26T10:38:34.187774+0000","last_became_peered":"2024-02-26T10:38:34.187774+0000","last_unstale":"2024-02-26T10:38:37.236673+0000","last_undegraded":"2024-02-26T10:38:37.236673+0000","last_fullsized":"2024-02-26T10:38:37.236673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178929+0000","last_change":"2024-02-26T10:38:35.198495+0000","last_active":"2024-02-26T10:38:36.178929+0000","last_peered":"2024-02-26T10:38:36.178929+0000","last_clean":"2024-02-26T10:38:36.178929+0000","last_became_active":"2024-02-26T10:38:35.198049+0000","last_became_peered":"2024-02-26T10:38:35.198049+0000","last_unstale":"2024-02-26T10:38:36.178929+0000","last_undegraded":"2024-02-26T10:38:36.178929+0000","last_fullsized":"2024-02-26T10:38:36.178929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173743+0000","last_change":"2024-02-26T10:38:34.182673+0000","last_active":"2024-02-26T10:38:36.173743+0000","last_peered":"2024-02-26T10:38:36.173743+0000","last_clean":"2024-02-26T10:38:36.173743+0000","last_became_active":"2024-02-26T10:38:34.182369+0000","last_became_peered":"2024-02-26T10:38:34.182369+0000","last_unstale":"2024-02-26T10:38:36.173743+0000","last_undegraded":"2024-02-26T10:38:36.173743+0000","last_fullsized":"2024-02-26T10:38:36.173743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175617+0000","last_change":"2024-02-26T10:38:35.193526+0000","last_active":"2024-02-26T10:38:36.175617+0000","last_peered":"2024-02-26T10:38:36.175617+0000","last_clean":"2024-02-26T10:38:36.175617+0000","last_became_active":"2024-02-26T10:38:35.193192+0000","last_became_peered":"2024-02-26T10:38:35.193192+0000","last_unstale":"2024-02-26T10:38:36.175617+0000","last_undegraded":"2024-02-26T10:38:36.175617+0000","last_fullsized":"2024-02-26T10:38:36.175617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:08.969223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182341+0000","last_change":"2024-02-26T10:38:34.178005+0000","last_active":"2024-02-26T10:38:36.182341+0000","last_peered":"2024-02-26T10:38:36.182341+0000","last_clean":"2024-02-26T10:38:36.182341+0000","last_became_active":"2024-02-26T10:38:34.177684+0000","last_became_peered":"2024-02-26T10:38:34.177684+0000","last_unstale":"2024-02-26T10:38:36.182341+0000","last_undegraded":"2024-02-26T10:38:36.182341+0000","last_fullsized":"2024-02-26T10:38:36.182341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.196333+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.191241+0000","last_became_peered":"2024-02-26T10:38:35.191241+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178541+0000","last_change":"2024-02-26T10:38:34.178253+0000","last_active":"2024-02-26T10:38:36.178541+0000","last_peered":"2024-02-26T10:38:36.178541+0000","last_clean":"2024-02-26T10:38:36.178541+0000","last_became_active":"2024-02-26T10:38:34.177880+0000","last_became_peered":"2024-02-26T10:38:34.177880+0000","last_unstale":"2024-02-26T10:38:36.178541+0000","last_undegraded":"2024-02-26T10:38:36.178541+0000","last_fullsized":"2024-02-26T10:38:36.178541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177760+0000","last_change":"2024-02-26T10:38:35.196164+0000","last_active":"2024-02-26T10:38:36.177760+0000","last_peered":"2024-02-26T10:38:36.177760+0000","last_clean":"2024-02-26T10:38:36.177760+0000","last_became_active":"2024-02-26T10:38:35.195846+0000","last_became_peered":"2024-02-26T10:38:35.195846+0000","last_unstale":"2024-02-26T10:38:36.177760+0000","last_undegraded":"2024-02-26T10:38:36.177760+0000","last_fullsized":"2024-02-26T10:38:36.177760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:36.847842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235392+0000","last_change":"2024-02-26T10:38:34.180900+0000","last_active":"2024-02-26T10:38:37.235392+0000","last_peered":"2024-02-26T10:38:37.235392+0000","last_clean":"2024-02-26T10:38:37.235392+0000","last_became_active":"2024-02-26T10:38:34.180316+0000","last_became_peered":"2024-02-26T10:38:34.180316+0000","last_unstale":"2024-02-26T10:38:37.235392+0000","last_undegraded":"2024-02-26T10:38:37.235392+0000","last_fullsized":"2024-02-26T10:38:37.235392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235901+0000","last_change":"2024-02-26T10:38:35.188962+0000","last_active":"2024-02-26T10:38:37.235901+0000","last_peered":"2024-02-26T10:38:37.235901+0000","last_clean":"2024-02-26T10:38:37.235901+0000","last_became_active":"2024-02-26T10:38:35.188506+0000","last_became_peered":"2024-02-26T10:38:35.188506+0000","last_unstale":"2024-02-26T10:38:37.235901+0000","last_undegraded":"2024-02-26T10:38:37.235901+0000","last_fullsized":"2024-02-26T10:38:37.235901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181772+0000","last_change":"2024-02-26T10:38:34.180239+0000","last_active":"2024-02-26T10:38:36.181772+0000","last_peered":"2024-02-26T10:38:36.181772+0000","last_clean":"2024-02-26T10:38:36.181772+0000","last_became_active":"2024-02-26T10:38:34.179109+0000","last_became_peered":"2024-02-26T10:38:34.179109+0000","last_unstale":"2024-02-26T10:38:36.181772+0000","last_undegraded":"2024-02-26T10:38:36.181772+0000","last_fullsized":"2024-02-26T10:38:36.181772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181887+0000","last_change":"2024-02-26T10:38:35.196600+0000","last_active":"2024-02-26T10:38:36.181887+0000","last_peered":"2024-02-26T10:38:36.181887+0000","last_clean":"2024-02-26T10:38:36.181887+0000","last_became_active":"2024-02-26T10:38:35.192063+0000","last_became_peered":"2024-02-26T10:38:35.192063+0000","last_unstale":"2024-02-26T10:38:36.181887+0000","last_undegraded":"2024-02-26T10:38:36.181887+0000","last_fullsized":"2024-02-26T10:38:36.181887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237658+0000","last_change":"2024-02-26T10:38:35.190544+0000","last_active":"2024-02-26T10:38:37.237658+0000","last_peered":"2024-02-26T10:38:37.237658+0000","last_clean":"2024-02-26T10:38:37.237658+0000","last_became_active":"2024-02-26T10:38:35.190066+0000","last_became_peered":"2024-02-26T10:38:35.190066+0000","last_unstale":"2024-02-26T10:38:37.237658+0000","last_undegraded":"2024-02-26T10:38:37.237658+0000","last_fullsized":"2024-02-26T10:38:37.237658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175919+0000","last_change":"2024-02-26T10:38:34.165622+0000","last_active":"2024-02-26T10:38:36.175919+0000","last_peered":"2024-02-26T10:38:36.175919+0000","last_clean":"2024-02-26T10:38:36.175919+0000","last_became_active":"2024-02-26T10:38:34.165165+0000","last_became_peered":"2024-02-26T10:38:34.165165+0000","last_unstale":"2024-02-26T10:38:36.175919+0000","last_undegraded":"2024-02-26T10:38:36.175919+0000","last_fullsized":"2024-02-26T10:38:36.175919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:02.556621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236130+0000","last_change":"2024-02-26T10:38:35.191334+0000","last_active":"2024-02-26T10:38:37.236130+0000","last_peered":"2024-02-26T10:38:37.236130+0000","last_clean":"2024-02-26T10:38:37.236130+0000","last_became_active":"2024-02-26T10:38:35.187187+0000","last_became_peered":"2024-02-26T10:38:35.187187+0000","last_unstale":"2024-02-26T10:38:37.236130+0000","last_undegraded":"2024-02-26T10:38:37.236130+0000","last_fullsized":"2024-02-26T10:38:37.236130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.238041+0000","last_change":"2024-02-26T10:38:35.189775+0000","last_active":"2024-02-26T10:38:37.238041+0000","last_peered":"2024-02-26T10:38:37.238041+0000","last_clean":"2024-02-26T10:38:37.238041+0000","last_became_active":"2024-02-26T10:38:35.189470+0000","last_became_peered":"2024-02-26T10:38:35.189470+0000","last_unstale":"2024-02-26T10:38:37.238041+0000","last_undegraded":"2024-02-26T10:38:37.238041+0000","last_fullsized":"2024-02-26T10:38:37.238041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:56:01.655512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174043+0000","last_change":"2024-02-26T10:38:34.180005+0000","last_active":"2024-02-26T10:38:36.174043+0000","last_peered":"2024-02-26T10:38:36.174043+0000","last_clean":"2024-02-26T10:38:36.174043+0000","last_became_active":"2024-02-26T10:38:34.179694+0000","last_became_peered":"2024-02-26T10:38:34.179694+0000","last_unstale":"2024-02-26T10:38:36.174043+0000","last_undegraded":"2024-02-26T10:38:36.174043+0000","last_fullsized":"2024-02-26T10:38:36.174043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236746+0000","last_change":"2024-02-26T10:38:35.190450+0000","last_active":"2024-02-26T10:38:37.236746+0000","last_peered":"2024-02-26T10:38:37.236746+0000","last_clean":"2024-02-26T10:38:37.236746+0000","last_became_active":"2024-02-26T10:38:35.187303+0000","last_became_peered":"2024-02-26T10:38:35.187303+0000","last_unstale":"2024-02-26T10:38:37.236746+0000","last_undegraded":"2024-02-26T10:38:37.236746+0000","last_fullsized":"2024-02-26T10:38:37.236746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235843+0000","last_change":"2024-02-26T10:38:35.196564+0000","last_active":"2024-02-26T10:38:37.235843+0000","last_peered":"2024-02-26T10:38:37.235843+0000","last_clean":"2024-02-26T10:38:37.235843+0000","last_became_active":"2024-02-26T10:38:35.196286+0000","last_became_peered":"2024-02-26T10:38:35.196286+0000","last_unstale":"2024-02-26T10:38:37.235843+0000","last_undegraded":"2024-02-26T10:38:37.235843+0000","last_fullsized":"2024-02-26T10:38:37.235843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235369+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:37.235369+0000","last_peered":"2024-02-26T10:38:37.235369+0000","last_clean":"2024-02-26T10:38:37.235369+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:37.235369+0000","last_undegraded":"2024-02-26T10:38:37.235369+0000","last_fullsized":"2024-02-26T10:38:37.235369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234740+0000","last_change":"2024-02-26T10:38:34.181165+0000","last_active":"2024-02-26T10:38:37.234740+0000","last_peered":"2024-02-26T10:38:37.234740+0000","last_clean":"2024-02-26T10:38:37.234740+0000","last_became_active":"2024-02-26T10:38:34.180374+0000","last_became_peered":"2024-02-26T10:38:34.180374+0000","last_unstale":"2024-02-26T10:38:37.234740+0000","last_undegraded":"2024-02-26T10:38:37.234740+0000","last_fullsized":"2024-02-26T10:38:37.234740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235370+0000","last_change":"2024-02-26T10:38:35.205573+0000","last_active":"2024-02-26T10:38:37.235370+0000","last_peered":"2024-02-26T10:38:37.235370+0000","last_clean":"2024-02-26T10:38:37.235370+0000","last_became_active":"2024-02-26T10:38:35.205295+0000","last_became_peered":"2024-02-26T10:38:35.205295+0000","last_unstale":"2024-02-26T10:38:37.235370+0000","last_undegraded":"2024-02-26T10:38:37.235370+0000","last_fullsized":"2024-02-26T10:38:37.235370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234753+0000","last_change":"2024-02-26T10:38:34.177423+0000","last_active":"2024-02-26T10:38:37.234753+0000","last_peered":"2024-02-26T10:38:37.234753+0000","last_clean":"2024-02-26T10:38:37.234753+0000","last_became_active":"2024-02-26T10:38:34.175531+0000","last_became_peered":"2024-02-26T10:38:34.175531+0000","last_unstale":"2024-02-26T10:38:37.234753+0000","last_undegraded":"2024-02-26T10:38:37.234753+0000","last_fullsized":"2024-02-26T10:38:37.234753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.176766+0000","last_change":"2024-02-26T10:38:35.182828+0000","last_active":"2024-02-26T10:38:36.176766+0000","last_peered":"2024-02-26T10:38:36.176766+0000","last_clean":"2024-02-26T10:38:36.176766+0000","last_became_active":"2024-02-26T10:38:35.182453+0000","last_became_peered":"2024-02-26T10:38:35.182453+0000","last_unstale":"2024-02-26T10:38:36.176766+0000","last_undegraded":"2024-02-26T10:38:36.176766+0000","last_fullsized":"2024-02-26T10:38:36.176766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:15.325827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173393+0000","last_change":"2024-02-26T10:38:35.180693+0000","last_active":"2024-02-26T10:38:36.173393+0000","last_peered":"2024-02-26T10:38:36.173393+0000","last_clean":"2024-02-26T10:38:36.173393+0000","last_became_active":"2024-02-26T10:38:35.178787+0000","last_became_peered":"2024-02-26T10:38:35.178787+0000","last_unstale":"2024-02-26T10:38:36.173393+0000","last_undegraded":"2024-02-26T10:38:36.173393+0000","last_fullsized":"2024-02-26T10:38:36.173393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235683+0000","last_change":"2024-02-26T10:38:34.184572+0000","last_active":"2024-02-26T10:38:37.235683+0000","last_peered":"2024-02-26T10:38:37.235683+0000","last_clean":"2024-02-26T10:38:37.235683+0000","last_became_active":"2024-02-26T10:38:34.183650+0000","last_became_peered":"2024-02-26T10:38:34.183650+0000","last_unstale":"2024-02-26T10:38:37.235683+0000","last_undegraded":"2024-02-26T10:38:37.235683+0000","last_fullsized":"2024-02-26T10:38:37.235683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175481+0000","last_change":"2024-02-26T10:38:35.186511+0000","last_active":"2024-02-26T10:38:36.175481+0000","last_peered":"2024-02-26T10:38:36.175481+0000","last_clean":"2024-02-26T10:38:36.175481+0000","last_became_active":"2024-02-26T10:38:35.185321+0000","last_became_peered":"2024-02-26T10:38:35.185321+0000","last_unstale":"2024-02-26T10:38:36.175481+0000","last_undegraded":"2024-02-26T10:38:36.175481+0000","last_fullsized":"2024-02-26T10:38:36.175481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:18.018830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174062+0000","last_change":"2024-02-26T10:38:34.165105+0000","last_active":"2024-02-26T10:38:36.174062+0000","last_peered":"2024-02-26T10:38:36.174062+0000","last_clean":"2024-02-26T10:38:36.174062+0000","last_became_active":"2024-02-26T10:38:34.164756+0000","last_became_peered":"2024-02-26T10:38:34.164756+0000","last_unstale":"2024-02-26T10:38:36.174062+0000","last_undegraded":"2024-02-26T10:38:36.174062+0000","last_fullsized":"2024-02-26T10:38:36.174062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237188+0000","last_change":"2024-02-26T10:38:35.191275+0000","last_active":"2024-02-26T10:38:37.237188+0000","last_peered":"2024-02-26T10:38:37.237188+0000","last_clean":"2024-02-26T10:38:37.237188+0000","last_became_active":"2024-02-26T10:38:35.190875+0000","last_became_peered":"2024-02-26T10:38:35.190875+0000","last_unstale":"2024-02-26T10:38:37.237188+0000","last_undegraded":"2024-02-26T10:38:37.237188+0000","last_fullsized":"2024-02-26T10:38:37.237188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237699+0000","last_change":"2024-02-26T10:38:34.176727+0000","last_active":"2024-02-26T10:38:37.237699+0000","last_peered":"2024-02-26T10:38:37.237699+0000","last_clean":"2024-02-26T10:38:37.237699+0000","last_became_active":"2024-02-26T10:38:34.176422+0000","last_became_peered":"2024-02-26T10:38:34.176422+0000","last_unstale":"2024-02-26T10:38:37.237699+0000","last_undegraded":"2024-02-26T10:38:37.237699+0000","last_fullsized":"2024-02-26T10:38:37.237699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174128+0000","last_change":"2024-02-26T10:38:35.174940+0000","last_active":"2024-02-26T10:38:36.174128+0000","last_peered":"2024-02-26T10:38:36.174128+0000","last_clean":"2024-02-26T10:38:36.174128+0000","last_became_active":"2024-02-26T10:38:35.174609+0000","last_became_peered":"2024-02-26T10:38:35.174609+0000","last_unstale":"2024-02-26T10:38:36.174128+0000","last_undegraded":"2024-02-26T10:38:36.174128+0000","last_fullsized":"2024-02-26T10:38:36.174128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174557+0000","last_change":"2024-02-26T10:38:34.176149+0000","last_active":"2024-02-26T10:38:36.174557+0000","last_peered":"2024-02-26T10:38:36.174557+0000","last_clean":"2024-02-26T10:38:36.174557+0000","last_became_active":"2024-02-26T10:38:34.175800+0000","last_became_peered":"2024-02-26T10:38:34.175800+0000","last_unstale":"2024-02-26T10:38:36.174557+0000","last_undegraded":"2024-02-26T10:38:36.174557+0000","last_fullsized":"2024-02-26T10:38:36.174557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236113+0000","last_change":"2024-02-26T10:38:35.191410+0000","last_active":"2024-02-26T10:38:37.236113+0000","last_peered":"2024-02-26T10:38:37.236113+0000","last_clean":"2024-02-26T10:38:37.236113+0000","last_became_active":"2024-02-26T10:38:35.190954+0000","last_became_peered":"2024-02-26T10:38:35.190954+0000","last_unstale":"2024-02-26T10:38:37.236113+0000","last_undegraded":"2024-02-26T10:38:37.236113+0000","last_fullsized":"2024-02-26T10:38:37.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182629+0000","last_change":"2024-02-26T10:38:34.185447+0000","last_active":"2024-02-26T10:38:36.182629+0000","last_peered":"2024-02-26T10:38:36.182629+0000","last_clean":"2024-02-26T10:38:36.182629+0000","last_became_active":"2024-02-26T10:38:34.185054+0000","last_became_peered":"2024-02-26T10:38:34.185054+0000","last_unstale":"2024-02-26T10:38:36.182629+0000","last_undegraded":"2024-02-26T10:38:36.182629+0000","last_fullsized":"2024-02-26T10:38:36.182629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179198+0000","last_change":"2024-02-26T10:38:35.181510+0000","last_active":"2024-02-26T10:38:36.179198+0000","last_peered":"2024-02-26T10:38:36.179198+0000","last_clean":"2024-02-26T10:38:36.179198+0000","last_became_active":"2024-02-26T10:38:35.180830+0000","last_became_peered":"2024-02-26T10:38:35.180830+0000","last_unstale":"2024-02-26T10:38:36.179198+0000","last_undegraded":"2024-02-26T10:38:36.179198+0000","last_fullsized":"2024-02-26T10:38:36.179198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234630+0000","last_change":"2024-02-26T10:38:34.174303+0000","last_active":"2024-02-26T10:38:37.234630+0000","last_peered":"2024-02-26T10:38:37.234630+0000","last_clean":"2024-02-26T10:38:37.234630+0000","last_became_active":"2024-02-26T10:38:34.174006+0000","last_became_peered":"2024-02-26T10:38:34.174006+0000","last_unstale":"2024-02-26T10:38:37.234630+0000","last_undegraded":"2024-02-26T10:38:37.234630+0000","last_fullsized":"2024-02-26T10:38:37.234630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175416+0000","last_change":"2024-02-26T10:38:35.176832+0000","last_active":"2024-02-26T10:38:36.175416+0000","last_peered":"2024-02-26T10:38:36.175416+0000","last_clean":"2024-02-26T10:38:36.175416+0000","last_became_active":"2024-02-26T10:38:35.176305+0000","last_became_peered":"2024-02-26T10:38:35.176305+0000","last_unstale":"2024-02-26T10:38:36.175416+0000","last_undegraded":"2024-02-26T10:38:36.175416+0000","last_fullsized":"2024-02-26T10:38:36.175416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:06.286416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173704+0000","last_change":"2024-02-26T10:38:34.169425+0000","last_active":"2024-02-26T10:38:36.173704+0000","last_peered":"2024-02-26T10:38:36.173704+0000","last_clean":"2024-02-26T10:38:36.173704+0000","last_became_active":"2024-02-26T10:38:34.168217+0000","last_became_peered":"2024-02-26T10:38:34.168217+0000","last_unstale":"2024-02-26T10:38:36.173704+0000","last_undegraded":"2024-02-26T10:38:36.173704+0000","last_fullsized":"2024-02-26T10:38:36.173704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236029+0000","last_change":"2024-02-26T10:38:35.172836+0000","last_active":"2024-02-26T10:38:37.236029+0000","last_peered":"2024-02-26T10:38:37.236029+0000","last_clean":"2024-02-26T10:38:37.236029+0000","last_became_active":"2024-02-26T10:38:35.172554+0000","last_became_peered":"2024-02-26T10:38:35.172554+0000","last_unstale":"2024-02-26T10:38:37.236029+0000","last_undegraded":"2024-02-26T10:38:37.236029+0000","last_fullsized":"2024-02-26T10:38:37.236029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173693+0000","last_change":"2024-02-26T10:38:34.166377+0000","last_active":"2024-02-26T10:38:36.173693+0000","last_peered":"2024-02-26T10:38:36.173693+0000","last_clean":"2024-02-26T10:38:36.173693+0000","last_became_active":"2024-02-26T10:38:34.165963+0000","last_became_peered":"2024-02-26T10:38:34.165963+0000","last_unstale":"2024-02-26T10:38:36.173693+0000","last_undegraded":"2024-02-26T10:38:36.173693+0000","last_fullsized":"2024-02-26T10:38:36.173693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235575+0000","last_change":"2024-02-26T10:38:35.185120+0000","last_active":"2024-02-26T10:38:37.235575+0000","last_peered":"2024-02-26T10:38:37.235575+0000","last_clean":"2024-02-26T10:38:37.235575+0000","last_became_active":"2024-02-26T10:38:35.184832+0000","last_became_peered":"2024-02-26T10:38:35.184832+0000","last_unstale":"2024-02-26T10:38:37.235575+0000","last_undegraded":"2024-02-26T10:38:37.235575+0000","last_fullsized":"2024-02-26T10:38:37.235575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236461+0000","last_change":"2024-02-26T10:38:34.180225+0000","last_active":"2024-02-26T10:38:37.236461+0000","last_peered":"2024-02-26T10:38:37.236461+0000","last_clean":"2024-02-26T10:38:37.236461+0000","last_became_active":"2024-02-26T10:38:34.179909+0000","last_became_peered":"2024-02-26T10:38:34.179909+0000","last_unstale":"2024-02-26T10:38:37.236461+0000","last_undegraded":"2024-02-26T10:38:37.236461+0000","last_fullsized":"2024-02-26T10:38:37.236461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:05:58.590708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182262+0000","last_change":"2024-02-26T10:38:35.188975+0000","last_active":"2024-02-26T10:38:36.182262+0000","last_peered":"2024-02-26T10:38:36.182262+0000","last_clean":"2024-02-26T10:38:36.182262+0000","last_became_active":"2024-02-26T10:38:35.188608+0000","last_became_peered":"2024-02-26T10:38:35.188608+0000","last_unstale":"2024-02-26T10:38:36.182262+0000","last_undegraded":"2024-02-26T10:38:36.182262+0000","last_fullsized":"2024-02-26T10:38:36.182262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236117+0000","last_change":"2024-02-26T10:38:35.196289+0000","last_active":"2024-02-26T10:38:37.236117+0000","last_peered":"2024-02-26T10:38:37.236117+0000","last_clean":"2024-02-26T10:38:37.236117+0000","last_became_active":"2024-02-26T10:38:35.194751+0000","last_became_peered":"2024-02-26T10:38:35.194751+0000","last_unstale":"2024-02-26T10:38:37.236117+0000","last_undegraded":"2024-02-26T10:38:37.236117+0000","last_fullsized":"2024-02-26T10:38:37.236117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236713+0000","last_change":"2024-02-26T10:38:34.186760+0000","last_active":"2024-02-26T10:38:37.236713+0000","last_peered":"2024-02-26T10:38:37.236713+0000","last_clean":"2024-02-26T10:38:37.236713+0000","last_became_active":"2024-02-26T10:38:34.186428+0000","last_became_peered":"2024-02-26T10:38:34.186428+0000","last_unstale":"2024-02-26T10:38:37.236713+0000","last_undegraded":"2024-02-26T10:38:37.236713+0000","last_fullsized":"2024-02-26T10:38:37.236713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236518+0000","last_change":"2024-02-26T10:38:35.202616+0000","last_active":"2024-02-26T10:38:37.236518+0000","last_peered":"2024-02-26T10:38:37.236518+0000","last_clean":"2024-02-26T10:38:37.236518+0000","last_became_active":"2024-02-26T10:38:35.201498+0000","last_became_peered":"2024-02-26T10:38:35.201498+0000","last_unstale":"2024-02-26T10:38:37.236518+0000","last_undegraded":"2024-02-26T10:38:37.236518+0000","last_fullsized":"2024-02-26T10:38:37.236518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235996+0000","last_change":"2024-02-26T10:38:34.186745+0000","last_active":"2024-02-26T10:38:37.235996+0000","last_peered":"2024-02-26T10:38:37.235996+0000","last_clean":"2024-02-26T10:38:37.235996+0000","last_became_active":"2024-02-26T10:38:34.186470+0000","last_became_peered":"2024-02-26T10:38:34.186470+0000","last_unstale":"2024-02-26T10:38:37.235996+0000","last_undegraded":"2024-02-26T10:38:37.235996+0000","last_fullsized":"2024-02-26T10:38:37.235996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178984+0000","last_change":"2024-02-26T10:38:35.206276+0000","last_active":"2024-02-26T10:38:36.178984+0000","last_peered":"2024-02-26T10:38:36.178984+0000","last_clean":"2024-02-26T10:38:36.178984+0000","last_became_active":"2024-02-26T10:38:35.205932+0000","last_became_peered":"2024-02-26T10:38:35.205932+0000","last_unstale":"2024-02-26T10:38:36.178984+0000","last_undegraded":"2024-02-26T10:38:36.178984+0000","last_fullsized":"2024-02-26T10:38:36.178984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235907+0000","last_change":"2024-02-26T10:38:34.165440+0000","last_active":"2024-02-26T10:38:37.235907+0000","last_peered":"2024-02-26T10:38:37.235907+0000","last_clean":"2024-02-26T10:38:37.235907+0000","last_became_active":"2024-02-26T10:38:34.165164+0000","last_became_peered":"2024-02-26T10:38:34.165164+0000","last_unstale":"2024-02-26T10:38:37.235907+0000","last_undegraded":"2024-02-26T10:38:37.235907+0000","last_fullsized":"2024-02-26T10:38:37.235907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235740+0000","last_change":"2024-02-26T10:38:35.199753+0000","last_active":"2024-02-26T10:38:37.235740+0000","last_peered":"2024-02-26T10:38:37.235740+0000","last_clean":"2024-02-26T10:38:37.235740+0000","last_became_active":"2024-02-26T10:38:35.199470+0000","last_became_peered":"2024-02-26T10:38:35.199470+0000","last_unstale":"2024-02-26T10:38:37.235740+0000","last_undegraded":"2024-02-26T10:38:37.235740+0000","last_fullsized":"2024-02-26T10:38:37.235740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179016+0000","last_change":"2024-02-26T10:38:34.184020+0000","last_active":"2024-02-26T10:38:36.179016+0000","last_peered":"2024-02-26T10:38:36.179016+0000","last_clean":"2024-02-26T10:38:36.179016+0000","last_became_active":"2024-02-26T10:38:34.183497+0000","last_became_peered":"2024-02-26T10:38:34.183497+0000","last_unstale":"2024-02-26T10:38:36.179016+0000","last_undegraded":"2024-02-26T10:38:36.179016+0000","last_fullsized":"2024-02-26T10:38:36.179016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:25:08.410476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":74327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":819200,"data_stored":665243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:44.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:44.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.939+0000 7fcaecc00640 1 Processor -- start 2024-02-26T10:38:44.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.940+0000 7fcaecc00640 1 -- start start 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcaecc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcaecc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcae8154d70 0x7fcae80c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcaecc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae81529f0 con 0x7fcae8152610 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcaecc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae81583d0 con 0x7fcae8154d70 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcaecc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8155310 con 0x7fcae80cb130 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcae7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcae8154d70 0x7fcae80c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcae7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcae8154d70 0x7fcae80c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33476/0 (socket says 172.21.15.67:33476) 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcae7e00640 1 -- 172.21.15.67:0/2013030698 learned_addr learned my addr 172.21.15.67:0/2013030698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:44.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.941+0000 7fcae7400640 1 --1- 172.21.15.67:0/2013030698 >> v1:172.21.15.67:6789/0 conn(0x7fcae8152610 0x7fcae8158860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54492/0 (socket says 172.21.15.67:54492) 2024-02-26T10:38:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.942+0000 7fcadea00640 1 --2- 172.21.15.67:0/2013030698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.942+0000 7fcae7e00640 1 -- 172.21.15.67:0/2013030698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 msgr2=0x7fcae80c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.942+0000 7fcae7e00640 1 --2- 172.21.15.67:0/2013030698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae80c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.942+0000 7fcae7e00640 1 -- 172.21.15.67:0/2013030698 >> v1:172.21.15.67:6789/0 conn(0x7fcae8152610 legacy=0x7fcae8158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.942+0000 7fcae6a00640 1 -- 172.21.15.67:0/2013030698 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247378115 0 0) 0x7fcae81529f0 con 0x7fcae8152610 2024-02-26T10:38:44.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.942+0000 7fcae7e00640 1 -- 172.21.15.67:0/2013030698 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae81555f0 con 0x7fcae8154d70 2024-02-26T10:38:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.943+0000 7fcae7e00640 1 --2- 172.21.15.67:0/2013030698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae80c8a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009e50 tx=0x7fcae002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd1003efd8aad65 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.943+0000 7fcae6a00640 1 -- 172.21.15.67:0/2013030698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae002ca70 con 0x7fcae8154d70 2024-02-26T10:38:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.943+0000 7fcae6a00640 1 -- 172.21.15.67:0/2013030698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae002cc10 con 0x7fcae8154d70 2024-02-26T10:38:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.943+0000 7fcae6a00640 1 -- 172.21.15.67:0/2013030698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae00343f0 con 0x7fcae8154d70 2024-02-26T10:38:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 -- 172.21.15.67:0/2013030698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 msgr2=0x7fcae80c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 --2- 172.21.15.67:0/2013030698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae80c8a50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009e50 tx=0x7fcae002bed0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 -- 172.21.15.67:0/2013030698 shutdown_connections 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 --2- 172.21.15.67:0/2013030698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae80c8a50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 --2- 172.21.15.67:0/2013030698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 -- 172.21.15.67:0/2013030698 >> 172.21.15.67:0/2013030698 conn(0x7fcae80ca610 msgr2=0x7fcae80caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 -- 172.21.15.67:0/2013030698 shutdown_connections 2024-02-26T10:38:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.944+0000 7fcaecc00640 1 -- 172.21.15.67:0/2013030698 wait complete. 2024-02-26T10:38:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.946+0000 7fcaecc00640 1 Processor -- start 2024-02-26T10:38:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.946+0000 7fcaecc00640 1 -- start start 2024-02-26T10:38:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcaecc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae8141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcaecc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcae8152610 0x7fcae8151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcaecc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae8157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcaecc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae806c1f0 con 0x7fcae8154d70 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcaecc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8155bb0 con 0x7fcae8152610 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcaecc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80c8f90 con 0x7fcae80cb130 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae8157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae8157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33490/0 (socket says 172.21.15.67:33490) 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 -- 172.21.15.67:0/1753855905 learned_addr learned my addr 172.21.15.67:0/1753855905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7400640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae8141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 -- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 msgr2=0x7fcae8141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcadea00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcae8152610 0x7fcae8151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae8141170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 -- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcae8152610 msgr2=0x7fcae8151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcae8152610 0x7fcae8151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcae7e00640 1 -- 172.21.15.67:0/1753855905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae81420c0 con 0x7fcae8154d70 2024-02-26T10:38:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.947+0000 7fcadea00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcae8152610 0x7fcae8151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.948+0000 7fcae7e00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae8157080 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcae0036040 tx=0x7fcae0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:44.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.948+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0044030 con 0x7fcae8154d70 2024-02-26T10:38:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.948+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae0040470 con 0x7fcae8154d70 2024-02-26T10:38:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.948+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0009510 con 0x7fcae8154d70 2024-02-26T10:38:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.948+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae8141930 con 0x7fcae8154d70 2024-02-26T10:38:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.948+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae80ab850 con 0x7fcae8154d70 2024-02-26T10:38:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.949+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae8074a30 con 0x7fcae8154d70 2024-02-26T10:38:44.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.950+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcae0047070 con 0x7fcae8154d70 2024-02-26T10:38:44.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.952+0000 7fcadfe00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcac406b500 0x7fcac406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:44.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.952+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcae00ac890 con 0x7fcae8154d70 2024-02-26T10:38:44.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.952+0000 7fcae7400640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcac406b500 0x7fcac406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:44.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.953+0000 7fcae7400640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcac406b500 0x7fcac406d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcad8000f60 tx=0x7fcad8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:44.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:44.956+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcae004e070 con 0x7fcae8154d70 2024-02-26T10:38:45.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.150+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcae81416b0 con 0x7fcac406b500 2024-02-26T10:38:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.169+0000 7fcadfe00640 1 -- 172.21.15.67:0/1753855905 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251838 (secure 0 0 0) 0x7fcad80070d0 con 0x7fcac406b500 2024-02-26T10:38:45.176 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:45.177 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.175+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcac406b500 msgr2=0x7fcac406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.175+0000 7fcaecc00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcac406b500 0x7fcac406d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcad8000f60 tx=0x7fcad8009040 comp rx=0 tx=0).stop 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.175+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 msgr2=0x7fcae8157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.175+0000 7fcaecc00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae8157080 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcae0036040 tx=0x7fcae0004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 shutdown_connections 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae8154d70 0x7fcae8157080 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcac406b500 0x7fcac406d9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcae8152610 0x7fcae8151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 --2- 172.21.15.67:0/1753855905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcae80cb130 0x7fcae8141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 >> 172.21.15.67:0/1753855905 conn(0x7fcae80ca610 msgr2=0x7fcae806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 shutdown_connections 2024-02-26T10:38:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.176+0000 7fcaecc00640 1 -- 172.21.15.67:0/1753855905 wait complete. 2024-02-26T10:38:45.197 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-02-26T10:38:44.323168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219204,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755516,"statfs":{"total":773094113280,"available":772869648384,"internally_reserved":0,"allocated":2957312,"data_stored":1779452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.173267"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237702+0000","last_change":"2024-02-26T10:38:35.196643+0000","last_active":"2024-02-26T10:38:37.237702+0000","last_peered":"2024-02-26T10:38:37.237702+0000","last_clean":"2024-02-26T10:38:37.237702+0000","last_became_active":"2024-02-26T10:38:35.195966+0000","last_became_peered":"2024-02-26T10:38:35.195966+0000","last_unstale":"2024-02-26T10:38:37.237702+0000","last_undegraded":"2024-02-26T10:38:37.237702+0000","last_fullsized":"2024-02-26T10:38:37.237702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174389+0000","last_change":"2024-02-26T10:38:35.189849+0000","last_active":"2024-02-26T10:38:36.174389+0000","last_peered":"2024-02-26T10:38:36.174389+0000","last_clean":"2024-02-26T10:38:36.174389+0000","last_became_active":"2024-02-26T10:38:35.189290+0000","last_became_peered":"2024-02-26T10:38:35.189290+0000","last_unstale":"2024-02-26T10:38:36.174389+0000","last_undegraded":"2024-02-26T10:38:36.174389+0000","last_fullsized":"2024-02-26T10:38:36.174389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178614+0000","last_change":"2024-02-26T10:38:35.184041+0000","last_active":"2024-02-26T10:38:36.178614+0000","last_peered":"2024-02-26T10:38:36.178614+0000","last_clean":"2024-02-26T10:38:36.178614+0000","last_became_active":"2024-02-26T10:38:35.181755+0000","last_became_peered":"2024-02-26T10:38:35.181755+0000","last_unstale":"2024-02-26T10:38:36.178614+0000","last_undegraded":"2024-02-26T10:38:36.178614+0000","last_fullsized":"2024-02-26T10:38:36.178614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235190+0000","last_change":"2024-02-26T10:38:35.199057+0000","last_active":"2024-02-26T10:38:37.235190+0000","last_peered":"2024-02-26T10:38:37.235190+0000","last_clean":"2024-02-26T10:38:37.235190+0000","last_became_active":"2024-02-26T10:38:35.198578+0000","last_became_peered":"2024-02-26T10:38:35.198578+0000","last_unstale":"2024-02-26T10:38:37.235190+0000","last_undegraded":"2024-02-26T10:38:37.235190+0000","last_fullsized":"2024-02-26T10:38:37.235190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235490+0000","last_change":"2024-02-26T10:38:35.188780+0000","last_active":"2024-02-26T10:38:37.235490+0000","last_peered":"2024-02-26T10:38:37.235490+0000","last_clean":"2024-02-26T10:38:37.235490+0000","last_became_active":"2024-02-26T10:38:35.187459+0000","last_became_peered":"2024-02-26T10:38:35.187459+0000","last_unstale":"2024-02-26T10:38:37.235490+0000","last_undegraded":"2024-02-26T10:38:37.235490+0000","last_fullsized":"2024-02-26T10:38:37.235490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235076+0000","last_change":"2024-02-26T10:38:35.204704+0000","last_active":"2024-02-26T10:38:37.235076+0000","last_peered":"2024-02-26T10:38:37.235076+0000","last_clean":"2024-02-26T10:38:37.235076+0000","last_became_active":"2024-02-26T10:38:35.204329+0000","last_became_peered":"2024-02-26T10:38:35.204329+0000","last_unstale":"2024-02-26T10:38:37.235076+0000","last_undegraded":"2024-02-26T10:38:37.235076+0000","last_fullsized":"2024-02-26T10:38:37.235076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237988+0000","last_change":"2024-02-26T10:38:35.193948+0000","last_active":"2024-02-26T10:38:37.237988+0000","last_peered":"2024-02-26T10:38:37.237988+0000","last_clean":"2024-02-26T10:38:37.237988+0000","last_became_active":"2024-02-26T10:38:35.192130+0000","last_became_peered":"2024-02-26T10:38:35.192130+0000","last_unstale":"2024-02-26T10:38:37.237988+0000","last_undegraded":"2024-02-26T10:38:37.237988+0000","last_fullsized":"2024-02-26T10:38:37.237988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:56:35.875738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234899+0000","last_change":"2024-02-26T10:38:35.205622+0000","last_active":"2024-02-26T10:38:37.234899+0000","last_peered":"2024-02-26T10:38:37.234899+0000","last_clean":"2024-02-26T10:38:37.234899+0000","last_became_active":"2024-02-26T10:38:35.205359+0000","last_became_peered":"2024-02-26T10:38:35.205359+0000","last_unstale":"2024-02-26T10:38:37.234899+0000","last_undegraded":"2024-02-26T10:38:37.234899+0000","last_fullsized":"2024-02-26T10:38:37.234899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237849+0000","last_change":"2024-02-26T10:38:35.202549+0000","last_active":"2024-02-26T10:38:37.237849+0000","last_peered":"2024-02-26T10:38:37.237849+0000","last_clean":"2024-02-26T10:38:37.237849+0000","last_became_active":"2024-02-26T10:38:35.201568+0000","last_became_peered":"2024-02-26T10:38:35.201568+0000","last_unstale":"2024-02-26T10:38:37.237849+0000","last_undegraded":"2024-02-26T10:38:37.237849+0000","last_fullsized":"2024-02-26T10:38:37.237849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182530+0000","last_change":"2024-02-26T10:38:35.206569+0000","last_active":"2024-02-26T10:38:36.182530+0000","last_peered":"2024-02-26T10:38:36.182530+0000","last_clean":"2024-02-26T10:38:36.182530+0000","last_became_active":"2024-02-26T10:38:35.206285+0000","last_became_peered":"2024-02-26T10:38:35.206285+0000","last_unstale":"2024-02-26T10:38:36.182530+0000","last_undegraded":"2024-02-26T10:38:36.182530+0000","last_fullsized":"2024-02-26T10:38:36.182530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182446+0000","last_change":"2024-02-26T10:38:35.209104+0000","last_active":"2024-02-26T10:38:36.182446+0000","last_peered":"2024-02-26T10:38:36.182446+0000","last_clean":"2024-02-26T10:38:36.182446+0000","last_became_active":"2024-02-26T10:38:35.208841+0000","last_became_peered":"2024-02-26T10:38:35.208841+0000","last_unstale":"2024-02-26T10:38:36.182446+0000","last_undegraded":"2024-02-26T10:38:36.182446+0000","last_fullsized":"2024-02-26T10:38:36.182446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175483+0000","last_change":"2024-02-26T10:38:35.198390+0000","last_active":"2024-02-26T10:38:36.175483+0000","last_peered":"2024-02-26T10:38:36.175483+0000","last_clean":"2024-02-26T10:38:36.175483+0000","last_became_active":"2024-02-26T10:38:35.197895+0000","last_became_peered":"2024-02-26T10:38:35.197895+0000","last_unstale":"2024-02-26T10:38:36.175483+0000","last_undegraded":"2024-02-26T10:38:36.175483+0000","last_fullsized":"2024-02-26T10:38:36.175483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:01.842347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.192330+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.192046+0000","last_became_peered":"2024-02-26T10:38:35.192046+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177330+0000","last_change":"2024-02-26T10:38:35.181873+0000","last_active":"2024-02-26T10:38:36.177330+0000","last_peered":"2024-02-26T10:38:36.177330+0000","last_clean":"2024-02-26T10:38:36.177330+0000","last_became_active":"2024-02-26T10:38:35.180543+0000","last_became_peered":"2024-02-26T10:38:35.180543+0000","last_unstale":"2024-02-26T10:38:36.177330+0000","last_undegraded":"2024-02-26T10:38:36.177330+0000","last_fullsized":"2024-02-26T10:38:36.177330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:04:47.351870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173628+0000","last_change":"2024-02-26T10:38:35.186385+0000","last_active":"2024-02-26T10:38:36.173628+0000","last_peered":"2024-02-26T10:38:36.173628+0000","last_clean":"2024-02-26T10:38:36.173628+0000","last_became_active":"2024-02-26T10:38:35.186103+0000","last_became_peered":"2024-02-26T10:38:35.186103+0000","last_unstale":"2024-02-26T10:38:36.173628+0000","last_undegraded":"2024-02-26T10:38:36.173628+0000","last_fullsized":"2024-02-26T10:38:36.173628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234734+0000","last_change":"2024-02-26T10:38:35.203847+0000","last_active":"2024-02-26T10:38:37.234734+0000","last_peered":"2024-02-26T10:38:37.234734+0000","last_clean":"2024-02-26T10:38:37.234734+0000","last_became_active":"2024-02-26T10:38:35.201819+0000","last_became_peered":"2024-02-26T10:38:35.201819+0000","last_unstale":"2024-02-26T10:38:37.234734+0000","last_undegraded":"2024-02-26T10:38:37.234734+0000","last_fullsized":"2024-02-26T10:38:37.234734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237593+0000","last_change":"2024-02-26T10:38:35.205600+0000","last_active":"2024-02-26T10:38:37.237593+0000","last_peered":"2024-02-26T10:38:37.237593+0000","last_clean":"2024-02-26T10:38:37.237593+0000","last_became_active":"2024-02-26T10:38:35.205272+0000","last_became_peered":"2024-02-26T10:38:35.205272+0000","last_unstale":"2024-02-26T10:38:37.237593+0000","last_undegraded":"2024-02-26T10:38:37.237593+0000","last_fullsized":"2024-02-26T10:38:37.237593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237554+0000","last_change":"2024-02-26T10:38:35.191539+0000","last_active":"2024-02-26T10:38:37.237554+0000","last_peered":"2024-02-26T10:38:37.237554+0000","last_clean":"2024-02-26T10:38:37.237554+0000","last_became_active":"2024-02-26T10:38:35.191258+0000","last_became_peered":"2024-02-26T10:38:35.191258+0000","last_unstale":"2024-02-26T10:38:37.237554+0000","last_undegraded":"2024-02-26T10:38:37.237554+0000","last_fullsized":"2024-02-26T10:38:37.237554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182184+0000","last_change":"2024-02-26T10:38:35.197155+0000","last_active":"2024-02-26T10:38:36.182184+0000","last_peered":"2024-02-26T10:38:36.182184+0000","last_clean":"2024-02-26T10:38:36.182184+0000","last_became_active":"2024-02-26T10:38:35.192752+0000","last_became_peered":"2024-02-26T10:38:35.192752+0000","last_unstale":"2024-02-26T10:38:36.182184+0000","last_undegraded":"2024-02-26T10:38:36.182184+0000","last_fullsized":"2024-02-26T10:38:36.182184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178148+0000","last_change":"2024-02-26T10:38:35.192588+0000","last_active":"2024-02-26T10:38:36.178148+0000","last_peered":"2024-02-26T10:38:36.178148+0000","last_clean":"2024-02-26T10:38:36.178148+0000","last_became_active":"2024-02-26T10:38:35.192244+0000","last_became_peered":"2024-02-26T10:38:35.192244+0000","last_unstale":"2024-02-26T10:38:36.178148+0000","last_undegraded":"2024-02-26T10:38:36.178148+0000","last_fullsized":"2024-02-26T10:38:36.178148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235257+0000","last_change":"2024-02-26T10:38:35.193557+0000","last_active":"2024-02-26T10:38:37.235257+0000","last_peered":"2024-02-26T10:38:37.235257+0000","last_clean":"2024-02-26T10:38:37.235257+0000","last_became_active":"2024-02-26T10:38:35.192814+0000","last_became_peered":"2024-02-26T10:38:35.192814+0000","last_unstale":"2024-02-26T10:38:37.235257+0000","last_undegraded":"2024-02-26T10:38:37.235257+0000","last_fullsized":"2024-02-26T10:38:37.235257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235534+0000","last_change":"2024-02-26T10:38:35.188741+0000","last_active":"2024-02-26T10:38:37.235534+0000","last_peered":"2024-02-26T10:38:37.235534+0000","last_clean":"2024-02-26T10:38:37.235534+0000","last_became_active":"2024-02-26T10:38:35.188296+0000","last_became_peered":"2024-02-26T10:38:35.188296+0000","last_unstale":"2024-02-26T10:38:37.235534+0000","last_undegraded":"2024-02-26T10:38:37.235534+0000","last_fullsized":"2024-02-26T10:38:37.235534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235999+0000","last_change":"2024-02-26T10:38:35.193665+0000","last_active":"2024-02-26T10:38:37.235999+0000","last_peered":"2024-02-26T10:38:37.235999+0000","last_clean":"2024-02-26T10:38:37.235999+0000","last_became_active":"2024-02-26T10:38:35.193314+0000","last_became_peered":"2024-02-26T10:38:35.193314+0000","last_unstale":"2024-02-26T10:38:37.235999+0000","last_undegraded":"2024-02-26T10:38:37.235999+0000","last_fullsized":"2024-02-26T10:38:37.235999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236084+0000","last_change":"2024-02-26T10:38:35.186570+0000","last_active":"2024-02-26T10:38:37.236084+0000","last_peered":"2024-02-26T10:38:37.236084+0000","last_clean":"2024-02-26T10:38:37.236084+0000","last_became_active":"2024-02-26T10:38:35.186248+0000","last_became_peered":"2024-02-26T10:38:35.186248+0000","last_unstale":"2024-02-26T10:38:37.236084+0000","last_undegraded":"2024-02-26T10:38:37.236084+0000","last_fullsized":"2024-02-26T10:38:37.236084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182707+0000","last_change":"2024-02-26T10:38:35.197545+0000","last_active":"2024-02-26T10:38:36.182707+0000","last_peered":"2024-02-26T10:38:36.182707+0000","last_clean":"2024-02-26T10:38:36.182707+0000","last_became_active":"2024-02-26T10:38:35.197220+0000","last_became_peered":"2024-02-26T10:38:35.197220+0000","last_unstale":"2024-02-26T10:38:36.182707+0000","last_undegraded":"2024-02-26T10:38:36.182707+0000","last_fullsized":"2024-02-26T10:38:36.182707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173665+0000","last_change":"2024-02-26T10:38:35.174943+0000","last_active":"2024-02-26T10:38:36.173665+0000","last_peered":"2024-02-26T10:38:36.173665+0000","last_clean":"2024-02-26T10:38:36.173665+0000","last_became_active":"2024-02-26T10:38:35.174618+0000","last_became_peered":"2024-02-26T10:38:35.174618+0000","last_unstale":"2024-02-26T10:38:36.173665+0000","last_undegraded":"2024-02-26T10:38:36.173665+0000","last_fullsized":"2024-02-26T10:38:36.173665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235860+0000","last_change":"2024-02-26T10:38:35.188806+0000","last_active":"2024-02-26T10:38:37.235860+0000","last_peered":"2024-02-26T10:38:37.235860+0000","last_clean":"2024-02-26T10:38:37.235860+0000","last_became_active":"2024-02-26T10:38:35.188539+0000","last_became_peered":"2024-02-26T10:38:35.188539+0000","last_unstale":"2024-02-26T10:38:37.235860+0000","last_undegraded":"2024-02-26T10:38:37.235860+0000","last_fullsized":"2024-02-26T10:38:37.235860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178458+0000","last_change":"2024-02-26T10:38:35.183491+0000","last_active":"2024-02-26T10:38:36.178458+0000","last_peered":"2024-02-26T10:38:36.178458+0000","last_clean":"2024-02-26T10:38:36.178458+0000","last_became_active":"2024-02-26T10:38:35.183153+0000","last_became_peered":"2024-02-26T10:38:35.183153+0000","last_unstale":"2024-02-26T10:38:36.178458+0000","last_undegraded":"2024-02-26T10:38:36.178458+0000","last_fullsized":"2024-02-26T10:38:36.178458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235539+0000","last_change":"2024-02-26T10:38:35.185342+0000","last_active":"2024-02-26T10:38:37.235539+0000","last_peered":"2024-02-26T10:38:37.235539+0000","last_clean":"2024-02-26T10:38:37.235539+0000","last_became_active":"2024-02-26T10:38:35.185067+0000","last_became_peered":"2024-02-26T10:38:35.185067+0000","last_unstale":"2024-02-26T10:38:37.235539+0000","last_undegraded":"2024-02-26T10:38:37.235539+0000","last_fullsized":"2024-02-26T10:38:37.235539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181690+0000","last_change":"2024-02-26T10:38:35.191702+0000","last_active":"2024-02-26T10:38:36.181690+0000","last_peered":"2024-02-26T10:38:36.181690+0000","last_clean":"2024-02-26T10:38:36.181690+0000","last_became_active":"2024-02-26T10:38:35.191416+0000","last_became_peered":"2024-02-26T10:38:35.191416+0000","last_unstale":"2024-02-26T10:38:36.181690+0000","last_undegraded":"2024-02-26T10:38:36.181690+0000","last_fullsized":"2024-02-26T10:38:36.181690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175512+0000","last_change":"2024-02-26T10:38:35.169056+0000","last_active":"2024-02-26T10:38:36.175512+0000","last_peered":"2024-02-26T10:38:36.175512+0000","last_clean":"2024-02-26T10:38:36.175512+0000","last_became_active":"2024-02-26T10:38:35.168640+0000","last_became_peered":"2024-02-26T10:38:35.168640+0000","last_unstale":"2024-02-26T10:38:36.175512+0000","last_undegraded":"2024-02-26T10:38:36.175512+0000","last_fullsized":"2024-02-26T10:38:36.175512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:44.237255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235514+0000","last_change":"2024-02-26T10:38:34.169112+0000","last_active":"2024-02-26T10:38:37.235514+0000","last_peered":"2024-02-26T10:38:37.235514+0000","last_clean":"2024-02-26T10:38:37.235514+0000","last_became_active":"2024-02-26T10:38:34.168796+0000","last_became_peered":"2024-02-26T10:38:34.168796+0000","last_unstale":"2024-02-26T10:38:37.235514+0000","last_undegraded":"2024-02-26T10:38:37.235514+0000","last_fullsized":"2024-02-26T10:38:37.235514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:29:24.719556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178388+0000","last_change":"2024-02-26T10:38:35.187182+0000","last_active":"2024-02-26T10:38:36.178388+0000","last_peered":"2024-02-26T10:38:36.178388+0000","last_clean":"2024-02-26T10:38:36.178388+0000","last_became_active":"2024-02-26T10:38:35.185661+0000","last_became_peered":"2024-02-26T10:38:35.185661+0000","last_unstale":"2024-02-26T10:38:36.178388+0000","last_undegraded":"2024-02-26T10:38:36.178388+0000","last_fullsized":"2024-02-26T10:38:36.178388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181759+0000","last_change":"2024-02-26T10:38:34.179842+0000","last_active":"2024-02-26T10:38:36.181759+0000","last_peered":"2024-02-26T10:38:36.181759+0000","last_clean":"2024-02-26T10:38:36.181759+0000","last_became_active":"2024-02-26T10:38:34.179535+0000","last_became_peered":"2024-02-26T10:38:34.179535+0000","last_unstale":"2024-02-26T10:38:36.181759+0000","last_undegraded":"2024-02-26T10:38:36.181759+0000","last_fullsized":"2024-02-26T10:38:36.181759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235171+0000","last_change":"2024-02-26T10:38:34.167613+0000","last_active":"2024-02-26T10:38:37.235171+0000","last_peered":"2024-02-26T10:38:37.235171+0000","last_clean":"2024-02-26T10:38:37.235171+0000","last_became_active":"2024-02-26T10:38:34.167252+0000","last_became_peered":"2024-02-26T10:38:34.167252+0000","last_unstale":"2024-02-26T10:38:37.235171+0000","last_undegraded":"2024-02-26T10:38:37.235171+0000","last_fullsized":"2024-02-26T10:38:37.235171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235040+0000","last_change":"2024-02-26T10:38:35.185939+0000","last_active":"2024-02-26T10:38:37.235040+0000","last_peered":"2024-02-26T10:38:37.235040+0000","last_clean":"2024-02-26T10:38:37.235040+0000","last_became_active":"2024-02-26T10:38:35.183226+0000","last_became_peered":"2024-02-26T10:38:35.183226+0000","last_unstale":"2024-02-26T10:38:37.235040+0000","last_undegraded":"2024-02-26T10:38:37.235040+0000","last_fullsized":"2024-02-26T10:38:37.235040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236134+0000","last_change":"2024-02-26T10:38:34.186454+0000","last_active":"2024-02-26T10:38:37.236134+0000","last_peered":"2024-02-26T10:38:37.236134+0000","last_clean":"2024-02-26T10:38:37.236134+0000","last_became_active":"2024-02-26T10:38:34.185676+0000","last_became_peered":"2024-02-26T10:38:34.185676+0000","last_unstale":"2024-02-26T10:38:37.236134+0000","last_undegraded":"2024-02-26T10:38:37.236134+0000","last_fullsized":"2024-02-26T10:38:37.236134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173408+0000","last_change":"2024-02-26T10:38:35.176977+0000","last_active":"2024-02-26T10:38:36.173408+0000","last_peered":"2024-02-26T10:38:36.173408+0000","last_clean":"2024-02-26T10:38:36.173408+0000","last_became_active":"2024-02-26T10:38:35.175624+0000","last_became_peered":"2024-02-26T10:38:35.175624+0000","last_unstale":"2024-02-26T10:38:36.173408+0000","last_undegraded":"2024-02-26T10:38:36.173408+0000","last_fullsized":"2024-02-26T10:38:36.173408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182463+0000","last_change":"2024-02-26T10:38:34.188391+0000","last_active":"2024-02-26T10:38:36.182463+0000","last_peered":"2024-02-26T10:38:36.182463+0000","last_clean":"2024-02-26T10:38:36.182463+0000","last_became_active":"2024-02-26T10:38:34.188077+0000","last_became_peered":"2024-02-26T10:38:34.188077+0000","last_unstale":"2024-02-26T10:38:36.182463+0000","last_undegraded":"2024-02-26T10:38:36.182463+0000","last_fullsized":"2024-02-26T10:38:36.182463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236526+0000","last_change":"2024-02-26T10:38:35.199765+0000","last_active":"2024-02-26T10:38:37.236526+0000","last_peered":"2024-02-26T10:38:37.236526+0000","last_clean":"2024-02-26T10:38:37.236526+0000","last_became_active":"2024-02-26T10:38:35.199478+0000","last_became_peered":"2024-02-26T10:38:35.199478+0000","last_unstale":"2024-02-26T10:38:37.236526+0000","last_undegraded":"2024-02-26T10:38:37.236526+0000","last_fullsized":"2024-02-26T10:38:37.236526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236673+0000","last_change":"2024-02-26T10:38:34.189159+0000","last_active":"2024-02-26T10:38:37.236673+0000","last_peered":"2024-02-26T10:38:37.236673+0000","last_clean":"2024-02-26T10:38:37.236673+0000","last_became_active":"2024-02-26T10:38:34.187774+0000","last_became_peered":"2024-02-26T10:38:34.187774+0000","last_unstale":"2024-02-26T10:38:37.236673+0000","last_undegraded":"2024-02-26T10:38:37.236673+0000","last_fullsized":"2024-02-26T10:38:37.236673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178929+0000","last_change":"2024-02-26T10:38:35.198495+0000","last_active":"2024-02-26T10:38:36.178929+0000","last_peered":"2024-02-26T10:38:36.178929+0000","last_clean":"2024-02-26T10:38:36.178929+0000","last_became_active":"2024-02-26T10:38:35.198049+0000","last_became_peered":"2024-02-26T10:38:35.198049+0000","last_unstale":"2024-02-26T10:38:36.178929+0000","last_undegraded":"2024-02-26T10:38:36.178929+0000","last_fullsized":"2024-02-26T10:38:36.178929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173743+0000","last_change":"2024-02-26T10:38:34.182673+0000","last_active":"2024-02-26T10:38:36.173743+0000","last_peered":"2024-02-26T10:38:36.173743+0000","last_clean":"2024-02-26T10:38:36.173743+0000","last_became_active":"2024-02-26T10:38:34.182369+0000","last_became_peered":"2024-02-26T10:38:34.182369+0000","last_unstale":"2024-02-26T10:38:36.173743+0000","last_undegraded":"2024-02-26T10:38:36.173743+0000","last_fullsized":"2024-02-26T10:38:36.173743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175617+0000","last_change":"2024-02-26T10:38:35.193526+0000","last_active":"2024-02-26T10:38:36.175617+0000","last_peered":"2024-02-26T10:38:36.175617+0000","last_clean":"2024-02-26T10:38:36.175617+0000","last_became_active":"2024-02-26T10:38:35.193192+0000","last_became_peered":"2024-02-26T10:38:35.193192+0000","last_unstale":"2024-02-26T10:38:36.175617+0000","last_undegraded":"2024-02-26T10:38:36.175617+0000","last_fullsized":"2024-02-26T10:38:36.175617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:08.969223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182341+0000","last_change":"2024-02-26T10:38:34.178005+0000","last_active":"2024-02-26T10:38:36.182341+0000","last_peered":"2024-02-26T10:38:36.182341+0000","last_clean":"2024-02-26T10:38:36.182341+0000","last_became_active":"2024-02-26T10:38:34.177684+0000","last_became_peered":"2024-02-26T10:38:34.177684+0000","last_unstale":"2024-02-26T10:38:36.182341+0000","last_undegraded":"2024-02-26T10:38:36.182341+0000","last_fullsized":"2024-02-26T10:38:36.182341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.196333+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.191241+0000","last_became_peered":"2024-02-26T10:38:35.191241+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178541+0000","last_change":"2024-02-26T10:38:34.178253+0000","last_active":"2024-02-26T10:38:36.178541+0000","last_peered":"2024-02-26T10:38:36.178541+0000","last_clean":"2024-02-26T10:38:36.178541+0000","last_became_active":"2024-02-26T10:38:34.177880+0000","last_became_peered":"2024-02-26T10:38:34.177880+0000","last_unstale":"2024-02-26T10:38:36.178541+0000","last_undegraded":"2024-02-26T10:38:36.178541+0000","last_fullsized":"2024-02-26T10:38:36.178541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177760+0000","last_change":"2024-02-26T10:38:35.196164+0000","last_active":"2024-02-26T10:38:36.177760+0000","last_peered":"2024-02-26T10:38:36.177760+0000","last_clean":"2024-02-26T10:38:36.177760+0000","last_became_active":"2024-02-26T10:38:35.195846+0000","last_became_peered":"2024-02-26T10:38:35.195846+0000","last_unstale":"2024-02-26T10:38:36.177760+0000","last_undegraded":"2024-02-26T10:38:36.177760+0000","last_fullsized":"2024-02-26T10:38:36.177760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:36.847842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235392+0000","last_change":"2024-02-26T10:38:34.180900+0000","last_active":"2024-02-26T10:38:37.235392+0000","last_peered":"2024-02-26T10:38:37.235392+0000","last_clean":"2024-02-26T10:38:37.235392+0000","last_became_active":"2024-02-26T10:38:34.180316+0000","last_became_peered":"2024-02-26T10:38:34.180316+0000","last_unstale":"2024-02-26T10:38:37.235392+0000","last_undegraded":"2024-02-26T10:38:37.235392+0000","last_fullsized":"2024-02-26T10:38:37.235392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235901+0000","last_change":"2024-02-26T10:38:35.188962+0000","last_active":"2024-02-26T10:38:37.235901+0000","last_peered":"2024-02-26T10:38:37.235901+0000","last_clean":"2024-02-26T10:38:37.235901+0000","last_became_active":"2024-02-26T10:38:35.188506+0000","last_became_peered":"2024-02-26T10:38:35.188506+0000","last_unstale":"2024-02-26T10:38:37.235901+0000","last_undegraded":"2024-02-26T10:38:37.235901+0000","last_fullsized":"2024-02-26T10:38:37.235901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181772+0000","last_change":"2024-02-26T10:38:34.180239+0000","last_active":"2024-02-26T10:38:36.181772+0000","last_peered":"2024-02-26T10:38:36.181772+0000","last_clean":"2024-02-26T10:38:36.181772+0000","last_became_active":"2024-02-26T10:38:34.179109+0000","last_became_peered":"2024-02-26T10:38:34.179109+0000","last_unstale":"2024-02-26T10:38:36.181772+0000","last_undegraded":"2024-02-26T10:38:36.181772+0000","last_fullsized":"2024-02-26T10:38:36.181772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181887+0000","last_change":"2024-02-26T10:38:35.196600+0000","last_active":"2024-02-26T10:38:36.181887+0000","last_peered":"2024-02-26T10:38:36.181887+0000","last_clean":"2024-02-26T10:38:36.181887+0000","last_became_active":"2024-02-26T10:38:35.192063+0000","last_became_peered":"2024-02-26T10:38:35.192063+0000","last_unstale":"2024-02-26T10:38:36.181887+0000","last_undegraded":"2024-02-26T10:38:36.181887+0000","last_fullsized":"2024-02-26T10:38:36.181887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237658+0000","last_change":"2024-02-26T10:38:35.190544+0000","last_active":"2024-02-26T10:38:37.237658+0000","last_peered":"2024-02-26T10:38:37.237658+0000","last_clean":"2024-02-26T10:38:37.237658+0000","last_became_active":"2024-02-26T10:38:35.190066+0000","last_became_peered":"2024-02-26T10:38:35.190066+0000","last_unstale":"2024-02-26T10:38:37.237658+0000","last_undegraded":"2024-02-26T10:38:37.237658+0000","last_fullsized":"2024-02-26T10:38:37.237658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175919+0000","last_change":"2024-02-26T10:38:34.165622+0000","last_active":"2024-02-26T10:38:36.175919+0000","last_peered":"2024-02-26T10:38:36.175919+0000","last_clean":"2024-02-26T10:38:36.175919+0000","last_became_active":"2024-02-26T10:38:34.165165+0000","last_became_peered":"2024-02-26T10:38:34.165165+0000","last_unstale":"2024-02-26T10:38:36.175919+0000","last_undegraded":"2024-02-26T10:38:36.175919+0000","last_fullsized":"2024-02-26T10:38:36.175919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:02.556621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236130+0000","last_change":"2024-02-26T10:38:35.191334+0000","last_active":"2024-02-26T10:38:37.236130+0000","last_peered":"2024-02-26T10:38:37.236130+0000","last_clean":"2024-02-26T10:38:37.236130+0000","last_became_active":"2024-02-26T10:38:35.187187+0000","last_became_peered":"2024-02-26T10:38:35.187187+0000","last_unstale":"2024-02-26T10:38:37.236130+0000","last_undegraded":"2024-02-26T10:38:37.236130+0000","last_fullsized":"2024-02-26T10:38:37.236130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.238041+0000","last_change":"2024-02-26T10:38:35.189775+0000","last_active":"2024-02-26T10:38:37.238041+0000","last_peered":"2024-02-26T10:38:37.238041+0000","last_clean":"2024-02-26T10:38:37.238041+0000","last_became_active":"2024-02-26T10:38:35.189470+0000","last_became_peered":"2024-02-26T10:38:35.189470+0000","last_unstale":"2024-02-26T10:38:37.238041+0000","last_undegraded":"2024-02-26T10:38:37.238041+0000","last_fullsized":"2024-02-26T10:38:37.238041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:56:01.655512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174043+0000","last_change":"2024-02-26T10:38:34.180005+0000","last_active":"2024-02-26T10:38:36.174043+0000","last_peered":"2024-02-26T10:38:36.174043+0000","last_clean":"2024-02-26T10:38:36.174043+0000","last_became_active":"2024-02-26T10:38:34.179694+0000","last_became_peered":"2024-02-26T10:38:34.179694+0000","last_unstale":"2024-02-26T10:38:36.174043+0000","last_undegraded":"2024-02-26T10:38:36.174043+0000","last_fullsized":"2024-02-26T10:38:36.174043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236746+0000","last_change":"2024-02-26T10:38:35.190450+0000","last_active":"2024-02-26T10:38:37.236746+0000","last_peered":"2024-02-26T10:38:37.236746+0000","last_clean":"2024-02-26T10:38:37.236746+0000","last_became_active":"2024-02-26T10:38:35.187303+0000","last_became_peered":"2024-02-26T10:38:35.187303+0000","last_unstale":"2024-02-26T10:38:37.236746+0000","last_undegraded":"2024-02-26T10:38:37.236746+0000","last_fullsized":"2024-02-26T10:38:37.236746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235843+0000","last_change":"2024-02-26T10:38:35.196564+0000","last_active":"2024-02-26T10:38:37.235843+0000","last_peered":"2024-02-26T10:38:37.235843+0000","last_clean":"2024-02-26T10:38:37.235843+0000","last_became_active":"2024-02-26T10:38:35.196286+0000","last_became_peered":"2024-02-26T10:38:35.196286+0000","last_unstale":"2024-02-26T10:38:37.235843+0000","last_undegraded":"2024-02-26T10:38:37.235843+0000","last_fullsized":"2024-02-26T10:38:37.235843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235369+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:37.235369+0000","last_peered":"2024-02-26T10:38:37.235369+0000","last_clean":"2024-02-26T10:38:37.235369+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:37.235369+0000","last_undegraded":"2024-02-26T10:38:37.235369+0000","last_fullsized":"2024-02-26T10:38:37.235369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234740+0000","last_change":"2024-02-26T10:38:34.181165+0000","last_active":"2024-02-26T10:38:37.234740+0000","last_peered":"2024-02-26T10:38:37.234740+0000","last_clean":"2024-02-26T10:38:37.234740+0000","last_became_active":"2024-02-26T10:38:34.180374+0000","last_became_peered":"2024-02-26T10:38:34.180374+0000","last_unstale":"2024-02-26T10:38:37.234740+0000","last_undegraded":"2024-02-26T10:38:37.234740+0000","last_fullsized":"2024-02-26T10:38:37.234740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235370+0000","last_change":"2024-02-26T10:38:35.205573+0000","last_active":"2024-02-26T10:38:37.235370+0000","last_peered":"2024-02-26T10:38:37.235370+0000","last_clean":"2024-02-26T10:38:37.235370+0000","last_became_active":"2024-02-26T10:38:35.205295+0000","last_became_peered":"2024-02-26T10:38:35.205295+0000","last_unstale":"2024-02-26T10:38:37.235370+0000","last_undegraded":"2024-02-26T10:38:37.235370+0000","last_fullsized":"2024-02-26T10:38:37.235370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234753+0000","last_change":"2024-02-26T10:38:34.177423+0000","last_active":"2024-02-26T10:38:37.234753+0000","last_peered":"2024-02-26T10:38:37.234753+0000","last_clean":"2024-02-26T10:38:37.234753+0000","last_became_active":"2024-02-26T10:38:34.175531+0000","last_became_peered":"2024-02-26T10:38:34.175531+0000","last_unstale":"2024-02-26T10:38:37.234753+0000","last_undegraded":"2024-02-26T10:38:37.234753+0000","last_fullsized":"2024-02-26T10:38:37.234753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.176766+0000","last_change":"2024-02-26T10:38:35.182828+0000","last_active":"2024-02-26T10:38:36.176766+0000","last_peered":"2024-02-26T10:38:36.176766+0000","last_clean":"2024-02-26T10:38:36.176766+0000","last_became_active":"2024-02-26T10:38:35.182453+0000","last_became_peered":"2024-02-26T10:38:35.182453+0000","last_unstale":"2024-02-26T10:38:36.176766+0000","last_undegraded":"2024-02-26T10:38:36.176766+0000","last_fullsized":"2024-02-26T10:38:36.176766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:15.325827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173393+0000","last_change":"2024-02-26T10:38:35.180693+0000","last_active":"2024-02-26T10:38:36.173393+0000","last_peered":"2024-02-26T10:38:36.173393+0000","last_clean":"2024-02-26T10:38:36.173393+0000","last_became_active":"2024-02-26T10:38:35.178787+0000","last_became_peered":"2024-02-26T10:38:35.178787+0000","last_unstale":"2024-02-26T10:38:36.173393+0000","last_undegraded":"2024-02-26T10:38:36.173393+0000","last_fullsized":"2024-02-26T10:38:36.173393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235683+0000","last_change":"2024-02-26T10:38:34.184572+0000","last_active":"2024-02-26T10:38:37.235683+0000","last_peered":"2024-02-26T10:38:37.235683+0000","last_clean":"2024-02-26T10:38:37.235683+0000","last_became_active":"2024-02-26T10:38:34.183650+0000","last_became_peered":"2024-02-26T10:38:34.183650+0000","last_unstale":"2024-02-26T10:38:37.235683+0000","last_undegraded":"2024-02-26T10:38:37.235683+0000","last_fullsized":"2024-02-26T10:38:37.235683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175481+0000","last_change":"2024-02-26T10:38:35.186511+0000","last_active":"2024-02-26T10:38:36.175481+0000","last_peered":"2024-02-26T10:38:36.175481+0000","last_clean":"2024-02-26T10:38:36.175481+0000","last_became_active":"2024-02-26T10:38:35.185321+0000","last_became_peered":"2024-02-26T10:38:35.185321+0000","last_unstale":"2024-02-26T10:38:36.175481+0000","last_undegraded":"2024-02-26T10:38:36.175481+0000","last_fullsized":"2024-02-26T10:38:36.175481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:18.018830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174062+0000","last_change":"2024-02-26T10:38:34.165105+0000","last_active":"2024-02-26T10:38:36.174062+0000","last_peered":"2024-02-26T10:38:36.174062+0000","last_clean":"2024-02-26T10:38:36.174062+0000","last_became_active":"2024-02-26T10:38:34.164756+0000","last_became_peered":"2024-02-26T10:38:34.164756+0000","last_unstale":"2024-02-26T10:38:36.174062+0000","last_undegraded":"2024-02-26T10:38:36.174062+0000","last_fullsized":"2024-02-26T10:38:36.174062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237188+0000","last_change":"2024-02-26T10:38:35.191275+0000","last_active":"2024-02-26T10:38:37.237188+0000","last_peered":"2024-02-26T10:38:37.237188+0000","last_clean":"2024-02-26T10:38:37.237188+0000","last_became_active":"2024-02-26T10:38:35.190875+0000","last_became_peered":"2024-02-26T10:38:35.190875+0000","last_unstale":"2024-02-26T10:38:37.237188+0000","last_undegraded":"2024-02-26T10:38:37.237188+0000","last_fullsized":"2024-02-26T10:38:37.237188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237699+0000","last_change":"2024-02-26T10:38:34.176727+0000","last_active":"2024-02-26T10:38:37.237699+0000","last_peered":"2024-02-26T10:38:37.237699+0000","last_clean":"2024-02-26T10:38:37.237699+0000","last_became_active":"2024-02-26T10:38:34.176422+0000","last_became_peered":"2024-02-26T10:38:34.176422+0000","last_unstale":"2024-02-26T10:38:37.237699+0000","last_undegraded":"2024-02-26T10:38:37.237699+0000","last_fullsized":"2024-02-26T10:38:37.237699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174128+0000","last_change":"2024-02-26T10:38:35.174940+0000","last_active":"2024-02-26T10:38:36.174128+0000","last_peered":"2024-02-26T10:38:36.174128+0000","last_clean":"2024-02-26T10:38:36.174128+0000","last_became_active":"2024-02-26T10:38:35.174609+0000","last_became_peered":"2024-02-26T10:38:35.174609+0000","last_unstale":"2024-02-26T10:38:36.174128+0000","last_undegraded":"2024-02-26T10:38:36.174128+0000","last_fullsized":"2024-02-26T10:38:36.174128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174557+0000","last_change":"2024-02-26T10:38:34.176149+0000","last_active":"2024-02-26T10:38:36.174557+0000","last_peered":"2024-02-26T10:38:36.174557+0000","last_clean":"2024-02-26T10:38:36.174557+0000","last_became_active":"2024-02-26T10:38:34.175800+0000","last_became_peered":"2024-02-26T10:38:34.175800+0000","last_unstale":"2024-02-26T10:38:36.174557+0000","last_undegraded":"2024-02-26T10:38:36.174557+0000","last_fullsized":"2024-02-26T10:38:36.174557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236113+0000","last_change":"2024-02-26T10:38:35.191410+0000","last_active":"2024-02-26T10:38:37.236113+0000","last_peered":"2024-02-26T10:38:37.236113+0000","last_clean":"2024-02-26T10:38:37.236113+0000","last_became_active":"2024-02-26T10:38:35.190954+0000","last_became_peered":"2024-02-26T10:38:35.190954+0000","last_unstale":"2024-02-26T10:38:37.236113+0000","last_undegraded":"2024-02-26T10:38:37.236113+0000","last_fullsized":"2024-02-26T10:38:37.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182629+0000","last_change":"2024-02-26T10:38:34.185447+0000","last_active":"2024-02-26T10:38:36.182629+0000","last_peered":"2024-02-26T10:38:36.182629+0000","last_clean":"2024-02-26T10:38:36.182629+0000","last_became_active":"2024-02-26T10:38:34.185054+0000","last_became_peered":"2024-02-26T10:38:34.185054+0000","last_unstale":"2024-02-26T10:38:36.182629+0000","last_undegraded":"2024-02-26T10:38:36.182629+0000","last_fullsized":"2024-02-26T10:38:36.182629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179198+0000","last_change":"2024-02-26T10:38:35.181510+0000","last_active":"2024-02-26T10:38:36.179198+0000","last_peered":"2024-02-26T10:38:36.179198+0000","last_clean":"2024-02-26T10:38:36.179198+0000","last_became_active":"2024-02-26T10:38:35.180830+0000","last_became_peered":"2024-02-26T10:38:35.180830+0000","last_unstale":"2024-02-26T10:38:36.179198+0000","last_undegraded":"2024-02-26T10:38:36.179198+0000","last_fullsized":"2024-02-26T10:38:36.179198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234630+0000","last_change":"2024-02-26T10:38:34.174303+0000","last_active":"2024-02-26T10:38:37.234630+0000","last_peered":"2024-02-26T10:38:37.234630+0000","last_clean":"2024-02-26T10:38:37.234630+0000","last_became_active":"2024-02-26T10:38:34.174006+0000","last_became_peered":"2024-02-26T10:38:34.174006+0000","last_unstale":"2024-02-26T10:38:37.234630+0000","last_undegraded":"2024-02-26T10:38:37.234630+0000","last_fullsized":"2024-02-26T10:38:37.234630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175416+0000","last_change":"2024-02-26T10:38:35.176832+0000","last_active":"2024-02-26T10:38:36.175416+0000","last_peered":"2024-02-26T10:38:36.175416+0000","last_clean":"2024-02-26T10:38:36.175416+0000","last_became_active":"2024-02-26T10:38:35.176305+0000","last_became_peered":"2024-02-26T10:38:35.176305+0000","last_unstale":"2024-02-26T10:38:36.175416+0000","last_undegraded":"2024-02-26T10:38:36.175416+0000","last_fullsized":"2024-02-26T10:38:36.175416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:06.286416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173704+0000","last_change":"2024-02-26T10:38:34.169425+0000","last_active":"2024-02-26T10:38:36.173704+0000","last_peered":"2024-02-26T10:38:36.173704+0000","last_clean":"2024-02-26T10:38:36.173704+0000","last_became_active":"2024-02-26T10:38:34.168217+0000","last_became_peered":"2024-02-26T10:38:34.168217+0000","last_unstale":"2024-02-26T10:38:36.173704+0000","last_undegraded":"2024-02-26T10:38:36.173704+0000","last_fullsized":"2024-02-26T10:38:36.173704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236029+0000","last_change":"2024-02-26T10:38:35.172836+0000","last_active":"2024-02-26T10:38:37.236029+0000","last_peered":"2024-02-26T10:38:37.236029+0000","last_clean":"2024-02-26T10:38:37.236029+0000","last_became_active":"2024-02-26T10:38:35.172554+0000","last_became_peered":"2024-02-26T10:38:35.172554+0000","last_unstale":"2024-02-26T10:38:37.236029+0000","last_undegraded":"2024-02-26T10:38:37.236029+0000","last_fullsized":"2024-02-26T10:38:37.236029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173693+0000","last_change":"2024-02-26T10:38:34.166377+0000","last_active":"2024-02-26T10:38:36.173693+0000","last_peered":"2024-02-26T10:38:36.173693+0000","last_clean":"2024-02-26T10:38:36.173693+0000","last_became_active":"2024-02-26T10:38:34.165963+0000","last_became_peered":"2024-02-26T10:38:34.165963+0000","last_unstale":"2024-02-26T10:38:36.173693+0000","last_undegraded":"2024-02-26T10:38:36.173693+0000","last_fullsized":"2024-02-26T10:38:36.173693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235575+0000","last_change":"2024-02-26T10:38:35.185120+0000","last_active":"2024-02-26T10:38:37.235575+0000","last_peered":"2024-02-26T10:38:37.235575+0000","last_clean":"2024-02-26T10:38:37.235575+0000","last_became_active":"2024-02-26T10:38:35.184832+0000","last_became_peered":"2024-02-26T10:38:35.184832+0000","last_unstale":"2024-02-26T10:38:37.235575+0000","last_undegraded":"2024-02-26T10:38:37.235575+0000","last_fullsized":"2024-02-26T10:38:37.235575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236461+0000","last_change":"2024-02-26T10:38:34.180225+0000","last_active":"2024-02-26T10:38:37.236461+0000","last_peered":"2024-02-26T10:38:37.236461+0000","last_clean":"2024-02-26T10:38:37.236461+0000","last_became_active":"2024-02-26T10:38:34.179909+0000","last_became_peered":"2024-02-26T10:38:34.179909+0000","last_unstale":"2024-02-26T10:38:37.236461+0000","last_undegraded":"2024-02-26T10:38:37.236461+0000","last_fullsized":"2024-02-26T10:38:37.236461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:05:58.590708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182262+0000","last_change":"2024-02-26T10:38:35.188975+0000","last_active":"2024-02-26T10:38:36.182262+0000","last_peered":"2024-02-26T10:38:36.182262+0000","last_clean":"2024-02-26T10:38:36.182262+0000","last_became_active":"2024-02-26T10:38:35.188608+0000","last_became_peered":"2024-02-26T10:38:35.188608+0000","last_unstale":"2024-02-26T10:38:36.182262+0000","last_undegraded":"2024-02-26T10:38:36.182262+0000","last_fullsized":"2024-02-26T10:38:36.182262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236117+0000","last_change":"2024-02-26T10:38:35.196289+0000","last_active":"2024-02-26T10:38:37.236117+0000","last_peered":"2024-02-26T10:38:37.236117+0000","last_clean":"2024-02-26T10:38:37.236117+0000","last_became_active":"2024-02-26T10:38:35.194751+0000","last_became_peered":"2024-02-26T10:38:35.194751+0000","last_unstale":"2024-02-26T10:38:37.236117+0000","last_undegraded":"2024-02-26T10:38:37.236117+0000","last_fullsized":"2024-02-26T10:38:37.236117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236713+0000","last_change":"2024-02-26T10:38:34.186760+0000","last_active":"2024-02-26T10:38:37.236713+0000","last_peered":"2024-02-26T10:38:37.236713+0000","last_clean":"2024-02-26T10:38:37.236713+0000","last_became_active":"2024-02-26T10:38:34.186428+0000","last_became_peered":"2024-02-26T10:38:34.186428+0000","last_unstale":"2024-02-26T10:38:37.236713+0000","last_undegraded":"2024-02-26T10:38:37.236713+0000","last_fullsized":"2024-02-26T10:38:37.236713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236518+0000","last_change":"2024-02-26T10:38:35.202616+0000","last_active":"2024-02-26T10:38:37.236518+0000","last_peered":"2024-02-26T10:38:37.236518+0000","last_clean":"2024-02-26T10:38:37.236518+0000","last_became_active":"2024-02-26T10:38:35.201498+0000","last_became_peered":"2024-02-26T10:38:35.201498+0000","last_unstale":"2024-02-26T10:38:37.236518+0000","last_undegraded":"2024-02-26T10:38:37.236518+0000","last_fullsized":"2024-02-26T10:38:37.236518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235996+0000","last_change":"2024-02-26T10:38:34.186745+0000","last_active":"2024-02-26T10:38:37.235996+0000","last_peered":"2024-02-26T10:38:37.235996+0000","last_clean":"2024-02-26T10:38:37.235996+0000","last_became_active":"2024-02-26T10:38:34.186470+0000","last_became_peered":"2024-02-26T10:38:34.186470+0000","last_unstale":"2024-02-26T10:38:37.235996+0000","last_undegraded":"2024-02-26T10:38:37.235996+0000","last_fullsized":"2024-02-26T10:38:37.235996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178984+0000","last_change":"2024-02-26T10:38:35.206276+0000","last_active":"2024-02-26T10:38:36.178984+0000","last_peered":"2024-02-26T10:38:36.178984+0000","last_clean":"2024-02-26T10:38:36.178984+0000","last_became_active":"2024-02-26T10:38:35.205932+0000","last_became_peered":"2024-02-26T10:38:35.205932+0000","last_unstale":"2024-02-26T10:38:36.178984+0000","last_undegraded":"2024-02-26T10:38:36.178984+0000","last_fullsized":"2024-02-26T10:38:36.178984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235907+0000","last_change":"2024-02-26T10:38:34.165440+0000","last_active":"2024-02-26T10:38:37.235907+0000","last_peered":"2024-02-26T10:38:37.235907+0000","last_clean":"2024-02-26T10:38:37.235907+0000","last_became_active":"2024-02-26T10:38:34.165164+0000","last_became_peered":"2024-02-26T10:38:34.165164+0000","last_unstale":"2024-02-26T10:38:37.235907+0000","last_undegraded":"2024-02-26T10:38:37.235907+0000","last_fullsized":"2024-02-26T10:38:37.235907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235740+0000","last_change":"2024-02-26T10:38:35.199753+0000","last_active":"2024-02-26T10:38:37.235740+0000","last_peered":"2024-02-26T10:38:37.235740+0000","last_clean":"2024-02-26T10:38:37.235740+0000","last_became_active":"2024-02-26T10:38:35.199470+0000","last_became_peered":"2024-02-26T10:38:35.199470+0000","last_unstale":"2024-02-26T10:38:37.235740+0000","last_undegraded":"2024-02-26T10:38:37.235740+0000","last_fullsized":"2024-02-26T10:38:37.235740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179016+0000","last_change":"2024-02-26T10:38:34.184020+0000","last_active":"2024-02-26T10:38:36.179016+0000","last_peered":"2024-02-26T10:38:36.179016+0000","last_clean":"2024-02-26T10:38:36.179016+0000","last_became_active":"2024-02-26T10:38:34.183497+0000","last_became_peered":"2024-02-26T10:38:34.183497+0000","last_unstale":"2024-02-26T10:38:36.179016+0000","last_undegraded":"2024-02-26T10:38:36.179016+0000","last_fullsized":"2024-02-26T10:38:36.179016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:25:08.410476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":74327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":819200,"data_stored":665243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:45.201 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:38:45.201 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-26T10:38:45.201 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-02-26T10:38:45.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-02-26T10:38:45.202 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-02-26T10:38:45.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.456+0000 7f57f2000640 1 Processor -- start 2024-02-26T10:38:45.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.456+0000 7f57f2000640 1 -- start start 2024-02-26T10:38:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.457+0000 7f57f2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f57ec0a94a0 0x7f57ec06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06db00 0x7f57ec0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f2000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec06df90 con 0x7f57ec06ac20 2024-02-26T10:38:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0a9a80 con 0x7f57ec0a94a0 2024-02-26T10:38:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0a9d60 con 0x7f57ec06db00 2024-02-26T10:38:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f0c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f57ec06ac20 0x7f57ec153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54494/0 (socket says 172.21.15.67:54494) 2024-02-26T10:38:45.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f0c00640 1 -- 172.21.15.67:0/1689588008 learned_addr learned my addr 172.21.15.67:0/1689588008 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:45.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57ebe00640 1 --2- 172.21.15.67:0/1689588008 >> v2:172.21.15.67:3300/0 conn(0x7f57ec0a94a0 0x7f57ec06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57f1600640 1 --2- 172.21.15.67:0/1689588008 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06db00 0x7f57ec0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57eb400640 1 -- 172.21.15.67:0/1689588008 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255686715 0 0) 0x7f57ec06df90 con 0x7f57ec06ac20 2024-02-26T10:38:45.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57eb400640 1 -- 172.21.15.67:0/1689588008 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d4003520 con 0x7f57ec06ac20 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57ebe00640 1 -- 172.21.15.67:0/1689588008 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06db00 msgr2=0x7f57ec0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57ebe00640 1 --2- 172.21.15.67:0/1689588008 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06db00 0x7f57ec0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.458+0000 7f57ebe00640 1 -- 172.21.15.67:0/1689588008 >> v1:172.21.15.67:6789/0 conn(0x7f57ec06ac20 legacy=0x7f57ec153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.459+0000 7f57ebe00640 1 -- 172.21.15.67:0/1689588008 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec0ab750 con 0x7f57ec0a94a0 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.460+0000 7f57ebe00640 1 --2- 172.21.15.67:0/1689588008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec0a94a0 0x7f57ec06b000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f57e00097a0 tx=0x7f57e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793cf7693e3c459a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.460+0000 7f57eb400640 1 -- 172.21.15.67:0/1689588008 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0035020 con 0x7f57ec0a94a0 2024-02-26T10:38:45.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.460+0000 7f57eb400640 1 -- 172.21.15.67:0/1689588008 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0031830 con 0x7f57ec0a94a0 2024-02-26T10:38:45.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.460+0000 7f57eb400640 1 -- 172.21.15.67:0/1689588008 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0031c00 con 0x7f57ec0a94a0 2024-02-26T10:38:45.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 -- 172.21.15.67:0/1689588008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec0a94a0 msgr2=0x7f57ec06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 --2- 172.21.15.67:0/1689588008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec0a94a0 0x7f57ec06b000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f57e00097a0 tx=0x7f57e002b8d0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 -- 172.21.15.67:0/1689588008 shutdown_connections 2024-02-26T10:38:45.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 --2- 172.21.15.67:0/1689588008 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06db00 0x7f57ec0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 --2- 172.21.15.67:0/1689588008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec0a94a0 0x7f57ec06b000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 -- 172.21.15.67:0/1689588008 >> 172.21.15.67:0/1689588008 conn(0x7f57ec0c63f0 msgr2=0x7f57ec0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:45.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 -- 172.21.15.67:0/1689588008 shutdown_connections 2024-02-26T10:38:45.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.461+0000 7f57f2000640 1 -- 172.21.15.67:0/1689588008 wait complete. 2024-02-26T10:38:45.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.463+0000 7f57f2000640 1 Processor -- start 2024-02-26T10:38:45.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.463+0000 7f57f2000640 1 -- start start 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06ac20 0x7f57ec1443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 0x7f57ec141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57ec0a94a0 0x7f57ec1defa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec150410 con 0x7f57ec06db00 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0ac490 con 0x7f57ec0a94a0 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec150590 con 0x7f57ec06ac20 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 0x7f57ec141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 0x7f57ec141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33520/0 (socket says 172.21.15.67:33520) 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 -- 172.21.15.67:0/4069241888 learned_addr learned my addr 172.21.15.67:0/4069241888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f0c00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06ac20 0x7f57ec1443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57f1600640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57ec0a94a0 0x7f57ec1defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 -- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06ac20 msgr2=0x7f57ec1443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06ac20 0x7f57ec1443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 -- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57ec0a94a0 msgr2=0x7f57ec1defa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57ec0a94a0 0x7f57ec1defa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.464+0000 7f57ebe00640 1 -- 172.21.15.67:0/4069241888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec1ec4e0 con 0x7f57ec06db00 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57f1600640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57ec0a94a0 0x7f57ec1defa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:45.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57ebe00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 0x7f57ec141dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57e0034040 tx=0x7f57e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:45.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0040070 con 0x7f57ec06db00 2024-02-26T10:38:45.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e003a910 con 0x7f57ec06db00 2024-02-26T10:38:45.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0035040 con 0x7f57ec06db00 2024-02-26T10:38:45.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec1dc330 con 0x7f57ec06db00 2024-02-26T10:38:45.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.465+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec1dfbf0 con 0x7f57ec06db00 2024-02-26T10:38:45.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.466+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec1dffd0 con 0x7f57ec06db00 2024-02-26T10:38:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.467+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57e0038020 con 0x7f57ec06db00 2024-02-26T10:38:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.468+0000 7f57e8c00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57d006b4d0 0x7f57d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.468+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57e00abc90 con 0x7f57ec06db00 2024-02-26T10:38:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.469+0000 7f57f0c00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57d006b4d0 0x7f57d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.470+0000 7f57f0c00640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57d006b4d0 0x7f57d006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57dc007fd0 tx=0x7f57dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:45.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.473+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57ec1dffd0 con 0x7f57ec06db00 2024-02-26T10:38:45.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.721+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f57ec074a30 con 0x7f57ec06db00 2024-02-26T10:38:45.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.730+0000 7f57e8c00640 1 -- 172.21.15.67:0/4069241888 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171781 (secure 0 0 0) 0x7f57e0031800 con 0x7f57ec06db00 2024-02-26T10:38:45.735 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57d006b4d0 msgr2=0x7f57d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57d006b4d0 0x7f57d006d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57dc007fd0 tx=0x7f57dc009040 comp rx=0 tx=0).stop 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 msgr2=0x7f57ec141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 0x7f57ec141dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57e0034040 tx=0x7f57e00046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 shutdown_connections 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57d006b4d0 0x7f57d006d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57ec0a94a0 0x7f57ec1defa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57ec06db00 0x7f57ec141dc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 --2- 172.21.15.67:0/4069241888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57ec06ac20 0x7f57ec1443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 >> 172.21.15.67:0/4069241888 conn(0x7f57ec0c63f0 msgr2=0x7f57ec14e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 shutdown_connections 2024-02-26T10:38:45.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.735+0000 7f57f2000640 1 -- 172.21.15.67:0/4069241888 wait complete. 2024-02-26T10:38:45.756 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":723923815},{"type":"v1","addr":"172.21.15.82:6833","nonce":723923815}]},"active_addr":"172.21.15.82:6833/723923815","active_change":"2024-02-26T10:38:20.302361+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":627968135}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3008906086}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2241567591}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2774684844}]}]} 2024-02-26T10:38:45.759 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-02-26T10:38:45.759 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-02-26T10:38:45.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.973+0000 7fbbbe800640 1 Processor -- start 2024-02-26T10:38:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.973+0000 7fbbbe800640 1 -- start start 2024-02-26T10:38:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.974+0000 7fbbbe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbbb806ac20 0x7fbbb806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.974+0000 7fbbbe800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb80c2310 0x7fbbb81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.974+0000 7fbbbe800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb8155f10 con 0x7fbbb8152f20 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.974+0000 7fbbbe800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb8153aa0 con 0x7fbbb806ac20 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.974+0000 7fbbbe800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c28d0 con 0x7fbbb80c2310 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbd400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbbb806ac20 0x7fbbb806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbde00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbbb8152f20 0x7fbbb8153390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54496/0 (socket says 172.21.15.67:54496) 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbd400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbbb806ac20 0x7fbbb806b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33526/0 (socket says 172.21.15.67:33526) 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbde00640 1 -- 172.21.15.67:0/1331379150 learned_addr learned my addr 172.21.15.67:0/1331379150 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbca00640 1 --2- 172.21.15.67:0/1331379150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb80c2310 0x7fbbb81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbd400640 1 -- 172.21.15.67:0/1331379150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb80c2310 msgr2=0x7fbbb81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbd400640 1 --2- 172.21.15.67:0/1331379150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb80c2310 0x7fbbb81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbd400640 1 -- 172.21.15.67:0/1331379150 >> v1:172.21.15.67:6789/0 conn(0x7fbbb8152f20 legacy=0x7fbbb8153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbb7e00640 1 -- 172.21.15.67:0/1331379150 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480346605 0 0) 0x7fbbb8155f10 con 0x7fbbb8152f20 2024-02-26T10:38:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.975+0000 7fbbbd400640 1 -- 172.21.15.67:0/1331379150 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8155150 con 0x7fbbb806ac20 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.976+0000 7fbbbd400640 1 --2- 172.21.15.67:0/1331379150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb806ac20 0x7fbbb806b000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eb4235d8e434c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbb7e00640 1 -- 172.21.15.67:0/1331379150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac038070 con 0x7fbbb806ac20 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbb7e00640 1 -- 172.21.15.67:0/1331379150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac031700 con 0x7fbbb806ac20 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbb7e00640 1 -- 172.21.15.67:0/1331379150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac031ad0 con 0x7fbbb806ac20 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 -- 172.21.15.67:0/1331379150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb806ac20 msgr2=0x7fbbb806b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 --2- 172.21.15.67:0/1331379150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb806ac20 0x7fbbb806b000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097a0 tx=0x7fbbac02b7a0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 -- 172.21.15.67:0/1331379150 shutdown_connections 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 --2- 172.21.15.67:0/1331379150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb80c2310 0x7fbbb81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 --2- 172.21.15.67:0/1331379150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb806ac20 0x7fbbb806b000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 -- 172.21.15.67:0/1331379150 >> 172.21.15.67:0/1331379150 conn(0x7fbbb80c63f0 msgr2=0x7fbbb80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.977+0000 7fbbbe800640 1 -- 172.21.15.67:0/1331379150 shutdown_connections 2024-02-26T10:38:45.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.978+0000 7fbbbe800640 1 -- 172.21.15.67:0/1331379150 wait complete. 2024-02-26T10:38:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.979+0000 7fbbbe800640 1 Processor -- start 2024-02-26T10:38:45.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.979+0000 7fbbbe800640 1 -- start start 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbe800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb806ac20 0x7fbbb81dd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbe800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 0x7fbbb814fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbe800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbb8152f20 0x7fbbb8150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbe800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81559f0 con 0x7fbbb80c2310 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbe800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8155710 con 0x7fbbb8152f20 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbe800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8148850 con 0x7fbbb806ac20 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 0x7fbbb814fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 0x7fbbb814fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33542/0 (socket says 172.21.15.67:33542) 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.980+0000 7fbbbca00640 1 -- 172.21.15.67:0/2765617335 learned_addr learned my addr 172.21.15.67:0/2765617335 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbd400640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb806ac20 0x7fbbb81dd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbde00640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbb8152f20 0x7fbbb8150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbca00640 1 -- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb806ac20 msgr2=0x7fbbb81dd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbca00640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb806ac20 0x7fbbb81dd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbca00640 1 -- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbb8152f20 msgr2=0x7fbbb8150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbca00640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbb8152f20 0x7fbbb8150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbca00640 1 -- 172.21.15.67:0/2765617335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb81dd6f0 con 0x7fbbb80c2310 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbd400640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb806ac20 0x7fbbb81dd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbde00640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbb8152f20 0x7fbbb8150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbca00640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 0x7fbbb814fa70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbba800d5f0 tx=0x7fbba800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80125f0 con 0x7fbbb80c2310 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8019070 con 0x7fbbb80c2310 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb80c4fd0 con 0x7fbbb80c2310 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.981+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80129d0 con 0x7fbbb80c2310 2024-02-26T10:38:45.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.982+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb80c4a10 con 0x7fbbb80c2310 2024-02-26T10:38:45.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.983+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb8074a30 con 0x7fbbb80c2310 2024-02-26T10:38:45.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.983+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbba8012b70 con 0x7fbbb80c2310 2024-02-26T10:38:45.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.985+0000 7fbbb5600640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb9c06b4d0 0x7fbb9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:45.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.985+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbba808a690 con 0x7fbbb80c2310 2024-02-26T10:38:45.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.985+0000 7fbbbd400640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb9c06b4d0 0x7fbb9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:45.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.986+0000 7fbbbd400640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb9c06b4d0 0x7fbb9c06d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbbac035000 tx=0x7fbbac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:45.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:45.989+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbba8059480 con 0x7fbbb80c2310 2024-02-26T10:38:46.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.189+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbb80c20f0 con 0x7fbbb80c2310 2024-02-26T10:38:46.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.192+0000 7fbbb5600640 1 -- 172.21.15.67:0/2765617335 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fbba8058d80 con 0x7fbbb80c2310 2024-02-26T10:38:46.195 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb9c06b4d0 msgr2=0x7fbb9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb9c06b4d0 0x7fbb9c06d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbbac035000 tx=0x7fbbac008030 comp rx=0 tx=0).stop 2024-02-26T10:38:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 msgr2=0x7fbbb814fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 0x7fbbb814fa70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbba800d5f0 tx=0x7fbba800dab0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 shutdown_connections 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb9c06b4d0 0x7fbb9c06d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbb8152f20 0x7fbbb8150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb80c2310 0x7fbbb814fa70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 --2- 172.21.15.67:0/2765617335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbbb806ac20 0x7fbbb81dd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 >> 172.21.15.67:0/2765617335 conn(0x7fbbb80c63f0 msgr2=0x7fbbb8153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 shutdown_connections 2024-02-26T10:38:46.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.195+0000 7fbbbe800640 1 -- 172.21.15.67:0/2765617335 wait complete. 2024-02-26T10:38:46.217 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-02-26T10:38:46.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:46.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.433+0000 7fcf7b200640 1 Processor -- start 2024-02-26T10:38:46.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.433+0000 7fcf7b200640 1 -- start start 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf7c06ac20 0x7fcf7c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c06b460 con 0x7fcf7c153610 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0a9a80 con 0x7fcf7c06ac20 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0a9d60 con 0x7fcf7c06b000 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf79e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf7c06ac20 0x7fcf7c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcf7c153610 0x7fcf7c0abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54498/0 (socket says 172.21.15.67:54498) 2024-02-26T10:38:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf7a800640 1 -- 172.21.15.67:0/3203952908 learned_addr learned my addr 172.21.15.67:0/3203952908 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.435+0000 7fcf71400640 1 --2- 172.21.15.67:0/3203952908 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.436+0000 7fcf79400640 1 -- 172.21.15.67:0/3203952908 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346632284 0 0) 0x7fcf7c06b460 con 0x7fcf7c153610 2024-02-26T10:38:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.436+0000 7fcf79400640 1 -- 172.21.15.67:0/3203952908 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf60003520 con 0x7fcf7c153610 2024-02-26T10:38:46.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.436+0000 7fcf79e00640 1 -- 172.21.15.67:0/3203952908 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 msgr2=0x7fcf7c06db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:46.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.436+0000 7fcf79e00640 1 --2- 172.21.15.67:0/3203952908 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c06db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.436+0000 7fcf79e00640 1 -- 172.21.15.67:0/3203952908 >> v1:172.21.15.67:6789/0 conn(0x7fcf7c153610 legacy=0x7fcf7c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.436+0000 7fcf79e00640 1 -- 172.21.15.67:0/3203952908 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c0ab750 con 0x7fcf7c06ac20 2024-02-26T10:38:46.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.437+0000 7fcf79e00640 1 --2- 172.21.15.67:0/3203952908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c0a94a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0097a0 tx=0x7fcf6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983464f6ced1fc11 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:46.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.437+0000 7fcf79400640 1 -- 172.21.15.67:0/3203952908 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c035020 con 0x7fcf7c06ac20 2024-02-26T10:38:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.437+0000 7fcf79400640 1 -- 172.21.15.67:0/3203952908 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c031830 con 0x7fcf7c06ac20 2024-02-26T10:38:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.437+0000 7fcf79400640 1 -- 172.21.15.67:0/3203952908 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c031c00 con 0x7fcf7c06ac20 2024-02-26T10:38:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 -- 172.21.15.67:0/3203952908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 msgr2=0x7fcf7c0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 --2- 172.21.15.67:0/3203952908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c0a94a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0097a0 tx=0x7fcf6c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 -- 172.21.15.67:0/3203952908 shutdown_connections 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 --2- 172.21.15.67:0/3203952908 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c06db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 --2- 172.21.15.67:0/3203952908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c0a94a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 -- 172.21.15.67:0/3203952908 >> 172.21.15.67:0/3203952908 conn(0x7fcf7c0c63f0 msgr2=0x7fcf7c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 -- 172.21.15.67:0/3203952908 shutdown_connections 2024-02-26T10:38:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.438+0000 7fcf7b200640 1 -- 172.21.15.67:0/3203952908 wait complete. 2024-02-26T10:38:46.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.440+0000 7fcf7b200640 1 Processor -- start 2024-02-26T10:38:46.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.440+0000 7fcf7b200640 1 -- start start 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c153610 0x7fcf7c0727c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c150410 con 0x7fcf7c06ac20 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c153aa0 con 0x7fcf7c153610 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c150590 con 0x7fcf7c06b000 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf79e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf79e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33564/0 (socket says 172.21.15.67:33564) 2024-02-26T10:38:46.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf79e00640 1 -- 172.21.15.67:0/2271643426 learned_addr learned my addr 172.21.15.67:0/2271643426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf7a800640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c153610 0x7fcf7c0727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf71400640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf79e00640 1 -- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 msgr2=0x7fcf7c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf79e00640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.441+0000 7fcf79e00640 1 -- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c153610 msgr2=0x7fcf7c0727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf79e00640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c153610 0x7fcf7c0727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf79e00640 1 -- 172.21.15.67:0/2271643426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c1ee900 con 0x7fcf7c06ac20 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf7a800640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c153610 0x7fcf7c0727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf79e00640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c071570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0098c0 tx=0x7fcf6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c048070 con 0x7fcf7c06ac20 2024-02-26T10:38:46.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c03a6f0 con 0x7fcf7c06ac20 2024-02-26T10:38:46.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c03ab00 con 0x7fcf7c06ac20 2024-02-26T10:38:46.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c14a3e0 con 0x7fcf7c06ac20 2024-02-26T10:38:46.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.442+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c071c60 con 0x7fcf7c06ac20 2024-02-26T10:38:46.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.443+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c074a30 con 0x7fcf7c06ac20 2024-02-26T10:38:46.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.444+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf6c04b070 con 0x7fcf7c06ac20 2024-02-26T10:38:46.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.445+0000 7fcf72a00640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5806b4d0 0x7fcf5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.445+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf6c0ab9e0 con 0x7fcf7c06ac20 2024-02-26T10:38:46.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.446+0000 7fcf71400640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5806b4d0 0x7fcf5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.447+0000 7fcf71400640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5806b4d0 0x7fcf5806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0ac1a0 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:46.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.450+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf6c07a970 con 0x7fcf7c06ac20 2024-02-26T10:38:46.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.649+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf7c06b9b0 con 0x7fcf7c06ac20 2024-02-26T10:38:46.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.652+0000 7fcf72a00640 1 -- 172.21.15.67:0/2271643426 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fcf6c07a270 con 0x7fcf7c06ac20 2024-02-26T10:38:46.654 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:46.655 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5806b4d0 msgr2=0x7fcf5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5806b4d0 0x7fcf5806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0ac1a0 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 msgr2=0x7fcf7c071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c071570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0098c0 tx=0x7fcf6c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 shutdown_connections 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5806b4d0 0x7fcf5806d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c153610 0x7fcf7c0727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 --2- 172.21.15.67:0/2271643426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c071570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 >> 172.21.15.67:0/2271643426 conn(0x7fcf7c0c63f0 msgr2=0x7fcf7c06daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 shutdown_connections 2024-02-26T10:38:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.655+0000 7fcf7b200640 1 -- 172.21.15.67:0/2271643426 wait complete. 2024-02-26T10:38:46.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-02-26T10:38:46.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-02-26T10:38:46.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-02-26T10:38:46.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-02-26T10:38:46.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-02-26T10:38:46.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-02-26T10:38:46.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-02-26T10:38:46.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-02-26T10:38:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.981+0000 7f097fe00640 1 Processor -- start 2024-02-26T10:38:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.981+0000 7f097fe00640 1 -- start start 2024-02-26T10:38:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.982+0000 7ff3ca000640 1 Processor -- start 2024-02-26T10:38:46.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.982+0000 7ff3ca000640 1 -- start start 2024-02-26T10:38:46.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f1f8be00640 1 Processor -- start 2024-02-26T10:38:46.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7ff3ca000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff3c4152390 0x7ff3c40c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f097fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f098006ac20 0x7f098006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f097fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f098006eff0 0x7f09800c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f097fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0980151280 0x7f09801516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f097fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800c2210 con 0x7f098006eff0 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f097fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0980150d80 con 0x7f098006ac20 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f097fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0980071440 con 0x7f0980151280 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f098006ac20 0x7f098006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f098006eff0 0x7f09800c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f098006eff0 0x7f09800c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33570/0 (socket says 172.21.15.67:33570) 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097e000640 1 -- 172.21.15.67:0/3648273711 learned_addr learned my addr 172.21.15.67:0/3648273711 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:46.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097f400640 1 --2- 172.21.15.67:0/3648273711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0980151280 0x7f09801516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7ff3ca000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff3c40c35d0 0x7ff3c4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7ff3ca000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40c0590 con 0x7ff3c406dd80 2024-02-26T10:38:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7ff3ca000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c4154e10 con 0x7ff3c40c35d0 2024-02-26T10:38:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.983+0000 7f1f8be00640 1 -- start start 2024-02-26T10:38:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f1f8be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f8400d070 0x7f1f840ea080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:46.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f1f8be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8400cc60 con 0x7f1f840dbd80 2024-02-26T10:38:46.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f1f8be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f84049750 con 0x7f1f840e1ce0 2024-02-26T10:38:46.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f1f8be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f840dc160 con 0x7f1f8400d070 2024-02-26T10:38:46.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7ff3ca000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40c08d0 con 0x7ff3c4152390 2024-02-26T10:38:46.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7ff3c2a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff3c406dd80 0x7ff3c4158850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:48218/0 (socket says 172.21.15.67:48218) 2024-02-26T10:38:46.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7ff3c2a00640 1 -- 172.21.15.67:0/2130731270 learned_addr learned my addr 172.21.15.67:0/2130731270 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097e000640 1 -- 172.21.15.67:0/3648273711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0980151280 msgr2=0x7f09801516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.984+0000 7f097e000640 1 --2- 172.21.15.67:0/3648273711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0980151280 0x7f09801516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f097e000640 1 -- 172.21.15.67:0/3648273711 >> v2:172.21.15.82:3300/0 conn(0x7f098006ac20 msgr2=0x7f098006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f097e000640 1 --2- 172.21.15.67:0/3648273711 >> v2:172.21.15.82:3300/0 conn(0x7f098006ac20 0x7f098006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7f097e000640 1 -- 172.21.15.67:0/3648273711 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0980071720 con 0x7f098006eff0 2024-02-26T10:38:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.985+0000 7ff3c3e00640 1 --2- 172.21.15.67:0/2130731270 >> v2:172.21.15.67:3300/0 conn(0x7ff3c40c35d0 0x7ff3c4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7f1f8b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f8400d070 0x7f1f840ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7f1f8b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f8400d070 0x7f1f840ea080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38802/0 (socket says 172.21.15.67:38802) 2024-02-26T10:38:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7f1f8b400640 1 -- 172.21.15.67:0/133419358 learned_addr learned my addr 172.21.15.67:0/133419358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.987+0000 7f1f8b400640 1 -- 172.21.15.67:0/133419358 >> v1:172.21.15.82:6789/0 conn(0x7f1f840e1ce0 legacy=0x7f1f840dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.987+0000 7f1f8b400640 1 -- 172.21.15.67:0/133419358 >> v1:172.21.15.67:6789/0 conn(0x7f1f840dbd80 legacy=0x7f1f84040a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.987+0000 7f1f8b400640 1 -- 172.21.15.67:0/133419358 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84049a30 con 0x7f1f8400d070 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c2000640 1 -- 172.21.15.67:0/2130731270 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834914722 0 0) 0x7ff3c40c0590 con 0x7ff3c406dd80 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c2000640 1 -- 172.21.15.67:0/2130731270 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a8003520 con 0x7ff3c406dd80 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c3e00640 1 -- 172.21.15.67:0/2130731270 >> v2:172.21.15.82:3300/0 conn(0x7ff3c4152390 msgr2=0x7ff3c40c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c3400640 1 --2- 172.21.15.67:0/2130731270 >> v2:172.21.15.82:3300/0 conn(0x7ff3c4152390 0x7ff3c40c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c3e00640 1 --2- 172.21.15.67:0/2130731270 >> v2:172.21.15.82:3300/0 conn(0x7ff3c4152390 0x7ff3c40c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c3e00640 1 -- 172.21.15.67:0/2130731270 >> v1:172.21.15.82:6789/0 conn(0x7ff3c406dd80 legacy=0x7ff3c4158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.986+0000 7ff3c3e00640 1 -- 172.21.15.67:0/2130731270 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40c02b0 con 0x7ff3c40c35d0 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.988+0000 7ff3c3e00640 1 --2- 172.21.15.67:0/2130731270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4158f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3c406bcc0 tx=0x7ff3b002bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeeb0bfd5752f22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:46.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.988+0000 7f097e000640 1 --2- 172.21.15.67:0/3648273711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f09800c2650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09800c0f80 tx=0x7f096c0349c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3291af4faad312a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f1f8b400640 1 --2- 172.21.15.67:0/133419358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f8400d070 0x7f1f840ea080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c000b60 tx=0x7f1f8c0a6950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc5db8f0e8f6c6b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f097d600640 1 -- 172.21.15.67:0/3648273711 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c03d070 con 0x7f098006eff0 2024-02-26T10:38:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f097d600640 1 -- 172.21.15.67:0/3648273711 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c039880 con 0x7f098006eff0 2024-02-26T10:38:46.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f097d600640 1 -- 172.21.15.67:0/3648273711 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c039c50 con 0x7f098006eff0 2024-02-26T10:38:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7ff3c2000640 1 -- 172.21.15.67:0/2130731270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b002d670 con 0x7ff3c40c35d0 2024-02-26T10:38:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7ff3c2000640 1 -- 172.21.15.67:0/2130731270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b002d810 con 0x7ff3c40c35d0 2024-02-26T10:38:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f1f89600640 1 -- 172.21.15.67:0/133419358 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0af070 con 0x7f1f8400d070 2024-02-26T10:38:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7ff3c2000640 1 -- 172.21.15.67:0/2130731270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b002dbe0 con 0x7ff3c40c35d0 2024-02-26T10:38:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f1f89600640 1 -- 172.21.15.67:0/133419358 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c0ab6d0 con 0x7f1f8400d070 2024-02-26T10:38:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.989+0000 7f1f89600640 1 -- 172.21.15.67:0/133419358 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0abb00 con 0x7f1f8400d070 2024-02-26T10:38:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.990+0000 7f097fe00640 1 -- 172.21.15.67:0/3648273711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 msgr2=0x7f09800c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.990+0000 7f097fe00640 1 --2- 172.21.15.67:0/3648273711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f09800c2650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f09800c0f80 tx=0x7f096c0349c0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.990+0000 7f08ae000640 1 Processor -- start 2024-02-26T10:38:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f1f8be00640 1 -- 172.21.15.67:0/133419358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f8400d070 msgr2=0x7f1f840ea080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f1f8be00640 1 --2- 172.21.15.67:0/133419358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f8400d070 0x7f1f840ea080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c000b60 tx=0x7f1f8c0a6950 comp rx=0 tx=0).stop 2024-02-26T10:38:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f1f8be00640 1 -- 172.21.15.67:0/133419358 shutdown_connections 2024-02-26T10:38:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f08ae000640 1 -- start start 2024-02-26T10:38:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f1f8be00640 1 --2- 172.21.15.67:0/133419358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f8400d070 0x7f1f840ea080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f1f8be00640 1 -- 172.21.15.67:0/133419358 >> 172.21.15.67:0/133419358 conn(0x7f1f8402f590 msgr2=0x7f1f8402f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f097fe00640 1 -- 172.21.15.67:0/3648273711 shutdown_connections 2024-02-26T10:38:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f097fe00640 1 --2- 172.21.15.67:0/3648273711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0980151280 0x7f09801516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f097fe00640 1 --2- 172.21.15.67:0/3648273711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f09800c2650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f097fe00640 1 --2- 172.21.15.67:0/3648273711 >> v2:172.21.15.82:3300/0 conn(0x7f098006ac20 0x7f098006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7f097fe00640 1 -- 172.21.15.67:0/3648273711 >> 172.21.15.67:0/3648273711 conn(0x7f09800c63f0 msgr2=0x7f09800c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7ff3ca000640 1 -- 172.21.15.67:0/2130731270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c40c35d0 msgr2=0x7ff3c4158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.991+0000 7ff3ca000640 1 --2- 172.21.15.67:0/2130731270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4158f60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3c406bcc0 tx=0x7ff3b002bac0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f1f8be00640 1 -- 172.21.15.67:0/133419358 shutdown_connections 2024-02-26T10:38:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f1f8be00640 1 -- 172.21.15.67:0/133419358 wait complete. 2024-02-26T10:38:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7ff3ca000640 1 -- 172.21.15.67:0/2130731270 shutdown_connections 2024-02-26T10:38:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7ff3ca000640 1 --2- 172.21.15.67:0/2130731270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4158f60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7ff3ca000640 1 --2- 172.21.15.67:0/2130731270 >> v2:172.21.15.82:3300/0 conn(0x7ff3c4152390 0x7ff3c40c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7ff3ca000640 1 -- 172.21.15.67:0/2130731270 >> 172.21.15.67:0/2130731270 conn(0x7ff3c406d680 msgr2=0x7ff3c40a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f097fe00640 1 -- 172.21.15.67:0/3648273711 shutdown_connections 2024-02-26T10:38:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f097fe00640 1 -- 172.21.15.67:0/3648273711 wait complete. 2024-02-26T10:38:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f08ae000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f08a8156970 0x7f08a8159db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f08ae000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08a806e8c0 0x7f08a80ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f08ae000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a80cf040 0x7f08a815a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f08ae000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a80b1ce0 con 0x7f08a8156970 2024-02-26T10:38:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f08ae000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a815a2f0 con 0x7f08a806e8c0 2024-02-26T10:38:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7f08ae000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a8156d50 con 0x7f08a80cf040 2024-02-26T10:38:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.993+0000 7f08a7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08a806e8c0 0x7f08a80ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.993+0000 7f08a7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08a806e8c0 0x7f08a80ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38804/0 (socket says 172.21.15.67:38804) 2024-02-26T10:38:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.993+0000 7f08a7e00640 1 -- 172.21.15.67:0/3919660694 learned_addr learned my addr 172.21.15.67:0/3919660694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.993+0000 7f08acc00640 1 --2- 172.21.15.67:0/3919660694 >> v2:172.21.15.67:3300/0 conn(0x7f08a8156970 0x7f08a8159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f1f8be00640 1 Processor -- start 2024-02-26T10:38:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f1f8be00640 1 -- start start 2024-02-26T10:38:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f097fe00640 1 Processor -- start 2024-02-26T10:38:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7ff3ca000640 1 -- 172.21.15.67:0/2130731270 shutdown_connections 2024-02-26T10:38:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.992+0000 7ff3ca000640 1 -- 172.21.15.67:0/2130731270 wait complete. 2024-02-26T10:38:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.995+0000 7ff3ca000640 1 Processor -- start 2024-02-26T10:38:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.995+0000 7f097fe00640 1 -- start start 2024-02-26T10:38:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.995+0000 7ff3ca000640 1 -- start start 2024-02-26T10:38:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f08acc00640 1 -- 172.21.15.67:0/3919660694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a80cf040 msgr2=0x7f08a815a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f08acc00640 1 --2- 172.21.15.67:0/3919660694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a80cf040 0x7f08a815a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f08acc00640 1 -- 172.21.15.67:0/3919660694 >> v2:172.21.15.82:3300/0 conn(0x7f08a806e8c0 msgr2=0x7f08a80ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f08acc00640 1 --2- 172.21.15.67:0/3919660694 >> v2:172.21.15.82:3300/0 conn(0x7f08a806e8c0 0x7f08a80ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.994+0000 7f08acc00640 1 -- 172.21.15.67:0/3919660694 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a80b3aa0 con 0x7f08a8156970 2024-02-26T10:38:47.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.996+0000 7f08acc00640 1 --2- 172.21.15.67:0/3919660694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a8156970 0x7f08a8159db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f08a806ed50 tx=0x7f08a002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6919fafeacf8144 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.996+0000 7f097fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f098006ac20 0x7f0980150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.996+0000 7f097fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f098013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.996+0000 7f097fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980151280 0x7f0980140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f097fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980071fc0 con 0x7f098006eff0 2024-02-26T10:38:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f097fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980071ce0 con 0x7f0980151280 2024-02-26T10:38:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f097fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0980151c30 con 0x7f098006ac20 2024-02-26T10:38:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f08a7400640 1 -- 172.21.15.67:0/3919660694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0038070 con 0x7f08a8156970 2024-02-26T10:38:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f08a7400640 1 -- 172.21.15.67:0/3919660694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a002f050 con 0x7f08a8156970 2024-02-26T10:38:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f08a7400640 1 -- 172.21.15.67:0/3919660694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00319b0 con 0x7f08a8156970 2024-02-26T10:38:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f097f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980151280 0x7f0980140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f8400d070 0x7f1f840de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.997+0000 7ff3ca000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3c406dd80 0x7ff3c40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f097e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f098013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f08ae000640 1 -- 172.21.15.67:0/3919660694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a8156970 msgr2=0x7f08a8159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f08ae000640 1 --2- 172.21.15.67:0/3919660694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a8156970 0x7f08a8159db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f08a806ed50 tx=0x7f08a002b7f0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7ff3ca000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4072770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f840dbd80 0x7f1f8402e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7ff3c3400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3c406dd80 0x7ff3c40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f097e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f098013fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33584/0 (socket says 172.21.15.67:33584) 2024-02-26T10:38:47.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f1f8aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f8400d070 0x7f1f840de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f097ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f098006ac20 0x7f0980150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f097e000640 1 -- 172.21.15.67:0/3884018044 learned_addr learned my addr 172.21.15.67:0/3884018044 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7f1f8aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f8400d070 0x7f1f840de0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33600/0 (socket says 172.21.15.67:33600) 2024-02-26T10:38:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7ff3c3400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3c406dd80 0x7ff3c40a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39436/0 (socket says 172.21.15.67:39436) 2024-02-26T10:38:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.998+0000 7ff3c3400640 1 -- 172.21.15.67:0/3373883467 learned_addr learned my addr 172.21.15.67:0/3373883467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8aa00640 1 -- 172.21.15.67:0/2503483394 learned_addr learned my addr 172.21.15.67:0/2503483394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 -- 172.21.15.67:0/3919660694 shutdown_connections 2024-02-26T10:38:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 --2- 172.21.15.67:0/3919660694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a80cf040 0x7f08a815a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 --2- 172.21.15.67:0/3919660694 >> v2:172.21.15.82:3300/0 conn(0x7f08a806e8c0 0x7f08a80ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 --2- 172.21.15.67:0/3919660694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a8156970 0x7f08a8159db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 -- 172.21.15.67:0/3919660694 >> 172.21.15.67:0/3919660694 conn(0x7f08a80b1610 msgr2=0x7f08a806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8be00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f840e1ce0 0x7f1f840d87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7ff3ca000640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c4152390 0x7ff3c41ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840de8e0 con 0x7f1f8400d070 2024-02-26T10:38:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 -- 172.21.15.67:0/3919660694 shutdown_connections 2024-02-26T10:38:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f08ae000640 1 -- 172.21.15.67:0/3919660694 wait complete. 2024-02-26T10:38:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8a000640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f840dbd80 0x7f1f8402e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84049ff0 con 0x7f1f840e1ce0 2024-02-26T10:38:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84041120 con 0x7f1f840dbd80 2024-02-26T10:38:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:46.999+0000 7f1f8b400640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f840e1ce0 0x7f1f840d87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7ff3c3e00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c4152390 0x7ff3c41ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7ff3c2a00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c1b30 con 0x7ff3c4152390 2024-02-26T10:38:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7f1f8a000640 1 -- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f840e1ce0 msgr2=0x7f1f840d87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7f1f8a000640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f840e1ce0 0x7f1f840d87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7f1f8a000640 1 -- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f8400d070 msgr2=0x7f1f840de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f097e000640 1 -- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f098006ac20 msgr2=0x7f0980150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c4152770 con 0x7ff3c40c35d0 2024-02-26T10:38:47.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.000+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c1cb0 con 0x7ff3c406dd80 2024-02-26T10:38:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f097e000640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f098006ac20 0x7f0980150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f097e000640 1 -- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980151280 msgr2=0x7f0980140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f097e000640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980151280 0x7f0980140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f097e000640 1 -- 172.21.15.67:0/3884018044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0980153630 con 0x7f098006eff0 2024-02-26T10:38:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7fe406e00640 1 Processor -- start 2024-02-26T10:38:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f1f8a000640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f8400d070 0x7f1f840de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f1f8a000640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840dcf20 con 0x7f1f840dbd80 2024-02-26T10:38:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7f1f8aa00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f8400d070 0x7f1f840de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.001+0000 7fe406e00640 1 -- start start 2024-02-26T10:38:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7f08ae000640 1 Processor -- start 2024-02-26T10:38:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7f097e000640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f098013fb50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f096c0144d0 tx=0x7f096c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7ff3c2a00640 1 -- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3c406dd80 msgr2=0x7ff3c40a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7ff3c2a00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3c406dd80 0x7ff3c40a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7ff3c2a00640 1 -- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c4152390 msgr2=0x7ff3c41ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7ff3c2a00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3c4152390 0x7ff3c41ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7ff3c2a00640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40c51f0 con 0x7ff3c40c35d0 2024-02-26T10:38:47.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7f08ae000640 1 -- start start 2024-02-26T10:38:47.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c03d040 con 0x7f098006eff0 2024-02-26T10:38:47.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7fa90c400640 1 Processor -- start 2024-02-26T10:38:47.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.002+0000 7fa90c400640 1 -- start start 2024-02-26T10:38:47.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7ff3c2a00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4072770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c3c0 tx=0x7ff3b4001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f7262800640 1 Processor -- start 2024-02-26T10:38:47.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f1f8a000640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f840dbd80 0x7f1f8402e360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c410 tx=0x7f1f7800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78017070 con 0x7f1f840dbd80 2024-02-26T10:38:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09801e76d0 con 0x7f098006eff0 2024-02-26T10:38:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0980154660 con 0x7f098006eff0 2024-02-26T10:38:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c039680 con 0x7f098006eff0 2024-02-26T10:38:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08ae000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a806e8c0 0x7f08a81f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f7262800640 1 -- start start 2024-02-26T10:38:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08ae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a80cf040 0x7f08a80cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fe406e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe40006ead0 0x7fe4001397c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fe406e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe400070f10 0x7fe400154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fe406e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4001512b0 con 0x7fe4000a8bb0 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fe406e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe400151590 con 0x7fe40006ead0 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fe406e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe400151870 con 0x7fe400070f10 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08ae000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08a8156970 0x7f08a80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08ae000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80b4060 con 0x7f08a80cf040 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08ae000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80b1f80 con 0x7f08a8156970 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08ae000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a815ad60 con 0x7f08a806e8c0 2024-02-26T10:38:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fa90c400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa90415a780 0x7fa904159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fa90c400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90406e720 con 0x7fa9040725b0 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fa90c400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90415abe0 con 0x7fa904156930 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7fa90c400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa904156d10 con 0x7fa90415a780 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4017070 con 0x7ff3c40c35d0 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4005ce0 con 0x7ff3c40c35d0 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4007310 con 0x7ff3c40c35d0 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c4071fb0 con 0x7ff3c40c35d0 2024-02-26T10:38:47.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c40c5770 con 0x7ff3c40c35d0 2024-02-26T10:38:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.003+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c03d040 con 0x7f098006eff0 2024-02-26T10:38:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08acc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a806e8c0 0x7f08a81f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08acc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a806e8c0 0x7f08a81f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39454/0 (socket says 172.21.15.67:39454) 2024-02-26T10:38:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.004+0000 7f08acc00640 1 -- 172.21.15.67:0/3015876692 learned_addr learned my addr 172.21.15.67:0/3015876692 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7fe3ffe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe4000a8bb0 0x7fe4000729e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:48248/0 (socket says 172.21.15.67:48248) 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7fe3ffe00640 1 -- 172.21.15.67:0/281073827 learned_addr learned my addr 172.21.15.67:0/281073827 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7fa909a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa904156930 0x7fa90415d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:48250/0 (socket says 172.21.15.67:48250) 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7fa909a00640 1 -- 172.21.15.67:0/656060636 learned_addr learned my addr 172.21.15.67:0/656060636 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f7262800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f7262800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f725c070f10 0x7f725c154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f7262800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c139ba0 con 0x7f725c1397c0 2024-02-26T10:38:47.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f7262800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c1512b0 con 0x7f725c070f10 2024-02-26T10:38:47.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f7262800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c151590 con 0x7f725c06ead0 2024-02-26T10:38:47.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7fe404e00640 1 --2- 172.21.15.67:0/281073827 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe400070f10 0x7fe400154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f08acc00640 1 -- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08a8156970 msgr2=0x7f08a80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f08acc00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08a8156970 0x7f08a80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f08acc00640 1 -- 172.21.15.67:0/3015876692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a80cf040 msgr2=0x7f08a80cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f08acc00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08a80cf040 0x7f08a80cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f08acc00640 1 -- 172.21.15.67:0/3015876692 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a81e6790 con 0x7f08a806e8c0 2024-02-26T10:38:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0958001030 con 0x7f098006eff0 2024-02-26T10:38:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f78005230 con 0x7f1f840dbd80 2024-02-26T10:38:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780045e0 con 0x7f1f840dbd80 2024-02-26T10:38:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f840de5f0 con 0x7f1f840dbd80 2024-02-26T10:38:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff39c001030 con 0x7ff3c40c35d0 2024-02-26T10:38:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39470/0 (socket says 172.21.15.67:39470) 2024-02-26T10:38:47.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 -- 172.21.15.67:0/3583852115 learned_addr learned my addr 172.21.15.67:0/3583852115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f08acc00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a806e8c0 0x7f08a81f25b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08a0000c00 tx=0x7f08a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7fa909000640 1 --2- 172.21.15.67:0/656060636 >> v2:172.21.15.67:3300/0 conn(0x7fa90415a780 0x7fa904159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725be00640 1 --2- 172.21.15.67:0/3583852115 >> v2:172.21.15.67:3300/0 conn(0x7f725c070f10 0x7f725c154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.005+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8417ddc0 con 0x7f1f840dbd80 2024-02-26T10:38:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7fe404e00640 1 -- 172.21.15.67:0/281073827 >> v2:172.21.15.67:3300/0 conn(0x7fe40006ead0 msgr2=0x7fe4001397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7fe404e00640 1 --2- 172.21.15.67:0/281073827 >> v2:172.21.15.67:3300/0 conn(0x7fe40006ead0 0x7fe4001397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f64001030 con 0x7f1f840dbd80 2024-02-26T10:38:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7fe404e00640 1 -- 172.21.15.67:0/281073827 >> v1:172.21.15.82:6789/0 conn(0x7fe4000a8bb0 legacy=0x7fe4000729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.006+0000 7fe404e00640 1 -- 172.21.15.67:0/281073827 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400151b50 con 0x7fe400070f10 2024-02-26T10:38:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 -- 172.21.15.67:0/3583852115 >> v2:172.21.15.67:3300/0 conn(0x7f725c070f10 msgr2=0x7f725c154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 --2- 172.21.15.67:0/3583852115 >> v2:172.21.15.67:3300/0 conn(0x7f725c070f10 0x7f725c154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 -- 172.21.15.67:0/3583852115 >> v1:172.21.15.67:6789/0 conn(0x7f725c1397c0 legacy=0x7f725c0729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f725aa00640 1 -- 172.21.15.67:0/3583852115 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c151870 con 0x7f725c06ead0 2024-02-26T10:38:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7fa903e00640 1 -- 172.21.15.67:0/656060636 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204561790 0 0) 0x7fa90415abe0 con 0x7fa904156930 2024-02-26T10:38:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7fa903e00640 1 -- 172.21.15.67:0/656060636 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8ec003520 con 0x7fa904156930 2024-02-26T10:38:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.007+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0042070 con 0x7f08a806e8c0 2024-02-26T10:38:47.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a003d870 con 0x7f08a806e8c0 2024-02-26T10:38:47.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0038040 con 0x7f08a806e8c0 2024-02-26T10:38:47.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7fa909000640 1 -- 172.21.15.67:0/656060636 >> v1:172.21.15.82:6789/0 conn(0x7fa904156930 legacy=0x7fa90415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a80cb7e0 con 0x7f08a806e8c0 2024-02-26T10:38:47.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a81e2990 con 0x7f08a806e8c0 2024-02-26T10:38:47.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7fa909000640 1 -- 172.21.15.67:0/656060636 >> v1:172.21.15.67:6789/0 conn(0x7fa9040725b0 legacy=0x7fa9040729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7fa909000640 1 -- 172.21.15.67:0/656060636 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040c0ce0 con 0x7fa90415a780 2024-02-26T10:38:47.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.009+0000 7fe404e00640 1 --2- 172.21.15.67:0/281073827 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe400070f10 0x7fe400154780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009d90 tx=0x7fe3f802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a005c6d1b66b5e64 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.008+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3b401a070 con 0x7ff3c40c35d0 2024-02-26T10:38:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.009+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08a81e2d90 con 0x7f08a806e8c0 2024-02-26T10:38:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.009+0000 7fe3ff400640 1 -- 172.21.15.67:0/281073827 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8039070 con 0x7fe400070f10 2024-02-26T10:38:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7fa909000640 1 --2- 172.21.15.67:0/656060636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa90415a780 0x7fa904159ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa90415adb0 tx=0x7fa8f802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b500883e97e2246b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7ff3bb400640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff3a406b5a0 0x7ff3a406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3b4089bd0 con 0x7ff3c40c35d0 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7ff3bb400640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7ff3a40737c0 0x7ff3a4075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3a4076290 con 0x7ff3a40737c0 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3b4089fc0 con 0x7ff3c40c35d0 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7fe3ff400640 1 -- 172.21.15.67:0/281073827 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f802cc50 con 0x7fe400070f10 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.010+0000 7fe3ff400640 1 -- 172.21.15.67:0/281073827 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8034440 con 0x7fe400070f10 2024-02-26T10:38:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7ff3c3e00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7ff3a40737c0 0x7ff3a4075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7fa903e00640 1 -- 172.21.15.67:0/656060636 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8034070 con 0x7fa90415a780 2024-02-26T10:38:47.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7fe406e00640 1 -- 172.21.15.67:0/281073827 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe400070f10 msgr2=0x7fe400154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7fe406e00640 1 --2- 172.21.15.67:0/281073827 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe400070f10 0x7fe400154780 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009d90 tx=0x7fe3f802bea0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7ff3c3e00640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7ff3a40737c0 0x7ff3a4075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7fabc3600640 1 Processor -- start 2024-02-26T10:38:47.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fabc3600640 1 -- start start 2024-02-26T10:38:47.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f096c040070 con 0x7f098006eff0 2024-02-26T10:38:47.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f7801a070 con 0x7f1f840dbd80 2024-02-26T10:38:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7fa903e00640 1 -- 172.21.15.67:0/656060636 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f8030800 con 0x7fa90415a780 2024-02-26T10:38:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.011+0000 7fa903e00640 1 -- 172.21.15.67:0/656060636 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8030bd0 con 0x7fa90415a780 2024-02-26T10:38:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fe406e00640 1 -- 172.21.15.67:0/281073827 shutdown_connections 2024-02-26T10:38:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fe406e00640 1 --2- 172.21.15.67:0/281073827 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe400070f10 0x7fe400154780 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fe406e00640 1 --2- 172.21.15.67:0/281073827 >> v2:172.21.15.67:3300/0 conn(0x7fe40006ead0 0x7fe4001397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fe406e00640 1 -- 172.21.15.67:0/281073827 >> 172.21.15.67:0/281073827 conn(0x7fe40006d680 msgr2=0x7fe4000a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7ff3c3400640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff3a406b5a0 0x7ff3a406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fa90c400640 1 -- 172.21.15.67:0/656060636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa90415a780 msgr2=0x7fa904159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7fa90c400640 1 --2- 172.21.15.67:0/656060636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa90415a780 0x7fa904159ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa90415adb0 tx=0x7fa8f802b940 comp rx=0 tx=0).stop 2024-02-26T10:38:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== osd.1 v2:172.21.15.67:6817/1684785661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff3a4076290 con 0x7ff3a40737c0 2024-02-26T10:38:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.012+0000 7f725aa00640 1 --2- 172.21.15.67:0/3583852115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c154780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f724800a910 tx=0x7f7248030100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d6a03c780d1758 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7f1f83400640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f6c06b4d0 0x7f1f6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fe406e00640 1 -- 172.21.15.67:0/281073827 shutdown_connections 2024-02-26T10:38:47.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fa90c400640 1 -- 172.21.15.67:0/656060636 shutdown_connections 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fa90c400640 1 --2- 172.21.15.67:0/656060636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa90415a780 0x7fa904159ba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fa90c400640 1 -- 172.21.15.67:0/656060636 >> 172.21.15.67:0/656060636 conn(0x7fa90406d680 msgr2=0x7fa9040a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabc3600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabbc0bdef0 0x7fabbc158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabc3600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc154f50 con 0x7fabbc155400 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabc3600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc0c7440 con 0x7fabbc157d30 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabc3600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc0c7720 con 0x7fabbc0bdef0 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabbbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabbc0bdef0 0x7fabbc158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabbbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabbc0bdef0 0x7fabbc158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33650/0 (socket says 172.21.15.67:33650) 2024-02-26T10:38:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fabbbe00640 1 -- 172.21.15.67:0/3821126501 learned_addr learned my addr 172.21.15.67:0/3821126501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7f725a000640 1 -- 172.21.15.67:0/3583852115 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248036070 con 0x7f725c06ead0 2024-02-26T10:38:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f78089940 con 0x7f1f840dbd80 2024-02-26T10:38:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f1f83400640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f1f6c073750 0x7f1f6c075b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f7800ae50 con 0x7f1f6c073750 2024-02-26T10:38:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f78089d30 con 0x7f1f840dbd80 2024-02-26T10:38:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f1f8aa00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f6c06b4d0 0x7f1f6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7ff3c3400640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff3a406b5a0 0x7ff3a406da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00f5f0 tx=0x7ff3bc007870 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f725a000640 1 -- 172.21.15.67:0/3583852115 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724803a400 con 0x7f725c06ead0 2024-02-26T10:38:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f725a000640 1 -- 172.21.15.67:0/3583852115 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724803a7d0 con 0x7f725c06ead0 2024-02-26T10:38:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7f1f8b400640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f1f6c073750 0x7f1f6c075b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7fabbb400640 1 -- 172.21.15.67:0/3821126501 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453819667 0 0) 0x7fabbc0c7440 con 0x7fabbc157d30 2024-02-26T10:38:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7fabbb400640 1 -- 172.21.15.67:0/3821126501 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba4003520 con 0x7fabbc157d30 2024-02-26T10:38:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7fa90c400640 1 -- 172.21.15.67:0/656060636 shutdown_connections 2024-02-26T10:38:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.014+0000 7fa90c400640 1 -- 172.21.15.67:0/656060636 wait complete. 2024-02-26T10:38:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.015+0000 7f1f8aa00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f6c06b4d0 0x7f1f6c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c001c30 tx=0x7f1f7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.015+0000 7f1f8b400640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f1f6c073750 0x7f1f6c075b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.015+0000 7f0976a00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f096006b6d0 0x7f096006db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.013+0000 7fe406e00640 1 -- 172.21.15.67:0/281073827 wait complete. 2024-02-26T10:38:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7fe406e00640 1 Processor -- start 2024-02-26T10:38:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f096c0b5d50 con 0x7f098006eff0 2024-02-26T10:38:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7f0976a00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f09600738f0 0x7f0960075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 --> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f096c041ec0 con 0x7f09600738f0 2024-02-26T10:38:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f096c0b6140 con 0x7f098006eff0 2024-02-26T10:38:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7fe406e00640 1 -- start start 2024-02-26T10:38:47.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7f7262800640 1 -- 172.21.15.67:0/3583852115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 msgr2=0x7f725c154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.016+0000 7f7262800640 1 --2- 172.21.15.67:0/3583852115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c154780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f724800a910 tx=0x7f7248030100 comp rx=0 tx=0).stop 2024-02-26T10:38:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.015+0000 7fabbbe00640 1 -- 172.21.15.67:0/3821126501 >> v1:172.21.15.82:6789/0 conn(0x7fabbc157d30 legacy=0x7fabbc0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.015+0000 7fabbbe00640 1 -- 172.21.15.67:0/3821126501 >> v1:172.21.15.67:6789/0 conn(0x7fabbc155400 legacy=0x7fabbc155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.015+0000 7fabbbe00640 1 -- 172.21.15.67:0/3821126501 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0ad2a0 con 0x7fabbc0bdef0 2024-02-26T10:38:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f7262800640 1 -- 172.21.15.67:0/3583852115 shutdown_connections 2024-02-26T10:38:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f7262800640 1 --2- 172.21.15.67:0/3583852115 >> v2:172.21.15.67:3300/0 conn(0x7f725c070f10 0x7f725c154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f7262800640 1 --2- 172.21.15.67:0/3583852115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c154780 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f7262800640 1 -- 172.21.15.67:0/3583852115 >> 172.21.15.67:0/3583852115 conn(0x7f725c06d680 msgr2=0x7f725c0a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f097ea00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f096006b6d0 0x7f096006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f097f400640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f09600738f0 0x7f0960075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7fa90c400640 1 Processor -- start 2024-02-26T10:38:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f7262800640 1 -- 172.21.15.67:0/3583852115 shutdown_connections 2024-02-26T10:38:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7fabbbe00640 1 --2- 172.21.15.67:0/3821126501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc0bdef0 0x7fabbc158e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faba8009870 tx=0x7faba802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b57605e1dbf750 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7fa90c400640 1 -- start start 2024-02-26T10:38:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7fabbb400640 1 -- 172.21.15.67:0/3821126501 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8034070 con 0x7fabbc0bdef0 2024-02-26T10:38:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7fabbb400640 1 -- 172.21.15.67:0/3821126501 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8030850 con 0x7fabbc0bdef0 2024-02-26T10:38:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7fabbb400640 1 -- 172.21.15.67:0/3821126501 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8030c20 con 0x7fabbc0bdef0 2024-02-26T10:38:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7f097f400640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f09600738f0 0x7f0960075d10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f7262800640 1 -- 172.21.15.67:0/3583852115 wait complete. 2024-02-26T10:38:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7fa90c400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9040725b0 0x7fa9041f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe406e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40006ead0 0x7fe400129ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe406e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe400070f10 0x7fe400136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa90c400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa904156930 0x7fa904151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== osd.0 v2:172.21.15.67:6800/1950536715 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1f7800ae50 con 0x7f1f6c073750 2024-02-26T10:38:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa909a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9040725b0 0x7fa9041f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe406e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4000a8bb0 0x7fe400137520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.017+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08a0045070 con 0x7f08a806e8c0 2024-02-26T10:38:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7f08a4c00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f088406b5a0 0x7f088406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.018+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08a00ac4b0 con 0x7f08a806e8c0 2024-02-26T10:38:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7f08a4c00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7f08840737c0 0x7f0884075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 --> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08a0035e90 con 0x7f08840737c0 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08a00ac8a0 con 0x7f08a806e8c0 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa909a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9040725b0 0x7fa9041f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33664/0 (socket says 172.21.15.67:33664) 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa90c400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa90415a780 0x7fa904151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa909a00640 1 -- 172.21.15.67:0/3721691751 learned_addr learned my addr 172.21.15.67:0/3721691751 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9041593a0 con 0x7fa9040725b0 2024-02-26T10:38:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406ec70 con 0x7fa90415a780 2024-02-26T10:38:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7f097ea00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f096006b6d0 0x7f096006db90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0970006fd0 tx=0x7f0970008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904159680 con 0x7fa904156930 2024-02-26T10:38:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe406e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400151070 con 0x7fe400070f10 2024-02-26T10:38:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe406e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400150ef0 con 0x7fe4000a8bb0 2024-02-26T10:38:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe406e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000730f0 con 0x7fe40006ead0 2024-02-26T10:38:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe3ffe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40006ead0 0x7fe400129ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.019+0000 7fe3f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe400070f10 0x7fe400136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7f08a7e00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f088406b5a0 0x7f088406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fe3ffe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40006ead0 0x7fe400129ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39478/0 (socket says 172.21.15.67:39478) 2024-02-26T10:38:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fe3ffe00640 1 -- 172.21.15.67:0/3984270895 learned_addr learned my addr 172.21.15.67:0/3984270895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa90a400640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa90415a780 0x7fa904151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fe404e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4000a8bb0 0x7fe400137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa909000640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa904156930 0x7fa904151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7f7262800640 1 Processor -- start 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7f7262800640 1 -- start start 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa909a00640 1 -- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa904156930 msgr2=0x7fa904151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa909a00640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa904156930 0x7fa904151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa909a00640 1 -- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa90415a780 msgr2=0x7fa904151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa909a00640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa90415a780 0x7fa904151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.020+0000 7fa909a00640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040a9640 con 0x7fa9040725b0 2024-02-26T10:38:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fabc3600640 1 -- 172.21.15.67:0/3821126501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc0bdef0 msgr2=0x7fabbc158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fabc3600640 1 --2- 172.21.15.67:0/3821126501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc0bdef0 0x7fabbc158e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faba8009870 tx=0x7faba802b990 comp rx=0 tx=0).stop 2024-02-26T10:38:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fe3ffe00640 1 -- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4000a8bb0 msgr2=0x7fe400137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fe3ffe00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4000a8bb0 0x7fe400137520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fe3ffe00640 1 -- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe400070f10 msgr2=0x7fe400136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fe3ffe00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe400070f10 0x7fe400136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fe3ffe00640 1 -- 172.21.15.67:0/3984270895 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400153fc0 con 0x7fe40006ead0 2024-02-26T10:38:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7f08ad600640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7f08840737c0 0x7f0884075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== osd.2 v2:172.21.15.67:6811/2562492504 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f096c041ec0 con 0x7f09600738f0 2024-02-26T10:38:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fabc3600640 1 -- 172.21.15.67:0/3821126501 shutdown_connections 2024-02-26T10:38:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fabc3600640 1 --2- 172.21.15.67:0/3821126501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc0bdef0 0x7fabbc158e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fabc3600640 1 -- 172.21.15.67:0/3821126501 >> 172.21.15.67:0/3821126501 conn(0x7fabbc072980 msgr2=0x7fabbc0ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fa909a00640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9040725b0 0x7fa9041f1f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00cbf0 tx=0x7fa8fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fe3ffe00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40006ead0 0x7fe400129ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c4b0 tx=0x7fe3f000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.021+0000 7fabc3600640 1 -- 172.21.15.67:0/3821126501 shutdown_connections 2024-02-26T10:38:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fabc3600640 1 -- 172.21.15.67:0/3821126501 wait complete. 2024-02-26T10:38:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc017070 con 0x7fa9040725b0 2024-02-26T10:38:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00ad50 con 0x7fa9040725b0 2024-02-26T10:38:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc017070 con 0x7fa9040725b0 2024-02-26T10:38:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7f08ad600640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7f08840737c0 0x7f0884075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7f7262800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c132e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f7262800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c070f10 0x7f725c133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f7262800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725c1397c0 0x7f725c1ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9040a9920 con 0x7fa9040725b0 2024-02-26T10:38:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa904153e50 con 0x7fa9040725b0 2024-02-26T10:38:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f7262800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c150ef0 con 0x7f725c070f10 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f7262800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c151e30 con 0x7f725c1397c0 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f7262800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c151070 con 0x7f725c06ead0 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f725b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c132e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7f725b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c132e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39498/0 (socket says 172.21.15.67:39498) 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.024+0000 7f725b400640 1 -- 172.21.15.67:0/1594007822 learned_addr learned my addr 172.21.15.67:0/1594007822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.024+0000 7f725be00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725c1397c0 0x7f725c1ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.024+0000 7f725aa00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c070f10 0x7f725c133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f08a7e00640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f088406b5a0 0x7f088406da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f08980072e0 tx=0x7f08980071f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f725b400640 1 -- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725c1397c0 msgr2=0x7f725c1ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f725b400640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725c1397c0 0x7f725c1ea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f725b400640 1 -- 172.21.15.67:0/1594007822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c070f10 msgr2=0x7f725c133340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f725b400640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f725c070f10 0x7f725c133340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f725b400640 1 -- 172.21.15.67:0/1594007822 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0c31b0 con 0x7f725c06ead0 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0017070 con 0x7fe40006ead0 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe400137a60 con 0x7fe40006ead0 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0005ce0 con 0x7fe40006ead0 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00072d0 con 0x7fe40006ead0 2024-02-26T10:38:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.022+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe400154230 con 0x7fe40006ead0 2024-02-26T10:38:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.023+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe40012ae40 con 0x7fe40006ead0 2024-02-26T10:38:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== osd.4 v2:172.21.15.82:6816/900472103 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f08a0035e90 con 0x7f08840737c0 2024-02-26T10:38:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.024+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa904074a30 con 0x7fa9040725b0 2024-02-26T10:38:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.025+0000 7f725b400640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c132e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f724c00b410 tx=0x7f724c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.026+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c015020 con 0x7f725c06ead0 2024-02-26T10:38:47.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.026+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c011830 con 0x7f725c06ead0 2024-02-26T10:38:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.026+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c011c20 con 0x7f725c06ead0 2024-02-26T10:38:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.026+0000 7f7262800640 1 -- 172.21.15.67:0/1594007822 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c0c1f80 con 0x7f725c06ead0 2024-02-26T10:38:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.026+0000 7f7262800640 1 -- 172.21.15.67:0/1594007822 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0c2540 con 0x7f725c06ead0 2024-02-26T10:38:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fabc3600640 1 Processor -- start 2024-02-26T10:38:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fabc3600640 1 -- start start 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.026+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3f001a070 con 0x7fe40006ead0 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fe3fcc00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe3dc06b4d0 0x7fe3dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3f0089c90 con 0x7fe40006ead0 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fe3fcc00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fe3dc0736d0 0x7fe3dc075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 --> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3f0016ec0 con 0x7fe3dc0736d0 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.027+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3f008a080 con 0x7fe40006ead0 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fe404e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fe3dc0736d0 0x7fe3dc075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fe3f7e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe3dc06b4d0 0x7fe3dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8fc01a070 con 0x7fa9040725b0 2024-02-26T10:38:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fabc3600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc0bdef0 0x7fabbc151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fabc3600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc155400 0x7fabbc1ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fabc3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabbc157d30 0x7fabbc1ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fabc3600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc156330 con 0x7fabbc155400 2024-02-26T10:38:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fabc3600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0c7a00 con 0x7fabbc0bdef0 2024-02-26T10:38:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.028+0000 7fabc3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0c58b0 con 0x7fabbc157d30 2024-02-26T10:38:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fe404e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fe3dc0736d0 0x7fe3dc075af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc155400 0x7fabbc1ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc155400 0x7fabbc1ebf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33694/0 (socket says 172.21.15.67:33694) 2024-02-26T10:38:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 -- 172.21.15.67:0/767962591 learned_addr learned my addr 172.21.15.67:0/767962591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fe3f7e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe3dc06b4d0 0x7fe3dc06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe4000c3800 tx=0x7fe3e8009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabc0c00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc0bdef0 0x7fabbc151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabc1600640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabbc157d30 0x7fabbc1ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f724c018020 con 0x7f725c06ead0 2024-02-26T10:38:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 -- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabbc157d30 msgr2=0x7fabbc1ec4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabbc157d30 0x7fabbc1ec4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 -- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc0bdef0 msgr2=0x7fabbc151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc0bdef0 0x7fabbc151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabbbe00640 1 -- 172.21.15.67:0/767962591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0c2550 con 0x7fabbc155400 2024-02-26T10:38:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.030+0000 7f7253400640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f723c06b5a0 0x7f723c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.030+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f724c08a050 con 0x7f725c06ead0 2024-02-26T10:38:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.030+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== osd.3 v2:172.21.15.67:6808/755501566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe3f0016ec0 con 0x7fe3dc0736d0 2024-02-26T10:38:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.029+0000 7fabc0c00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabbc0bdef0 0x7fabbc151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.030+0000 7fabbbe00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc155400 0x7fabbc1ebf80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.030+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8048070 con 0x7fabbc155400 2024-02-26T10:38:47.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.031+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8030650 con 0x7fabbc155400 2024-02-26T10:38:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.031+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8048070 con 0x7fabbc155400 2024-02-26T10:38:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.031+0000 7fa901600640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8e406b6d0 0x7fa8e406db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.031+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8fc08bdb0 con 0x7fa9040725b0 2024-02-26T10:38:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.031+0000 7f7262800640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f725c14e5d0 0x7f725c14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7f725aa00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f723c06b5a0 0x7f723c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7f7262800640 1 -- 172.21.15.67:0/1594007822 --> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f725c1ead10 con 0x7f725c14e5d0 2024-02-26T10:38:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0c2830 con 0x7fabbc155400 2024-02-26T10:38:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7fa901600640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7fa8e40738d0 0x7fa8e4075cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8fc020e50 con 0x7fa8e40738d0 2024-02-26T10:38:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8fc08f050 con 0x7fa9040725b0 2024-02-26T10:38:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc0c2df0 con 0x7fabbc155400 2024-02-26T10:38:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7f725be00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f725c14e5d0 0x7f725c14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.032+0000 7fa90a400640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7fa8e40738d0 0x7fa8e4075cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.034+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fabbc0c2300 con 0x7fabbc155400 2024-02-26T10:38:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.034+0000 7fa90a400640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7fa8e40738d0 0x7fa8e4075cf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.034+0000 7f725be00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f725c14e5d0 0x7f725c14ed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.034+0000 7fa909000640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8e406b6d0 0x7fa8e406db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.035+0000 7f725aa00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f723c06b5a0 0x7f723c06da60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f724803b000 tx=0x7f724800d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.035+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faba8037070 con 0x7fabbc155400 2024-02-26T10:38:47.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.036+0000 7fa909000640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8e406b6d0 0x7fa8e406db90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8036000 tx=0x7fa8f803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== osd.5 v2:172.21.15.82:6800/431290960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa8fc020e50 con 0x7fa8e40738d0 2024-02-26T10:38:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.036+0000 7fabb8c00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab9c06b700 0x7fab9c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.036+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faba80ab860 con 0x7fabbc155400 2024-02-26T10:38:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7fabb8c00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7fab9c0738a0 0x7fab9c075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 --> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab9c076370 con 0x7fab9c0738a0 2024-02-26T10:38:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba80b02c0 con 0x7fabbc155400 2024-02-26T10:38:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7fabc0c00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab9c06b700 0x7fab9c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== osd.6 v2:172.21.15.82:6808/1261740917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f725c1ead10 con 0x7f725c14e5d0 2024-02-26T10:38:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.037+0000 7fabc1600640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7fab9c0738a0 0x7fab9c075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.038+0000 7fabc0c00640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab9c06b700 0x7fab9c06dbc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fabb0001670 tx=0x7fabb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.041+0000 7fabc1600640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7fab9c0738a0 0x7fab9c075cc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.044+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== osd.7 v2:172.21.15.82:6824/3253207267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab9c076370 con 0x7fab9c0738a0 2024-02-26T10:38:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.052+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 --> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff39c002dd0 con 0x7ff3a40737c0 2024-02-26T10:38:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.053+0000 7ff3bb400640 1 -- 172.21.15.67:0/3373883467 <== osd.1 v2:172.21.15.67:6817/1684785661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff39c002dd0 con 0x7ff3a40737c0 2024-02-26T10:38:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7ff3a40737c0 msgr2=0x7ff3a4075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3ca000640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7ff3a40737c0 0x7ff3a4075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff3a406b5a0 msgr2=0x7ff3a406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3ca000640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff3a406b5a0 0x7ff3a406da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00f5f0 tx=0x7ff3bc007870 comp rx=0 tx=0).stop 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c40c35d0 msgr2=0x7ff3c4072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3ca000640 1 --2- 172.21.15.67:0/3373883467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c40c35d0 0x7ff3c4072770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c3c0 tx=0x7ff3b4001e70 comp rx=0 tx=0).stop 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7ff3c3e00640 1 -- 172.21.15.67:0/3373883467 reap_dead start 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.054+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 --> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08a8074a30 con 0x7f08840737c0 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.055+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 shutdown_connections 2024-02-26T10:38:47.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.055+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 >> 172.21.15.67:0/3373883467 conn(0x7ff3c406d680 msgr2=0x7ff3c4151630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.055+0000 7f08a4c00640 1 -- 172.21.15.67:0/3015876692 <== osd.4 v2:172.21.15.82:6816/900472103 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08a8074a30 con 0x7f08840737c0 2024-02-26T10:38:47.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.056+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 shutdown_connections 2024-02-26T10:38:47.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.056+0000 7ff3ca000640 1 -- 172.21.15.67:0/3373883467 wait complete. 2024-02-26T10:38:47.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.056+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7f08840737c0 msgr2=0x7f0884075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.056+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 --> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0958002dd0 con 0x7f09600738f0 2024-02-26T10:38:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.056+0000 7f08ae000640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7f08840737c0 0x7f0884075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.057+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f088406b5a0 msgr2=0x7f088406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.057+0000 7f08ae000640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f088406b5a0 0x7f088406da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f08980072e0 tx=0x7f08980071f0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.057+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a806e8c0 msgr2=0x7f08a81f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.057+0000 7f08ae000640 1 --2- 172.21.15.67:0/3015876692 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08a806e8c0 0x7f08a81f25b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08a0000c00 tx=0x7f08a00046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.057+0000 7f0976a00640 1 -- 172.21.15.67:0/3884018044 <== osd.2 v2:172.21.15.67:6811/2562492504 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0958002dd0 con 0x7f09600738f0 2024-02-26T10:38:47.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.057+0000 7f08ad600640 1 -- 172.21.15.67:0/3015876692 reap_dead start 2024-02-26T10:38:47.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 shutdown_connections 2024-02-26T10:38:47.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 >> 172.21.15.67:0/3015876692 conn(0x7f08a80b1610 msgr2=0x7f08a80cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 shutdown_connections 2024-02-26T10:38:47.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f08ae000640 1 -- 172.21.15.67:0/3015876692 wait complete. 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f09600738f0 msgr2=0x7f0960075d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f09600738f0 0x7f0960075d10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f096006b6d0 msgr2=0x7f096006db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f096006b6d0 0x7f096006db90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0970006fd0 tx=0x7f0970008040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 msgr2=0x7f098013fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 --2- 172.21.15.67:0/3884018044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f098006eff0 0x7f098013fb50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f096c0144d0 tx=0x7f096c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 --> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f64002dd0 con 0x7f1f6c073750 2024-02-26T10:38:47.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097f400640 1 -- 172.21.15.67:0/3884018044 reap_dead start 2024-02-26T10:38:47.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 shutdown_connections 2024-02-26T10:38:47.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.058+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 >> 172.21.15.67:0/3884018044 conn(0x7f09800c63f0 msgr2=0x7f09800c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.059+0000 7f1f83400640 1 -- 172.21.15.67:0/2503483394 <== osd.0 v2:172.21.15.67:6800/1950536715 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f64002dd0 con 0x7f1f6c073750 2024-02-26T10:38:47.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.060+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f1f6c073750 msgr2=0x7f1f6c075b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.060+0000 7f1f8be00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f1f6c073750 0x7f1f6c075b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.061+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f6c06b4d0 msgr2=0x7f1f6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.060+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 shutdown_connections 2024-02-26T10:38:47.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.060+0000 7f097fe00640 1 -- 172.21.15.67:0/3884018044 wait complete. 2024-02-26T10:38:47.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.061+0000 7f1f8be00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f6c06b4d0 0x7f1f6c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c001c30 tx=0x7f1f7c009040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.061+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f840dbd80 msgr2=0x7f1f8402e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.061+0000 7f1f8be00640 1 --2- 172.21.15.67:0/2503483394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f840dbd80 0x7f1f8402e360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c410 tx=0x7f1f7800c8d0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.061+0000 7f1f8b400640 1 -- 172.21.15.67:0/2503483394 reap_dead start 2024-02-26T10:38:47.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.061+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 --> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe400074a30 con 0x7fe3dc0736d0 2024-02-26T10:38:47.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.062+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 shutdown_connections 2024-02-26T10:38:47.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.062+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 >> 172.21.15.67:0/2503483394 conn(0x7f1f8402f590 msgr2=0x7f1f8404b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.062+0000 7fe3fcc00640 1 -- 172.21.15.67:0/3984270895 <== osd.3 v2:172.21.15.67:6808/755501566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe400074a30 con 0x7fe3dc0736d0 2024-02-26T10:38:47.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.062+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 shutdown_connections 2024-02-26T10:38:47.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.062+0000 7f1f8be00640 1 -- 172.21.15.67:0/2503483394 wait complete. 2024-02-26T10:38:47.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.063+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fe3dc0736d0 msgr2=0x7fe3dc075af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.063+0000 7fe406e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fe3dc0736d0 0x7fe3dc075af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe3dc06b4d0 msgr2=0x7fe3dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe3dc06b4d0 0x7fe3dc06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe4000c3800 tx=0x7fe3e8009110 comp rx=0 tx=0).stop 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40006ead0 msgr2=0x7fe400129ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 --2- 172.21.15.67:0/3984270895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40006ead0 0x7fe400129ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c4b0 tx=0x7fe3f000c8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe404e00640 1 -- 172.21.15.67:0/3984270895 reap_dead start 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 shutdown_connections 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 >> 172.21.15.67:0/3984270895 conn(0x7fe40006d680 msgr2=0x7fe400150710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 shutdown_connections 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.064+0000 7fe406e00640 1 -- 172.21.15.67:0/3984270895 wait complete. 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.071+0000 7f7262800640 1 -- 172.21.15.67:0/1594007822 --> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f725c1500a0 con 0x7f725c14e5d0 2024-02-26T10:38:47.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.075+0000 7f7253400640 1 -- 172.21.15.67:0/1594007822 <== osd.6 v2:172.21.15.82:6808/1261740917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f725c1500a0 con 0x7f725c14e5d0 2024-02-26T10:38:47.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.075+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 --> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fabbc0c5c60 con 0x7fab9c0738a0 2024-02-26T10:38:47.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.075+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f725c14e5d0 msgr2=0x7f725c14ed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.075+0000 7f7250c00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f725c14e5d0 0x7f725c14ed60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 --> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9040a9af0 con 0x7fa8e40738d0 2024-02-26T10:38:47.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f723c06b5a0 msgr2=0x7f723c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7f7250c00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f723c06b5a0 0x7f723c06da60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f724803b000 tx=0x7f724800d040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 msgr2=0x7f725c132e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7fabb8c00640 1 -- 172.21.15.67:0/767962591 <== osd.7 v2:172.21.15.82:6824/3253207267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fabbc0c5c60 con 0x7fab9c0738a0 2024-02-26T10:38:47.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7f7250c00640 1 --2- 172.21.15.67:0/1594007822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f725c06ead0 0x7f725c132e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f724c00b410 tx=0x7f724c00b8d0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.076+0000 7f725be00640 1 -- 172.21.15.67:0/1594007822 reap_dead start 2024-02-26T10:38:47.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 shutdown_connections 2024-02-26T10:38:47.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 >> 172.21.15.67:0/1594007822 conn(0x7f725c06d680 msgr2=0x7f725c133bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fa901600640 1 -- 172.21.15.67:0/3721691751 <== osd.5 v2:172.21.15.82:6800/431290960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa9040a9af0 con 0x7fa8e40738d0 2024-02-26T10:38:47.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 shutdown_connections 2024-02-26T10:38:47.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7fab9c0738a0 msgr2=0x7fab9c075cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fabc3600640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7fab9c0738a0 0x7fab9c075cc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7f7250c00640 1 -- 172.21.15.67:0/1594007822 wait complete. 2024-02-26T10:38:47.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab9c06b700 msgr2=0x7fab9c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fabc3600640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab9c06b700 0x7fab9c06dbc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fabb0001670 tx=0x7fabb0009040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc155400 msgr2=0x7fabbc1ebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fabc3600640 1 --2- 172.21.15.67:0/767962591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabbc155400 0x7fabbc1ebf80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba8004730 comp rx=0 tx=0).stop 2024-02-26T10:38:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.077+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7fa8e40738d0 msgr2=0x7fa8e4075cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fabc1600640 1 -- 172.21.15.67:0/767962591 reap_dead start 2024-02-26T10:38:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 shutdown_connections 2024-02-26T10:38:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 >> 172.21.15.67:0/767962591 conn(0x7fabbc072980 msgr2=0x7fabbc06c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7fa8e40738d0 0x7fa8e4075cf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8e406b6d0 msgr2=0x7fa8e406db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8e406b6d0 0x7fa8e406db90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8036000 tx=0x7fa8f803b040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9040725b0 msgr2=0x7fa9041f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 --2- 172.21.15.67:0/3721691751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9040725b0 0x7fa9041f1f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00cbf0 tx=0x7fa8fc00a430 comp rx=0 tx=0).stop 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90a400640 1 -- 172.21.15.67:0/3721691751 reap_dead start 2024-02-26T10:38:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 shutdown_connections 2024-02-26T10:38:47.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 >> 172.21.15.67:0/3721691751 conn(0x7fa90406d680 msgr2=0x7fa9040729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 shutdown_connections 2024-02-26T10:38:47.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.078+0000 7fa90c400640 1 -- 172.21.15.67:0/3721691751 wait complete. 2024-02-26T10:38:47.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.079+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 shutdown_connections 2024-02-26T10:38:47.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.079+0000 7fabc3600640 1 -- 172.21.15.67:0/767962591 wait complete. 2024-02-26T10:38:47.107 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-02-26T10:38:47.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-26T10:38:47.108 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:47.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-26T10:38:47.109 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:47.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-26T10:38:47.110 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:47.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-26T10:38:47.111 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-02-26T10:38:47.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-26T10:38:47.112 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:47.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-26T10:38:47.113 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:47.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-26T10:38:47.114 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:47.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-26T10:38:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.446+0000 7f78b3200640 1 Processor -- start 2024-02-26T10:38:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.447+0000 7f78b3200640 1 -- start start 2024-02-26T10:38:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.448+0000 7f78b3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a800f890 0x7f78a800ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.448+0000 7f78b3200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78a80de830 0x7f78a8007600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.448+0000 7f78b3200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a804e190 con 0x7f78a8007220 2024-02-26T10:38:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.448+0000 7f78b3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80dee60 con 0x7f78a800f890 2024-02-26T10:38:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.448+0000 7f78b3200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80df140 con 0x7f78a80de830 2024-02-26T10:38:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.449+0000 7f78b1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a800f890 0x7f78a800ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.450+0000 7f78b1e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f78a8007220 0x7f78a800e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:48264/0 (socket says 172.21.15.67:48264) 2024-02-26T10:38:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.450+0000 7f78b1e00640 1 -- 172.21.15.67:0/1169852755 learned_addr learned my addr 172.21.15.67:0/1169852755 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.449+0000 7f78b2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78a80de830 0x7f78a8007600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.450+0000 7f34a2000640 1 Processor -- start 2024-02-26T10:38:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.451+0000 7f78b1400640 1 -- 172.21.15.67:0/1169852755 >> v2:172.21.15.82:3300/0 conn(0x7f78a80de830 msgr2=0x7f78a8007600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.451+0000 7f78b1400640 1 --2- 172.21.15.67:0/1169852755 >> v2:172.21.15.82:3300/0 conn(0x7f78a80de830 0x7f78a8007600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.450+0000 7f34a2000640 1 -- start start 2024-02-26T10:38:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.451+0000 7f78b1400640 1 -- 172.21.15.67:0/1169852755 >> v1:172.21.15.82:6789/0 conn(0x7f78a8007220 legacy=0x7f78a800e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.451+0000 7f78b1400640 1 -- 172.21.15.67:0/1169852755 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80df420 con 0x7f78a800f890 2024-02-26T10:38:47.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f78b2800640 1 --2- 172.21.15.67:0/1169852755 >> v2:172.21.15.82:3300/0 conn(0x7f78a80de830 0x7f78a8007600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f34a2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 0x7f349c158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f34a2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f349c0bdef0 0x7f349c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f34a2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c0c56e0 0x7f349c155400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f34a2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c154f50 con 0x7f349c0bdef0 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f34a2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0c7440 con 0x7f349c157d30 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f34a2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c7720 con 0x7f349c0c56e0 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 0x7f349c158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 0x7f349c158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38884/0 (socket says 172.21.15.67:38884) 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349b400640 1 -- 172.21.15.67:0/1645240861 learned_addr learned my addr 172.21.15.67:0/1645240861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349aa00640 1 --2- 172.21.15.67:0/1645240861 >> v2:172.21.15.67:3300/0 conn(0x7f349c0bdef0 0x7f349c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.452+0000 7f71d9600640 1 Processor -- start 2024-02-26T10:38:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349be00640 1 --2- 172.21.15.67:0/1645240861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c0c56e0 0x7f349c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349aa00640 1 -- 172.21.15.67:0/1645240861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c0c56e0 msgr2=0x7f349c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349aa00640 1 --2- 172.21.15.67:0/1645240861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c0c56e0 0x7f349c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349aa00640 1 -- 172.21.15.67:0/1645240861 >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 msgr2=0x7f349c158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349aa00640 1 --2- 172.21.15.67:0/1645240861 >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 0x7f349c158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349aa00640 1 -- 172.21.15.67:0/1645240861 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0ad2a0 con 0x7f349c0bdef0 2024-02-26T10:38:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f349b400640 1 --2- 172.21.15.67:0/1645240861 >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 0x7f349c158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f78b1400640 1 --2- 172.21.15.67:0/1169852755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a800f890 0x7f78a800ee60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f78a804d5b0 tx=0x7f789c02bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b425120f0af7ab4f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.454+0000 7f78b0a00640 1 -- 172.21.15.67:0/1169852755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c02d660 con 0x7f78a800f890 2024-02-26T10:38:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.454+0000 7f78b0a00640 1 -- 172.21.15.67:0/1169852755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c02d800 con 0x7f78a800f890 2024-02-26T10:38:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.454+0000 7f78b0a00640 1 -- 172.21.15.67:0/1169852755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c02dbd0 con 0x7f78a800f890 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.455+0000 7f349aa00640 1 --2- 172.21.15.67:0/1645240861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0bdef0 0x7f349c0c51a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f349c073330 tx=0x7f348802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd11c4035e83fe4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f349a000640 1 -- 172.21.15.67:0/1645240861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488037070 con 0x7f349c0bdef0 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f78b3200640 1 -- 172.21.15.67:0/1169852755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a800f890 msgr2=0x7f78a800ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f78b3200640 1 --2- 172.21.15.67:0/1169852755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a800f890 0x7f78a800ee60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f78a804d5b0 tx=0x7f789c02bab0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.453+0000 7f71d9600640 1 -- start start 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f71d9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d4152670 0x7f71d40c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f71d9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71d40c4be0 0x7f71d4158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f71d9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d40c07d0 con 0x7f71d40c6b50 2024-02-26T10:38:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f71d9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d40c5c60 con 0x7f71d40c4be0 2024-02-26T10:38:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f71d9600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c5f40 con 0x7f71d4152670 2024-02-26T10:38:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f78b3200640 1 -- 172.21.15.67:0/1169852755 shutdown_connections 2024-02-26T10:38:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f349a000640 1 -- 172.21.15.67:0/1645240861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3488030710 con 0x7f349c0bdef0 2024-02-26T10:38:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f78b3200640 1 --2- 172.21.15.67:0/1169852755 >> v2:172.21.15.82:3300/0 conn(0x7f78a80de830 0x7f78a8007600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f78b3200640 1 --2- 172.21.15.67:0/1169852755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a800f890 0x7f78a800ee60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f78b3200640 1 -- 172.21.15.67:0/1169852755 >> 172.21.15.67:0/1169852755 conn(0x7f78a804c680 msgr2=0x7f78a804ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.456+0000 7f349a000640 1 -- 172.21.15.67:0/1645240861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488030ae0 con 0x7f349c0bdef0 2024-02-26T10:38:47.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.457+0000 7f78b3200640 1 -- 172.21.15.67:0/1169852755 shutdown_connections 2024-02-26T10:38:47.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.457+0000 7f71d3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f71d40c6b50 0x7f71d40ca710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54546/0 (socket says 172.21.15.67:54546) 2024-02-26T10:38:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.457+0000 7f71d3400640 1 -- 172.21.15.67:0/648115995 learned_addr learned my addr 172.21.15.67:0/648115995 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.457+0000 7f71d2000640 1 --2- 172.21.15.67:0/648115995 >> v2:172.21.15.67:3300/0 conn(0x7f71d40c4be0 0x7f71d4158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f34a2000640 1 -- 172.21.15.67:0/1645240861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0bdef0 msgr2=0x7f349c0c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f71d1600640 1 -- 172.21.15.67:0/648115995 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850067515 0 0) 0x7f71d40c07d0 con 0x7f71d40c6b50 2024-02-26T10:38:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f71d1600640 1 -- 172.21.15.67:0/648115995 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b8003520 con 0x7f71d40c6b50 2024-02-26T10:38:47.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f34a2000640 1 --2- 172.21.15.67:0/1645240861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0bdef0 0x7f349c0c51a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f349c073330 tx=0x7f348802b710 comp rx=0 tx=0).stop 2024-02-26T10:38:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f71d2000640 1 -- 172.21.15.67:0/648115995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d4152670 msgr2=0x7f71d40c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f71d2000640 1 --2- 172.21.15.67:0/648115995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d4152670 0x7f71d40c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f71d2000640 1 -- 172.21.15.67:0/648115995 >> v1:172.21.15.67:6789/0 conn(0x7f71d40c6b50 legacy=0x7f71d40ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.458+0000 7f71d2000640 1 -- 172.21.15.67:0/648115995 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40c0ab0 con 0x7f71d40c4be0 2024-02-26T10:38:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.457+0000 7f78b3200640 1 -- 172.21.15.67:0/1169852755 wait complete. 2024-02-26T10:38:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.459+0000 7f78b3200640 1 Processor -- start 2024-02-26T10:38:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f78b3200640 1 -- start start 2024-02-26T10:38:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f71d2000640 1 --2- 172.21.15.67:0/648115995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d40c4be0 0x7f71d4158410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71c40097a0 tx=0x7f71c402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13168c3c8e47d438 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f71d1600640 1 -- 172.21.15.67:0/648115995 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4031780 con 0x7f71d40c4be0 2024-02-26T10:38:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f71d1600640 1 -- 172.21.15.67:0/648115995 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4037070 con 0x7f71d40c4be0 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f71d1600640 1 -- 172.21.15.67:0/648115995 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4031b40 con 0x7f71d40c4be0 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f34a2000640 1 -- 172.21.15.67:0/1645240861 shutdown_connections 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f34a2000640 1 --2- 172.21.15.67:0/1645240861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c0c56e0 0x7f349c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f34a2000640 1 --2- 172.21.15.67:0/1645240861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0bdef0 0x7f349c0c51a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f34a2000640 1 --2- 172.21.15.67:0/1645240861 >> v2:172.21.15.82:3300/0 conn(0x7f349c157d30 0x7f349c158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.460+0000 7f34a2000640 1 -- 172.21.15.67:0/1645240861 >> 172.21.15.67:0/1645240861 conn(0x7f349c072980 msgr2=0x7f349c0ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f34a2000640 1 -- 172.21.15.67:0/1645240861 shutdown_connections 2024-02-26T10:38:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f6f6e800640 1 Processor -- start 2024-02-26T10:38:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f71d9600640 1 -- 172.21.15.67:0/648115995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d40c4be0 msgr2=0x7f71d4158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f71d9600640 1 --2- 172.21.15.67:0/648115995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d40c4be0 0x7f71d4158410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71c40097a0 tx=0x7f71c402b820 comp rx=0 tx=0).stop 2024-02-26T10:38:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f34a2000640 1 -- 172.21.15.67:0/1645240861 wait complete. 2024-02-26T10:38:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f035f800640 1 Processor -- start 2024-02-26T10:38:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.461+0000 7f6f6e800640 1 -- start start 2024-02-26T10:38:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.462+0000 7f035f800640 1 -- start start 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.462+0000 7f78b3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a8007220 0x7f78a8033ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f78b3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78a800f890 0x7f78a80dd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f78b3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 0x7f78a8177e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f68158d80 0x7f6f680c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f6f6e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f68155840 con 0x7f6f68152460 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f6f6e800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f68155b20 con 0x7f6f680c54e0 2024-02-26T10:38:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f6f6e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68158330 con 0x7f6f68158d80 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.462+0000 7f71d9600640 1 -- 172.21.15.67:0/648115995 shutdown_connections 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.462+0000 7f71d9600640 1 --2- 172.21.15.67:0/648115995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d40c4be0 0x7f71d4158410 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71c40097a0 tx=0x7f71c402b820 comp rx=0 tx=0).stop 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.462+0000 7f71d9600640 1 --2- 172.21.15.67:0/648115995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d4152670 0x7f71d40c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.462+0000 7f71d9600640 1 -- 172.21.15.67:0/648115995 >> 172.21.15.67:0/648115995 conn(0x7f71d4072350 msgr2=0x7f71d4072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f34a2000640 1 Processor -- start 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f6f6d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6f68152460 0x7f6f68156230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54560/0 (socket says 172.21.15.67:54560) 2024-02-26T10:38:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.463+0000 7f6f6d400640 1 -- 172.21.15.67:0/2298492176 learned_addr learned my addr 172.21.15.67:0/2298492176 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 0x7f78a8177e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 0x7f78a8177e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33732/0 (socket says 172.21.15.67:33732) 2024-02-26T10:38:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78a800f890 0x7f78a80dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78a800f890 0x7f78a80dd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39532/0 (socket says 172.21.15.67:39532) 2024-02-26T10:38:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f34a2000640 1 -- start start 2024-02-26T10:38:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f71d9600640 1 -- 172.21.15.67:0/648115995 shutdown_connections 2024-02-26T10:38:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7ff7b1a00640 1 Processor -- start 2024-02-26T10:38:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a8007220 0x7f78a8033ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a8007220 0x7f78a8033ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38890/0 (socket says 172.21.15.67:38890) 2024-02-26T10:38:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b2800640 1 -- 172.21.15.67:0/2901919618 learned_addr learned my addr 172.21.15.67:0/2901919618 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f6ca00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f68158d80 0x7f6f680c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042623553 0 0) 0x7f6f68155840 con 0x7f6f68152460 2024-02-26T10:38:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7ff7b1a00640 1 -- start start 2024-02-26T10:38:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8049fa0 con 0x7f78a80de830 2024-02-26T10:38:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a800e230 con 0x7f78a8007220 2024-02-26T10:38:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80dc1b0 con 0x7f78a800f890 2024-02-26T10:38:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f035f800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 0x7f0358158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f71d9600640 1 -- 172.21.15.67:0/648115995 wait complete. 2024-02-26T10:38:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f035f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f0358155840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f035f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0358154f50 con 0x7f03580bdef0 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f035f800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03580c7440 con 0x7f0358157d30 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f035f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580c7720 con 0x7f0358155400 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f035ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 0x7f0358158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f035d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f0358155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f035ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 0x7f0358158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33756/0 (socket says 172.21.15.67:33756) 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f035ce00640 1 -- 172.21.15.67:0/147648460 learned_addr learned my addr 172.21.15.67:0/147648460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7ff7b1a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 0x7ff7ac158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7ff7b1a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff7ac0bdef0 0x7ff7ac0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f34a2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349c0bdef0 0x7f349c0c3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f34a2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0c56e0 0x7f349c1ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f34a2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c157d30 0x7f349c1ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7ff7b1a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac0c56e0 0x7ff7ac155400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f50003520 con 0x7f6f68152460 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1106520478 0 0) 0x7f6f50003520 con 0x7f6f68152460 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f68158d80 msgr2=0x7f6f680c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f68158d80 0x7f6f680c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 >> v1:172.21.15.82:6789/0 conn(0x7f6f680c54e0 legacy=0x7f6f68154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f68158580 con 0x7f6f68152460 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f6f5c001670 con 0x7f6f68152460 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.464+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 >> v1:172.21.15.67:6789/0 conn(0x7f6f68152460 legacy=0x7f6f68156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f67e00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f500039e0 0x7f6f50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f67e00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f50004800 0x7f6f50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f67e00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f50005210 0x7f6f5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68158330 con 0x7f6f500039e0 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5000c0f0 con 0x7f6f50005210 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5000c2f0 con 0x7f6f50004800 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f6d400640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f500039e0 0x7f6f50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.465+0000 7f6f6de00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f50005210 0x7f6f5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7ff7aa000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff7ac0bdef0 0x7ff7ac0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7ff7aaa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 0x7ff7ac158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7ff7aaa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 0x7ff7ac158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38910/0 (socket says 172.21.15.67:38910) 2024-02-26T10:38:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f6f6ca00640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f50004800 0x7f6f50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f6f6d400640 1 -- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f50004800 msgr2=0x7f6f50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f6f6d400640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f50004800 0x7f6f50004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f6f6d400640 1 -- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f50005210 msgr2=0x7f6f5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7ff7aaa00640 1 -- 172.21.15.67:0/995095733 learned_addr learned my addr 172.21.15.67:0/995095733 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7ff7b1a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac154f50 con 0x7ff7ac0bdef0 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f6f6d400640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f50005210 0x7f6f5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f6f6d400640 1 -- 172.21.15.67:0/2298492176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f68158580 con 0x7f6f500039e0 2024-02-26T10:38:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0c7440 con 0x7ff7ac157d30 2024-02-26T10:38:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7ff7ab400640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac0c56e0 0x7ff7ac155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0c7720 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.466+0000 7f71d9600640 1 Processor -- start 2024-02-26T10:38:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f349b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349c0bdef0 0x7f349c0c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f349aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0c56e0 0x7f349c1ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f349b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349c0bdef0 0x7f349c0c3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38924/0 (socket says 172.21.15.67:38924) 2024-02-26T10:38:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f78b2800640 1 -- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78a800f890 msgr2=0x7f78a80dd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f78b2800640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78a800f890 0x7f78a80dd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f78b2800640 1 -- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a8007220 msgr2=0x7f78a8033ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f349b400640 1 -- 172.21.15.67:0/2005421973 learned_addr learned my addr 172.21.15.67:0/2005421973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f78b2800640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a8007220 0x7f78a8033ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f78b2800640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8047e00 con 0x7f78a80de830 2024-02-26T10:38:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c156330 con 0x7f349c0c56e0 2024-02-26T10:38:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c7a00 con 0x7f349c0bdef0 2024-02-26T10:38:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c155940 con 0x7f349c157d30 2024-02-26T10:38:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.467+0000 7f035d800640 1 -- 172.21.15.67:0/147648460 >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 msgr2=0x7f0358158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f71d9600640 1 -- start start 2024-02-26T10:38:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f6f6d400640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f500039e0 0x7f6f50003e00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c001050 tx=0x7f6f50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6515fbb6782ad630 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349be00640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c157d30 0x7f349c1ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c007d10 con 0x7f6f500039e0 2024-02-26T10:38:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f035d800640 1 --2- 172.21.15.67:0/147648460 >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 0x7f0358158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f035d800640 1 -- 172.21.15.67:0/147648460 >> v1:172.21.15.82:6789/0 conn(0x7f03580bdef0 legacy=0x7f03580c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f035d800640 1 -- 172.21.15.67:0/147648460 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580ad2a0 con 0x7f0358155400 2024-02-26T10:38:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7ff7aa000640 1 -- 172.21.15.67:0/995095733 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac0c56e0 msgr2=0x7ff7ac155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f78b2800640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 0x7f78a8177e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78a000e880 tx=0x7f78a000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7ff7aa000640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac0c56e0 0x7ff7ac155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7ff7aa000640 1 -- 172.21.15.67:0/995095733 >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 msgr2=0x7ff7ac158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7ff7aa000640 1 --2- 172.21.15.67:0/995095733 >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 0x7ff7ac158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7ff7aa000640 1 -- 172.21.15.67:0/995095733 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0ad2a0 con 0x7ff7ac0bdef0 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7ff7aaa00640 1 --2- 172.21.15.67:0/995095733 >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 0x7ff7ac158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349aa00640 1 -- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c157d30 msgr2=0x7f349c1ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349aa00640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c157d30 0x7f349c1ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349aa00640 1 -- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349c0bdef0 msgr2=0x7f349c0c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349aa00640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349c0bdef0 0x7f349c0c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349aa00640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0c83f0 con 0x7f349c0c56e0 2024-02-26T10:38:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.468+0000 7f349be00640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c157d30 0x7f349c1ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:47.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7ff7ab400640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac0c56e0 0x7ff7ac155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0017070 con 0x7f78a80de830 2024-02-26T10:38:47.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f035ce00640 1 --2- 172.21.15.67:0/147648460 >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 0x7f0358158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f349aa00640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0c56e0 0x7f349c1ec3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3488032000 tx=0x7f34880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f71d9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d40c6b50 0x7f71d414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488040020 con 0x7f349c0c56e0 2024-02-26T10:38:47.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34880329d0 con 0x7f349c0c56e0 2024-02-26T10:38:47.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488037040 con 0x7f349c0c56e0 2024-02-26T10:38:47.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a80480e0 con 0x7f78a80de830 2024-02-26T10:38:47.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a80ddea0 con 0x7f78a80de830 2024-02-26T10:38:47.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c0c86d0 con 0x7f349c0c56e0 2024-02-26T10:38:47.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c030070 con 0x7f6f500039e0 2024-02-26T10:38:47.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.469+0000 7f6f67e00640 1 -- 172.21.15.67:0/2298492176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c02cd60 con 0x7f6f500039e0 2024-02-26T10:38:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0004020 con 0x7f78a80de830 2024-02-26T10:38:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0017070 con 0x7f78a80de830 2024-02-26T10:38:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f035d800640 1 --2- 172.21.15.67:0/147648460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f0358155840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0350009d90 tx=0x7f035002bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec124457965d6da server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f71d2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d40c6b50 0x7f71d414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f71d2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d40c6b50 0x7f71d414a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38926/0 (socket says 172.21.15.67:38926) 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d4152670 0x7f71d4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d2a00640 1 -- 172.21.15.67:0/3528863324 learned_addr learned my addr 172.21.15.67:0/3528863324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d414ca40 0x7f71d413f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40cae20 con 0x7f71d4152670 2024-02-26T10:38:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c0c2a20 con 0x7f349c0c56e0 2024-02-26T10:38:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40c0480 con 0x7f71d40c6b50 2024-02-26T10:38:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d40cafa0 con 0x7f71d414ca40 2024-02-26T10:38:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f6f6e800640 1 -- 172.21.15.67:0/2298492176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f500039e0 msgr2=0x7f6f50003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.470+0000 7f6f6e800640 1 --2- 172.21.15.67:0/2298492176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f500039e0 0x7f6f50003e00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c001050 tx=0x7f6f50005830 comp rx=0 tx=0).stop 2024-02-26T10:38:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f6f6de00640 1 -- 172.21.15.67:0/2298492176 reap_dead start 2024-02-26T10:38:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f0357e00640 1 -- 172.21.15.67:0/147648460 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350039070 con 0x7f0358155400 2024-02-26T10:38:47.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f71d3400640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d414ca40 0x7f71d413f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f6f6e800640 1 -- 172.21.15.67:0/2298492176 shutdown_connections 2024-02-26T10:38:47.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.471+0000 7f6f6e800640 1 -- 172.21.15.67:0/2298492176 >> 172.21.15.67:0/2298492176 conn(0x7f6f680ac1b0 msgr2=0x7f6f680ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f6f6e800640 1 -- 172.21.15.67:0/2298492176 shutdown_connections 2024-02-26T10:38:47.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f6f6e800640 1 -- 172.21.15.67:0/2298492176 wait complete. 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f0357e00640 1 -- 172.21.15.67:0/147648460 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035002cb40 con 0x7f0358155400 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f0357e00640 1 -- 172.21.15.67:0/147648460 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350034440 con 0x7f0358155400 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f71d3400640 1 -- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d40c6b50 msgr2=0x7f71d414a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f71d3400640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d40c6b50 0x7f71d414a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f71d3400640 1 -- 172.21.15.67:0/3528863324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d4152670 msgr2=0x7f71d4148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f71d3400640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d4152670 0x7f71d4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f71d3400640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d414a5e0 con 0x7f71d414ca40 2024-02-26T10:38:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3474005290 con 0x7f349c0c56e0 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 -- 172.21.15.67:0/147648460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 msgr2=0x7f0358155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 --2- 172.21.15.67:0/147648460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f0358155840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0350009d90 tx=0x7f035002bd90 comp rx=0 tx=0).stop 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 -- 172.21.15.67:0/147648460 shutdown_connections 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 --2- 172.21.15.67:0/147648460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f0358155840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 --2- 172.21.15.67:0/147648460 >> v2:172.21.15.67:3300/0 conn(0x7f0358157d30 0x7f0358158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 -- 172.21.15.67:0/147648460 >> 172.21.15.67:0/147648460 conn(0x7f0358072980 msgr2=0x7f03580ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 -- 172.21.15.67:0/147648460 shutdown_connections 2024-02-26T10:38:47.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.472+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7888005290 con 0x7f78a80de830 2024-02-26T10:38:47.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.473+0000 7f035f800640 1 -- 172.21.15.67:0/147648460 wait complete. 2024-02-26T10:38:47.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.474+0000 7f6f6e800640 1 Processor -- start 2024-02-26T10:38:47.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.474+0000 7f6f6e800640 1 -- start start 2024-02-26T10:38:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.474+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78a00041d0 con 0x7f78a80de830 2024-02-26T10:38:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.474+0000 7ff7aa000640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0c51a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac073330 tx=0x7ff79802b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8eab35ccdcef3d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.475+0000 7ff7a9600640 1 -- 172.21.15.67:0/995095733 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798037070 con 0x7ff7ac0bdef0 2024-02-26T10:38:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.475+0000 7ff7a9600640 1 -- 172.21.15.67:0/995095733 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798030710 con 0x7ff7ac0bdef0 2024-02-26T10:38:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.475+0000 7ff7a9600640 1 -- 172.21.15.67:0/995095733 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798030ae0 con 0x7ff7ac0bdef0 2024-02-26T10:38:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.475+0000 7f71d3400640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d414ca40 0x7f71d413f2d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00caa0 tx=0x7f71cc00cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f681f26e0 0x7f6f680be440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 0x7f6f6814f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f035f800640 1 Processor -- start 2024-02-26T10:38:47.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6814fef0 0x7f6f680bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68155490 con 0x7f6f681f2ac0 2024-02-26T10:38:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681588f0 con 0x7f6f681f26e0 2024-02-26T10:38:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c8e30 con 0x7f6f6814fef0 2024-02-26T10:38:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 0x7f6f6814f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3488043020 con 0x7f349c0c56e0 2024-02-26T10:38:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 0x7f6f6814f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33782/0 (socket says 172.21.15.67:33782) 2024-02-26T10:38:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f6f6ca00640 1 -- 172.21.15.67:0/1354565070 learned_addr learned my addr 172.21.15.67:0/1354565070 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6d400640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f681f26e0 0x7f6f680be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6de00640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6814fef0 0x7f6f680bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0bdef0 msgr2=0x7ff7ac0c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0c51a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac073330 tx=0x7ff79802b730 comp rx=0 tx=0).stop 2024-02-26T10:38:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6ca00640 1 -- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6814fef0 msgr2=0x7f6f680bedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6ca00640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6814fef0 0x7f6f680bedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6ca00640 1 -- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f681f26e0 msgr2=0x7f6f680be440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6ca00640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f681f26e0 0x7f6f680be440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f6f6ca00640 1 -- 172.21.15.67:0/1354565070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680be980 con 0x7f6f681f2ac0 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.476+0000 7f035f800640 1 -- start start 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f035f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03580bdef0 0x7f03580ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f035f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f03580cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f035f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0358157d30 0x7f03580711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f035f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580c7a00 con 0x7f0358157d30 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f035f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0358158110 con 0x7f03580bdef0 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f035f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0358156330 con 0x7f0358155400 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f78a6000640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f789006b6d0 0x7f789006db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.477+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78a0089ea0 con 0x7f78a80de830 2024-02-26T10:38:47.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 shutdown_connections 2024-02-26T10:38:47.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc017070 con 0x7f71d414ca40 2024-02-26T10:38:47.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00ab40 con 0x7f71d414ca40 2024-02-26T10:38:47.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc012400 con 0x7f71d414ca40 2024-02-26T10:38:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac0c56e0 0x7ff7ac155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f6f6ca00640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 0x7f6f6814f9b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f540107c0 tx=0x7f6f54010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f035d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0358157d30 0x7f03580711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/995095733 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0c51a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/995095733 >> v2:172.21.15.82:3300/0 conn(0x7ff7ac157d30 0x7ff7ac158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f035ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03580bdef0 0x7f03580ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f035ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03580bdef0 0x7f03580ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38946/0 (socket says 172.21.15.67:38946) 2024-02-26T10:38:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f035ce00640 1 -- 172.21.15.67:0/3460510920 learned_addr learned my addr 172.21.15.67:0/3460510920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f3493400640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f347c06b520 0x7f347c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34880ab890 con 0x7f349c0c56e0 2024-02-26T10:38:47.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f034fe00640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f03580cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 >> 172.21.15.67:0/995095733 conn(0x7ff7ac072980 msgr2=0x7ff7ac0ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d41f0430 con 0x7f71d414ca40 2024-02-26T10:38:47.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.478+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d4147870 con 0x7f71d414ca40 2024-02-26T10:38:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f211b600640 1 Processor -- start 2024-02-26T10:38:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f349b400640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f347c06b520 0x7f347c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f035d800640 1 -- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 msgr2=0x7f03580cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f035d800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f03580cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f035d800640 1 -- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03580bdef0 msgr2=0x7f03580ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f035d800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03580bdef0 0x7f03580ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f035d800640 1 -- 172.21.15.67:0/3460510920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580cb230 con 0x7f0358157d30 2024-02-26T10:38:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f211b600640 1 -- start start 2024-02-26T10:38:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.479+0000 7f78b1e00640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f789006b6d0 0x7f789006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 shutdown_connections 2024-02-26T10:38:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7f035d800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0358157d30 0x7f03580711a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0350033000 tx=0x7f0350004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7f124a800640 1 Processor -- start 2024-02-26T10:38:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7ff7b1a00640 1 -- 172.21.15.67:0/995095733 wait complete. 2024-02-26T10:38:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7f124a800640 1 -- start start 2024-02-26T10:38:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7f78b1e00640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f789006b6d0 0x7f789006db90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78ac04fd70 tx=0x7f78ac04ff50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.480+0000 7f349b400640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f347c06b520 0x7f347c06d9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f348c009630 tx=0x7f348c009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.481+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac005290 con 0x7f71d414ca40 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.481+0000 7f211b600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2114157d30 0x7f2114158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.481+0000 7f211b600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21140bdef0 0x7f21140c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.481+0000 7f211b600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2114154f50 con 0x7f21140c56e0 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.481+0000 7f211b600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21140c7440 con 0x7f21140bdef0 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.481+0000 7f211b600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21140c7720 con 0x7f2114157d30 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21140bdef0 0x7f21140c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f21140bdef0 0x7f21140c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33804/0 (socket says 172.21.15.67:33804) 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 -- 172.21.15.67:0/1043115139 learned_addr learned my addr 172.21.15.67:0/1043115139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2118c00640 1 --2- 172.21.15.67:0/1043115139 >> v2:172.21.15.82:3300/0 conn(0x7f2114157d30 0x7f2114158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f124a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1244072fe0 0x7f124406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f124a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1244159210 0x7f1244159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f124a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124415c8a0 con 0x7f1244156810 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f124a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124415cb80 con 0x7f1244159210 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f124a800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124415ce60 con 0x7f1244072fe0 2024-02-26T10:38:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7ff7b1a00640 1 Processor -- start 2024-02-26T10:38:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 -- 172.21.15.67:0/1043115139 >> v2:172.21.15.82:3300/0 conn(0x7f2114157d30 msgr2=0x7f2114158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 --2- 172.21.15.67:0/1043115139 >> v2:172.21.15.82:3300/0 conn(0x7f2114157d30 0x7f2114158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 -- 172.21.15.67:0/1043115139 >> v1:172.21.15.82:6789/0 conn(0x7f21140c56e0 legacy=0x7f2114155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f2113e00640 1 -- 172.21.15.67:0/1043115139 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21140ad2a0 con 0x7f21140bdef0 2024-02-26T10:38:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f1243400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1244156810 0x7f12440c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54562/0 (socket says 172.21.15.67:54562) 2024-02-26T10:38:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.482+0000 7f1243400640 1 -- 172.21.15.67:0/1675185574 learned_addr learned my addr 172.21.15.67:0/1675185574 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7f1242a00640 1 --2- 172.21.15.67:0/1675185574 >> v2:172.21.15.82:3300/0 conn(0x7f1244072fe0 0x7f124406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7f1243e00640 1 --2- 172.21.15.67:0/1675185574 >> v2:172.21.15.67:3300/0 conn(0x7f1244159210 0x7f1244159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 -- start start 2024-02-26T10:38:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78a00586c0 con 0x7f78a80de830 2024-02-26T10:38:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 0x7ff7ac0be910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 0x7ff7ac1edea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac156330 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0c7a00 con 0x7ff7ac0bdef0 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.483+0000 7ff7b1a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac155940 con 0x7ff7ac157d30 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71cc01a070 con 0x7f71d414ca40 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7f1243e00640 1 -- 172.21.15.67:0/1675185574 >> v2:172.21.15.82:3300/0 conn(0x7f1244072fe0 msgr2=0x7f124406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7f1243e00640 1 --2- 172.21.15.67:0/1675185574 >> v2:172.21.15.82:3300/0 conn(0x7f1244072fe0 0x7f124406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7f1243e00640 1 -- 172.21.15.67:0/1675185574 >> v1:172.21.15.67:6789/0 conn(0x7f1244156810 legacy=0x7f12440c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7f1243e00640 1 -- 172.21.15.67:0/1675185574 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124415d140 con 0x7f1244159210 2024-02-26T10:38:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 0x7ff7ac0be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 0x7ff7ac0be910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33822/0 (socket says 172.21.15.67:33822) 2024-02-26T10:38:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7ff7ab400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 0x7ff7ac1edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7ff7ab400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 0x7ff7ac1edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39580/0 (socket says 172.21.15.67:39580) 2024-02-26T10:38:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7ff7ab400640 1 -- 172.21.15.67:0/1463621337 learned_addr learned my addr 172.21.15.67:0/1463621337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.484+0000 7f2113e00640 1 --2- 172.21.15.67:0/1043115139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21140bdef0 0x7f21140c51a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2114073330 tx=0x7f210002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e87b740c61ccb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aaa00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aa000640 1 -- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 msgr2=0x7ff7ac1edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7f71caa00640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f71b406b5d0 0x7f71b406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aa000640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 0x7ff7ac1edea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aa000640 1 -- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7ac0bdef0 msgr2=0x7ff7ac0cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aa000640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aa000640 1 -- 172.21.15.67:0/1463621337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0c24a0 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aaa00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7ab400640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 0x7ff7ac1edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71cc08a3e0 con 0x7f71d414ca40 2024-02-26T10:38:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.485+0000 7ff7aa000640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 0x7ff7ac0be910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0c80c0 tx=0x7ff7980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.486+0000 7f71d2a00640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f71b406b5d0 0x7f71b406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.486+0000 7f1243e00640 1 --2- 172.21.15.67:0/1675185574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244159210 0x7f1244159660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f124406bcc0 tx=0x7f123c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f41fdff0e295ef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.487+0000 7f71d2a00640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f71b406b5d0 0x7f71b406da90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009950 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400f040 con 0x7f6f681f2ac0 2024-02-26T10:38:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798040020 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5400eb60 con 0x7f6f681f2ac0 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400f040 con 0x7f6f681f2ac0 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f681f3020 con 0x7f6f681f2ac0 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350044020 con 0x7f0358157d30 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035003f470 con 0x7f0358157d30 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350039040 con 0x7f0358157d30 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac0c2780 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac0c2d40 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f2113400640 1 -- 172.21.15.67:0/1043115139 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210002e070 con 0x7f21140bdef0 2024-02-26T10:38:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f2113400640 1 -- 172.21.15.67:0/1043115139 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2100037070 con 0x7f21140bdef0 2024-02-26T10:38:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f1242000640 1 -- 172.21.15.67:0/1675185574 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c039070 con 0x7f1244159210 2024-02-26T10:38:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f1242000640 1 -- 172.21.15.67:0/1675185574 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c02cbe0 con 0x7f1244159210 2024-02-26T10:38:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f1242000640 1 -- 172.21.15.67:0/1675185574 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c034440 con 0x7f1244159210 2024-02-26T10:38:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f680c9360 con 0x7f6f681f2ac0 2024-02-26T10:38:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.490+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03581efad0 con 0x7f0358157d30 2024-02-26T10:38:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03580c8620 con 0x7f0358157d30 2024-02-26T10:38:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff798030710 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.489+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798037040 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.490+0000 7f211b600640 1 -- 172.21.15.67:0/1043115139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21140bdef0 msgr2=0x7f21140c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.490+0000 7f211b600640 1 --2- 172.21.15.67:0/1043115139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21140bdef0 0x7f21140c51a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2114073330 tx=0x7f210002b880 comp rx=0 tx=0).stop 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f211b600640 1 -- 172.21.15.67:0/1043115139 shutdown_connections 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f211b600640 1 --2- 172.21.15.67:0/1043115139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21140bdef0 0x7f21140c51a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f211b600640 1 --2- 172.21.15.67:0/1043115139 >> v2:172.21.15.82:3300/0 conn(0x7f2114157d30 0x7f2114158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f211b600640 1 -- 172.21.15.67:0/1043115139 >> 172.21.15.67:0/1043115139 conn(0x7f2114072980 msgr2=0x7f21140ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6806acd0 con 0x7f6f681f2ac0 2024-02-26T10:38:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f124a800640 1 -- 172.21.15.67:0/1675185574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244159210 msgr2=0x7f1244159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.491+0000 7f124a800640 1 --2- 172.21.15.67:0/1675185574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244159210 0x7f1244159660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f124406bcc0 tx=0x7f123c02be30 comp rx=0 tx=0).stop 2024-02-26T10:38:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f348804a030 con 0x7f349c0c56e0 2024-02-26T10:38:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff784005290 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f211b600640 1 -- 172.21.15.67:0/1043115139 shutdown_connections 2024-02-26T10:38:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f124a800640 1 -- 172.21.15.67:0/1675185574 shutdown_connections 2024-02-26T10:38:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f124a800640 1 --2- 172.21.15.67:0/1675185574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244159210 0x7f1244159660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f124a800640 1 --2- 172.21.15.67:0/1675185574 >> v2:172.21.15.82:3300/0 conn(0x7f1244072fe0 0x7f124406e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f124a800640 1 -- 172.21.15.67:0/1675185574 >> 172.21.15.67:0/1675185574 conn(0x7f12440a84a0 msgr2=0x7f12440a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0358074a30 con 0x7f0358157d30 2024-02-26T10:38:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f211b600640 1 -- 172.21.15.67:0/1043115139 wait complete. 2024-02-26T10:38:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.492+0000 7f124a800640 1 -- 172.21.15.67:0/1675185574 shutdown_connections 2024-02-26T10:38:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.493+0000 7f124a800640 1 -- 172.21.15.67:0/1675185574 wait complete. 2024-02-26T10:38:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.494+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0350047020 con 0x7f0358157d30 2024-02-26T10:38:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.494+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f5400ed10 con 0x7f6f681f2ac0 2024-02-26T10:38:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.495+0000 7f124a800640 1 Processor -- start 2024-02-26T10:38:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.495+0000 7f124a800640 1 -- start start 2024-02-26T10:38:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.496+0000 7f211b600640 1 Processor -- start 2024-02-26T10:38:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.496+0000 7f0355600640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f033406b520 0x7f033406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.496+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03500795a0 con 0x7f0358157d30 2024-02-26T10:38:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.495+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff798043020 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7ff7a2a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78c06b500 0x7ff78c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7980ab5b0 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.496+0000 7f211b600640 1 -- start start 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f124a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 0x7f12441552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f124a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244156810 0x7f12441ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f124a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1244159210 0x7f12441f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f124a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124406ec40 con 0x7f1244156810 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f124a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124415d660 con 0x7f1244072fe0 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f124a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1244159ba0 con 0x7f1244159210 2024-02-26T10:38:47.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.497+0000 7f035ce00640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f033406b520 0x7f033406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.498+0000 7f1243400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 0x7f12441552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.498+0000 7f1243400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 0x7f12441552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38976/0 (socket says 172.21.15.67:38976) 2024-02-26T10:38:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.498+0000 7f1243400640 1 -- 172.21.15.67:0/1193727083 learned_addr learned my addr 172.21.15.67:0/1193727083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.498+0000 7f1243e00640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1244159210 0x7f12441f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.496+0000 7f6f65600640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f4806b6d0 0x7f6f4806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.498+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f5408a250 con 0x7f6f681f2ac0 2024-02-26T10:38:47.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.498+0000 7f6f6d400640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f4806b6d0 0x7f6f4806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7ff7aaa00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78c06b500 0x7ff78c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f211b600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140bdef0 0x7f21140bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f211b600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c56e0 0x7f21140c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f1242a00640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244156810 0x7f12441ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f1243400640 1 -- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1244159210 msgr2=0x7f12441f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f1243400640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1244159210 0x7f12441f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f1243400640 1 -- 172.21.15.67:0/1193727083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244156810 msgr2=0x7f12441ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f1243400640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244156810 0x7f12441ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f211b600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 0x7f21140c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f211b600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114156330 con 0x7f2114157d30 2024-02-26T10:38:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f211b600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21140c7a00 con 0x7f21140bdef0 2024-02-26T10:38:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f1243400640 1 -- 172.21.15.67:0/1193727083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440aafc0 con 0x7f1244072fe0 2024-02-26T10:38:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f211b600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2114155b10 con 0x7f21140c56e0 2024-02-26T10:38:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f2119600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 0x7f21140c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f2119600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 0x7f21140c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33840/0 (socket says 172.21.15.67:33840) 2024-02-26T10:38:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f2119600640 1 -- 172.21.15.67:0/2952948213 learned_addr learned my addr 172.21.15.67:0/2952948213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.499+0000 7f2113e00640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c56e0 0x7f21140c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f035ce00640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f033406b520 0x7f033406d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0340009630 tx=0x7f0340008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2118c00640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140bdef0 0x7f21140bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2119600640 1 -- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c56e0 msgr2=0x7f21140c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2119600640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c56e0 0x7f21140c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2119600640 1 -- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140bdef0 msgr2=0x7f21140bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2119600640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140bdef0 0x7f21140bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2119600640 1 -- 172.21.15.67:0/2952948213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21141e4b40 con 0x7f2114157d30 2024-02-26T10:38:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f6f6d400640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f4806b6d0 0x7f6f4806db90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034000 tx=0x7f6f5c039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f1243400640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 0x7f12441552f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f123400c4a0 tx=0x7f123400c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7f2119600640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 0x7f21140c4360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f210c00c7b0 tx=0x7f210c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.500+0000 7ff7aaa00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78c06b500 0x7ff78c06d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009ce0 tx=0x7ff7a400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.501+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234017070 con 0x7f1244072fe0 2024-02-26T10:38:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.501+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1234005ce0 con 0x7f1244072fe0 2024-02-26T10:38:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.501+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234007270 con 0x7f1244072fe0 2024-02-26T10:38:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.501+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12440ab2a0 con 0x7f1244072fe0 2024-02-26T10:38:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.501+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12441eedc0 con 0x7f1244072fe0 2024-02-26T10:38:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.502+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71cc021020 con 0x7f71d414ca40 2024-02-26T10:38:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.503+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12441eebb0 con 0x7f1244072fe0 2024-02-26T10:38:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.503+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f6806acd0 con 0x7f6f681f2ac0 2024-02-26T10:38:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.507+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f123401a070 con 0x7f1244072fe0 2024-02-26T10:38:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.508+0000 7f123b400640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f122406b500 0x7f122406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.509+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c004b60 con 0x7f2114157d30 2024-02-26T10:38:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.509+0000 7f211b600640 1 -- 172.21.15.67:0/2952948213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21141e4e20 con 0x7f2114157d30 2024-02-26T10:38:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.509+0000 7f211b600640 1 -- 172.21.15.67:0/2952948213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21141e09c0 con 0x7f2114157d30 2024-02-26T10:38:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.509+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1234089960 con 0x7f1244072fe0 2024-02-26T10:38:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.510+0000 7f1242a00640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f122406b500 0x7f122406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.511+0000 7f211b600640 1 -- 172.21.15.67:0/2952948213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21141e5900 con 0x7f2114157d30 2024-02-26T10:38:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.511+0000 7f1242a00640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f122406b500 0x7f122406d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12440c2d50 tx=0x7f1230008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.511+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff79804a030 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.516+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c004d00 con 0x7f2114157d30 2024-02-26T10:38:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.516+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c006d50 con 0x7f2114157d30 2024-02-26T10:38:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.517+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f210c00a850 con 0x7f2114157d30 2024-02-26T10:38:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.518+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f035004e030 con 0x7f0358157d30 2024-02-26T10:38:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.520+0000 7f2110c00640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f406b4d0 0x7f20f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.520+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f210c08ac40 con 0x7f2114157d30 2024-02-26T10:38:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.520+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f210c020070 con 0x7f2114157d30 2024-02-26T10:38:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.521+0000 7f2118c00640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f406b4d0 0x7f20f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.522+0000 7f2118c00640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f406b4d0 0x7f20f406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2108006fd0 tx=0x7f2108008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:47.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.525+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1234058200 con 0x7f1244072fe0 2024-02-26T10:38:47.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.913+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7888005580 con 0x7f78a80de830 2024-02-26T10:38:47.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.914+0000 7f78a6000640 1 -- 172.21.15.67:0/2901919618 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78a005c760 con 0x7f78a80de830 2024-02-26T10:38:47.919 INFO:teuthology.orchestra.run.smithi067.stdout:60129542148 2024-02-26T10:38:47.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.918+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3474005850 con 0x7f349c0c56e0 2024-02-26T10:38:47.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.918+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f789006b6d0 msgr2=0x7f789006db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.918+0000 7f78b3200640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f789006b6d0 0x7f789006db90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78ac04fd70 tx=0x7f78ac04ff50 comp rx=0 tx=0).stop 2024-02-26T10:38:47.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.918+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 msgr2=0x7f78a8177e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.918+0000 7f78b3200640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 0x7f78a8177e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78a000e880 tx=0x7f78a000ed40 comp rx=0 tx=0).stop 2024-02-26T10:38:47.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 shutdown_connections 2024-02-26T10:38:47.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80de830 0x7f78a8177e40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78a800f890 0x7f78a80dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f789006b6d0 0x7f789006db90 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78ac04fd70 tx=0x7f78ac04ff50 comp rx=0 tx=0).stop 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 --2- 172.21.15.67:0/2901919618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a8007220 0x7f78a8033ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 >> 172.21.15.67:0/2901919618 conn(0x7f78a804c680 msgr2=0x7f78a804e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 shutdown_connections 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f3493400640 1 -- 172.21.15.67:0/2005421973 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f348807a880 con 0x7f349c0c56e0 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.919+0000 7f78b3200640 1 -- 172.21.15.67:0/2901919618 wait complete. 2024-02-26T10:38:47.922 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:47.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.924+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f347c06b520 msgr2=0x7f347c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.924+0000 7f34a2000640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f347c06b520 0x7f347c06d9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f348c009630 tx=0x7f348c009240 comp rx=0 tx=0).stop 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0c56e0 msgr2=0x7f349c1ec3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0c56e0 0x7f349c1ec3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3488032000 tx=0x7f34880046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 shutdown_connections 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f349c157d30 0x7f349c1ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f349c0c56e0 0x7f349c1ec3a0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3488032000 tx=0x7f34880046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f347c06b520 0x7f347c06d9e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 --2- 172.21.15.67:0/2005421973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349c0bdef0 0x7f349c0c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 >> 172.21.15.67:0/2005421973 conn(0x7f349c072980 msgr2=0x7f349c153750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 shutdown_connections 2024-02-26T10:38:47.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.925+0000 7f34a2000640 1 -- 172.21.15.67:0/2005421973 wait complete. 2024-02-26T10:38:47.945 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:47.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.941+0000 7f211b600640 1 -- 172.21.15.67:0/2952948213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f21140af8d0 con 0x7f2114157d30 2024-02-26T10:38:47.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.941+0000 7f2110c00640 1 -- 172.21.15.67:0/2952948213 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f210c019350 con 0x7f2114157d30 2024-02-26T10:38:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.945+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f68074a30 con 0x7f6f681f2ac0 2024-02-26T10:38:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.945+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71ac005850 con 0x7f71d414ca40 2024-02-26T10:38:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 -- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f406b4d0 msgr2=0x7f20f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f406b4d0 0x7f20f406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2108006fd0 tx=0x7f2108008040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 -- 172.21.15.67:0/2952948213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 msgr2=0x7f21140c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 0x7f21140c4360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f210c00c7b0 tx=0x7f210c00cc70 comp rx=0 tx=0).stop 2024-02-26T10:38:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f6f65600640 1 -- 172.21.15.67:0/1354565070 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f680be980 con 0x7f6f681f2ac0 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 -- 172.21.15.67:0/2952948213 shutdown_connections 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2114157d30 0x7f21140c4360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c56e0 0x7f21140c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f406b4d0 0x7f20f406d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 --2- 172.21.15.67:0/2952948213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140bdef0 0x7f21140bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 -- 172.21.15.67:0/2952948213 >> 172.21.15.67:0/2952948213 conn(0x7f2114072980 msgr2=0x7f2114154610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.949 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 -- 172.21.15.67:0/2952948213 shutdown_connections 2024-02-26T10:38:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.946+0000 7f2106000640 1 -- 172.21.15.67:0/2952948213 wait complete. 2024-02-26T10:38:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.947+0000 7f71caa00640 1 -- 172.21.15.67:0/3528863324 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71cc059240 con 0x7f71d414ca40 2024-02-26T10:38:47.950 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-02-26T10:38:47.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f4806b6d0 msgr2=0x7f6f4806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f4806b6d0 0x7f6f4806db90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034000 tx=0x7f6f5c039040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 msgr2=0x7f6f6814f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 0x7f6f6814f9b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f540107c0 tx=0x7f6f54010c80 comp rx=0 tx=0).stop 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 shutdown_connections 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6814fef0 0x7f6f680bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f681f2ac0 0x7f6f6814f9b0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f540107c0 tx=0x7f6f54010c80 comp rx=0 tx=0).stop 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f4806b6d0 0x7f6f4806db90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 --2- 172.21.15.67:0/1354565070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f681f26e0 0x7f6f680be440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.949+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 >> 172.21.15.67:0/1354565070 conn(0x7f6f680ac1b0 msgr2=0x7f6f680aa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.950+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 shutdown_connections 2024-02-26T10:38:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.950+0000 7f6f6e800640 1 -- 172.21.15.67:0/1354565070 wait complete. 2024-02-26T10:38:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f71b406b5d0 msgr2=0x7f71b406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f71b406b5d0 0x7f71b406da90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009950 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d414ca40 msgr2=0x7f71d413f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d414ca40 0x7f71d413f2d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00caa0 tx=0x7f71cc00cea0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.4 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 shutdown_connections 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71d414ca40 0x7f71d413f2d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71d4152670 0x7f71d4148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f71b406b5d0 0x7f71b406da90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 --2- 172.21.15.67:0/3528863324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d40c6b50 0x7f71d414a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.951+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 >> 172.21.15.67:0/3528863324 conn(0x7f71d4072350 msgr2=0x7f71d4151430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.952+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 shutdown_connections 2024-02-26T10:38:47.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.953+0000 7f71d9600640 1 -- 172.21.15.67:0/3528863324 wait complete. 2024-02-26T10:38:47.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.954+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff784005850 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.955+0000 7ff7a2a00640 1 -- 172.21.15.67:0/1463621337 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff79807a3a0 con 0x7ff7ac0c56e0 2024-02-26T10:38:47.957 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:47.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78c06b500 msgr2=0x7ff78c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78c06b500 0x7ff78c06d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009ce0 tx=0x7ff7a400b040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 msgr2=0x7ff7ac0be910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 0x7ff7ac0be910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0c80c0 tx=0x7ff7980046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 shutdown_connections 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7ac157d30 0x7ff7ac1edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7ac0c56e0 0x7ff7ac0be910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78c06b500 0x7ff78c06d9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 --2- 172.21.15.67:0/1463621337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7ac0bdef0 0x7ff7ac0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 >> 172.21.15.67:0/1463621337 conn(0x7ff7ac072980 msgr2=0x7ff7ac1564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 shutdown_connections 2024-02-26T10:38:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1244074a30 con 0x7f1244072fe0 2024-02-26T10:38:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.958+0000 7ff7b1a00640 1 -- 172.21.15.67:0/1463621337 wait complete. 2024-02-26T10:38:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.959+0000 7f123b400640 1 -- 172.21.15.67:0/1193727083 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f123405c2a0 con 0x7f1244072fe0 2024-02-26T10:38:47.963 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-02-26T10:38:47.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.961+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0358158430 con 0x7f0358157d30 2024-02-26T10:38:47.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.962+0000 7f0355600640 1 -- 172.21.15.67:0/3460510920 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f035007bc00 con 0x7f0358157d30 2024-02-26T10:38:47.964 INFO:teuthology.orchestra.run.smithi067.stdout:60129542148 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f122406b500 msgr2=0x7f122406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f122406b500 0x7f122406d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12440c2d50 tx=0x7f1230008040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 msgr2=0x7f12441552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 0x7f12441552f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f123400c4a0 tx=0x7f123400c960 comp rx=0 tx=0).stop 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 shutdown_connections 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1244159210 0x7f12441f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1244156810 0x7f12441ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f122406b500 0x7f122406d9c0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12440c2d50 tx=0x7f1230008040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 --2- 172.21.15.67:0/1193727083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1244072fe0 0x7f12441552f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 >> 172.21.15.67:0/1193727083 conn(0x7f12440a84a0 msgr2=0x7f12440a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 shutdown_connections 2024-02-26T10:38:47.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.963+0000 7f124a800640 1 -- 172.21.15.67:0/1193727083 wait complete. 2024-02-26T10:38:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f033406b520 msgr2=0x7f033406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f033406b520 0x7f033406d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0340009630 tx=0x7f0340008040 comp rx=0 tx=0).stop 2024-02-26T10:38:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0358157d30 msgr2=0x7f03580711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0358157d30 0x7f03580711a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0350033000 tx=0x7f0350004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:47.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 shutdown_connections 2024-02-26T10:38:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0358157d30 0x7f03580711a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0358155400 0x7f03580cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f033406b520 0x7f033406d9e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 --2- 172.21.15.67:0/3460510920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03580bdef0 0x7f03580ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.966+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 >> 172.21.15.67:0/3460510920 conn(0x7f0358072980 msgr2=0x7f03580aae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.968+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 shutdown_connections 2024-02-26T10:38:47.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:47.968+0000 7f035f800640 1 -- 172.21.15.67:0/3460510920 wait complete. 2024-02-26T10:38:47.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.5 2024-02-26T10:38:47.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.3 2024-02-26T10:38:47.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2024-02-26T10:38:47.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-02-26T10:38:47.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2024-02-26T10:38:47.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-02-26T10:38:48.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-26T10:38:48.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-26T10:38:48.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-26T10:38:48.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-26T10:38:48.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-26T10:38:48.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-26T10:38:48.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-26T10:38:48.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-26T10:38:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.258+0000 7f8544e00640 1 Processor -- start 2024-02-26T10:38:49.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.258+0000 7f8544e00640 1 -- start start 2024-02-26T10:38:49.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.260+0000 7f8544e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8538007210 0x7f85380de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.260+0000 7f8544e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f853800f880 0x7f85380075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.260+0000 7f8544e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853804e180 con 0x7f853800e740 2024-02-26T10:38:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.260+0000 7f8544e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85380dee50 con 0x7f8538007210 2024-02-26T10:38:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.260+0000 7f8544e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85380df130 con 0x7f853800f880 2024-02-26T10:38:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.261+0000 7f853e800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f853800e740 0x7f853800eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:48284/0 (socket says 172.21.15.67:48284) 2024-02-26T10:38:49.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.261+0000 7f853e800640 1 -- 172.21.15.67:0/1175948633 learned_addr learned my addr 172.21.15.67:0/1175948633 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:49.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.262+0000 7f853d400640 1 --2- 172.21.15.67:0/1175948633 >> v2:172.21.15.82:3300/0 conn(0x7f853800f880 0x7f85380075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.262+0000 7f853ca00640 1 -- 172.21.15.67:0/1175948633 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882123 0 0) 0x7f853804e180 con 0x7f853800e740 2024-02-26T10:38:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.263+0000 7f853ca00640 1 -- 172.21.15.67:0/1175948633 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8524003520 con 0x7f853800e740 2024-02-26T10:38:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.263+0000 7f853de00640 1 --2- 172.21.15.67:0/1175948633 >> v2:172.21.15.67:3300/0 conn(0x7f8538007210 0x7f85380de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.263+0000 7f853ca00640 1 -- 172.21.15.67:0/1175948633 <== mon.1 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071658622 0 0) 0x7f8524003520 con 0x7f853800e740 2024-02-26T10:38:49.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.264+0000 7f853d400640 1 -- 172.21.15.67:0/1175948633 >> v2:172.21.15.67:3300/0 conn(0x7f8538007210 msgr2=0x7f85380de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.264+0000 7f853d400640 1 --2- 172.21.15.67:0/1175948633 >> v2:172.21.15.67:3300/0 conn(0x7f8538007210 0x7f85380de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.264+0000 7f853d400640 1 -- 172.21.15.67:0/1175948633 >> v1:172.21.15.82:6789/0 conn(0x7f853800e740 legacy=0x7f853800eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.264+0000 7f853d400640 1 -- 172.21.15.67:0/1175948633 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85380df410 con 0x7f853800f880 2024-02-26T10:38:49.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.265+0000 7f853de00640 1 --2- 172.21.15.67:0/1175948633 >> v2:172.21.15.67:3300/0 conn(0x7f8538007210 0x7f85380de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:49.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.266+0000 7f853d400640 1 --2- 172.21.15.67:0/1175948633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f853800f880 0x7f85380075f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85400507a0 tx=0x7f85400a6750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31122bd8fbb16d5c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.267+0000 7f853ca00640 1 -- 172.21.15.67:0/1175948633 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c001670 con 0x7f853800f880 2024-02-26T10:38:49.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.267+0000 7f853ca00640 1 -- 172.21.15.67:0/1175948633 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85400ab5d0 con 0x7f853800f880 2024-02-26T10:38:49.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.269+0000 7f8544e00640 1 -- 172.21.15.67:0/1175948633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f853800f880 msgr2=0x7f85380075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.269+0000 7f8544e00640 1 --2- 172.21.15.67:0/1175948633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f853800f880 0x7f85380075f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85400507a0 tx=0x7f85400a6750 comp rx=0 tx=0).stop 2024-02-26T10:38:49.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.269+0000 7f8544e00640 1 -- 172.21.15.67:0/1175948633 shutdown_connections 2024-02-26T10:38:49.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.269+0000 7f8544e00640 1 --2- 172.21.15.67:0/1175948633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f853800f880 0x7f85380075f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.270+0000 7f8544e00640 1 --2- 172.21.15.67:0/1175948633 >> v2:172.21.15.67:3300/0 conn(0x7f8538007210 0x7f85380de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.270+0000 7f8544e00640 1 -- 172.21.15.67:0/1175948633 >> 172.21.15.67:0/1175948633 conn(0x7f853804c670 msgr2=0x7f853804ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.270+0000 7f8544e00640 1 -- 172.21.15.67:0/1175948633 shutdown_connections 2024-02-26T10:38:49.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.270+0000 7f8544e00640 1 -- 172.21.15.67:0/1175948633 wait complete. 2024-02-26T10:38:49.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.275+0000 7f8544e00640 1 Processor -- start 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.275+0000 7f8544e00640 1 -- start start 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.276+0000 7f8544e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 0x7f85380c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.276+0000 7f8544e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f853800e740 0x7f85380e21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.277+0000 7f8544e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853800f880 0x7f85380dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.277+0000 7f853d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f853800e740 0x7f85380e21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.278+0000 7f853d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f853800e740 0x7f85380e21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39602/0 (socket says 172.21.15.67:39602) 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.278+0000 7f853de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 0x7f85380c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.278+0000 7f853de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 0x7f85380c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38990/0 (socket says 172.21.15.67:38990) 2024-02-26T10:38:49.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.278+0000 7f853d400640 1 -- 172.21.15.67:0/4224462326 learned_addr learned my addr 172.21.15.67:0/4224462326 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.278+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8538049f90 con 0x7f853800f880 2024-02-26T10:38:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.278+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853800e220 con 0x7f8538007210 2024-02-26T10:38:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.279+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85380dc1a0 con 0x7f853800e740 2024-02-26T10:38:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.279+0000 7f853e800640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853800f880 0x7f85380dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.282+0000 7f853de00640 1 -- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f853800e740 msgr2=0x7f85380e21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.282+0000 7f853de00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f853800e740 0x7f85380e21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.282+0000 7f853de00640 1 -- 172.21.15.67:0/4224462326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853800f880 msgr2=0x7f85380dd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.283+0000 7f853de00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853800f880 0x7f85380dd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.283+0000 7f853de00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85380e1cc0 con 0x7f8538007210 2024-02-26T10:38:49.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.284+0000 7f853e800640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853800f880 0x7f85380dd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:49.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.285+0000 7f853de00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 0x7f85380c7550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f853000c820 tx=0x7f853000cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.295+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530015070 con 0x7f8538007210 2024-02-26T10:38:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.295+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530004220 con 0x7f8538007210 2024-02-26T10:38:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.295+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85380e0580 con 0x7f8538007210 2024-02-26T10:38:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.295+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85380dffc0 con 0x7f8538007210 2024-02-26T10:38:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.295+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000ce10 con 0x7f8538007210 2024-02-26T10:38:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.297+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518005290 con 0x7f8538007210 2024-02-26T10:38:49.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.300+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8530007830 con 0x7f8538007210 2024-02-26T10:38:49.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.302+0000 7f8536000640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852006b5a0 0x7f852006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.302+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8530089f50 con 0x7f8538007210 2024-02-26T10:38:49.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.302+0000 7f853d400640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852006b5a0 0x7f852006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.304+0000 7f853d400640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852006b5a0 0x7f852006da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85400736f0 tx=0x7f85400ab000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.304+0000 7f233ce00640 1 Processor -- start 2024-02-26T10:38:49.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.305+0000 7f233ce00640 1 -- start start 2024-02-26T10:38:49.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.307+0000 7f233ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2338157d30 0x7f2338158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.307+0000 7f233ce00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f23380bdef0 0x7f23380c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.307+0000 7f233ce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2338154f50 con 0x7f23380c56e0 2024-02-26T10:38:49.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.307+0000 7f233ce00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23380c7440 con 0x7f23380bdef0 2024-02-26T10:38:49.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.307+0000 7f233ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380c7720 con 0x7f2338157d30 2024-02-26T10:38:49.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.308+0000 7fc0ff200640 1 Processor -- start 2024-02-26T10:38:49.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.308+0000 7f2335e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2338157d30 0x7f2338158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.308+0000 7f2335400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f23380bdef0 0x7f23380c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.308+0000 7f2336800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f23380c56e0 0x7f2338155400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54570/0 (socket says 172.21.15.67:54570) 2024-02-26T10:38:49.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.308+0000 7f2336800640 1 -- 172.21.15.67:0/964163780 learned_addr learned my addr 172.21.15.67:0/964163780 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:49.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.308+0000 7fc0ff200640 1 -- start start 2024-02-26T10:38:49.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.309+0000 7f2334a00640 1 -- 172.21.15.67:0/964163780 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984561898 0 0) 0x7f2338154f50 con 0x7f23380c56e0 2024-02-26T10:38:49.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.309+0000 7f2334a00640 1 -- 172.21.15.67:0/964163780 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c003520 con 0x7f23380c56e0 2024-02-26T10:38:49.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.309+0000 7f2335e00640 1 -- 172.21.15.67:0/964163780 >> v2:172.21.15.82:3300/0 conn(0x7f23380bdef0 msgr2=0x7f23380c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.309+0000 7f2335e00640 1 --2- 172.21.15.67:0/964163780 >> v2:172.21.15.82:3300/0 conn(0x7f23380bdef0 0x7f23380c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.309+0000 7f2335e00640 1 -- 172.21.15.67:0/964163780 >> v1:172.21.15.67:6789/0 conn(0x7f23380c56e0 legacy=0x7f2338155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0ff200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0f80afeb0 0x7fc0f80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.309+0000 7f2335e00640 1 -- 172.21.15.67:0/964163780 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380ad2a0 con 0x7f2338157d30 2024-02-26T10:38:49.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0ff200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0f80c7710 0x7fc0f8155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0ff200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f8158ea0 con 0x7fc0f806da40 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0ff200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f80c86c0 con 0x7fc0f80afeb0 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0ff200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f8159180 con 0x7fc0f80c7710 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0f7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc0f806da40 0x7fc0f8154e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54586/0 (socket says 172.21.15.67:54586) 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0f7400640 1 -- 172.21.15.67:0/962938775 learned_addr learned my addr 172.21.15.67:0/962938775 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.310+0000 7fc0f7e00640 1 --2- 172.21.15.67:0/962938775 >> v2:172.21.15.67:3300/0 conn(0x7fc0f80afeb0 0x7fc0f80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0fd200640 1 --2- 172.21.15.67:0/962938775 >> v2:172.21.15.82:3300/0 conn(0x7fc0f80c7710 0x7fc0f8155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f6a00640 1 -- 172.21.15.67:0/962938775 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465091638 0 0) 0x7fc0f8158ea0 con 0x7fc0f806da40 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f6a00640 1 -- 172.21.15.67:0/962938775 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0dc003520 con 0x7fc0f806da40 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f7e00640 1 -- 172.21.15.67:0/962938775 >> v2:172.21.15.82:3300/0 conn(0x7fc0f80c7710 msgr2=0x7fc0f8155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f7e00640 1 --2- 172.21.15.67:0/962938775 >> v2:172.21.15.82:3300/0 conn(0x7fc0f80c7710 0x7fc0f8155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.312+0000 7f2335e00640 1 --2- 172.21.15.67:0/964163780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2338157d30 0x7f2338158e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23280097a0 tx=0x7f232802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af34dcccb26cebe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f6a00640 1 -- 172.21.15.67:0/962938775 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2744730587 0 0) 0x7fc0dc003520 con 0x7fc0f806da40 2024-02-26T10:38:49.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f7e00640 1 -- 172.21.15.67:0/962938775 >> v1:172.21.15.67:6789/0 conn(0x7fc0f806da40 legacy=0x7fc0f8154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.311+0000 7fc0f7e00640 1 -- 172.21.15.67:0/962938775 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80c52e0 con 0x7fc0f80afeb0 2024-02-26T10:38:49.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.312+0000 7f432f800640 1 Processor -- start 2024-02-26T10:38:49.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.312+0000 7f2334a00640 1 -- 172.21.15.67:0/964163780 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328031780 con 0x7f2338157d30 2024-02-26T10:38:49.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.312+0000 7f2334a00640 1 -- 172.21.15.67:0/964163780 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328037070 con 0x7f2338157d30 2024-02-26T10:38:49.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.312+0000 7f2334a00640 1 -- 172.21.15.67:0/964163780 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328031b20 con 0x7f2338157d30 2024-02-26T10:38:49.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.312+0000 7f432f800640 1 -- start start 2024-02-26T10:38:49.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.313+0000 7fc0f7e00640 1 --2- 172.21.15.67:0/962938775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f80afeb0 0x7fc0f80c71d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80b0940 tx=0x7fc0e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5027e0329341feea server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.313+0000 7f233ce00640 1 -- 172.21.15.67:0/964163780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2338157d30 msgr2=0x7f2338158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.313+0000 7f233ce00640 1 --2- 172.21.15.67:0/964163780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2338157d30 0x7f2338158e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23280097a0 tx=0x7f232802b820 comp rx=0 tx=0).stop 2024-02-26T10:38:49.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7fc0f6a00640 1 -- 172.21.15.67:0/962938775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8035020 con 0x7fc0f80afeb0 2024-02-26T10:38:49.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7fc0f6a00640 1 -- 172.21.15.67:0/962938775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e802d040 con 0x7fc0f80afeb0 2024-02-26T10:38:49.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7fc0f6a00640 1 -- 172.21.15.67:0/962938775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4001670 con 0x7fc0f80afeb0 2024-02-26T10:38:49.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7f233ce00640 1 -- 172.21.15.67:0/964163780 shutdown_connections 2024-02-26T10:38:49.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7f233ce00640 1 --2- 172.21.15.67:0/964163780 >> v2:172.21.15.82:3300/0 conn(0x7f23380bdef0 0x7f23380c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7f233ce00640 1 --2- 172.21.15.67:0/964163780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2338157d30 0x7f2338158e10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7f233ce00640 1 -- 172.21.15.67:0/964163780 >> 172.21.15.67:0/964163780 conn(0x7f2338072980 msgr2=0x7f23380ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7f233ce00640 1 -- 172.21.15.67:0/964163780 shutdown_connections 2024-02-26T10:38:49.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.314+0000 7f233ce00640 1 -- 172.21.15.67:0/964163780 wait complete. 2024-02-26T10:38:49.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.315+0000 7f432f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 0x7f43280729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.315+0000 7fc0ff200640 1 -- 172.21.15.67:0/962938775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f80afeb0 msgr2=0x7fc0f80c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.315+0000 7fc0ff200640 1 --2- 172.21.15.67:0/962938775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f80afeb0 0x7fc0f80c71d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80b0940 tx=0x7fc0e802b980 comp rx=0 tx=0).stop 2024-02-26T10:38:49.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.315+0000 7fcee2800640 1 Processor -- start 2024-02-26T10:38:49.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f432f800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4328139ba0 con 0x7f432806ead0 2024-02-26T10:38:49.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f432f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43281512b0 con 0x7f4328070f10 2024-02-26T10:38:49.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f432f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4328151590 con 0x7f43281397c0 2024-02-26T10:38:49.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.315+0000 7fcee2800640 1 -- start start 2024-02-26T10:38:49.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f432ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 0x7f43280729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85300587f0 con 0x7f8538007210 2024-02-26T10:38:49.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f432ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 0x7f43280729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39634/0 (socket says 172.21.15.67:39634) 2024-02-26T10:38:49.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7f432ce00640 1 -- 172.21.15.67:0/2073698195 learned_addr learned my addr 172.21.15.67:0/2073698195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7fc0ff200640 1 -- 172.21.15.67:0/962938775 shutdown_connections 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7fc0ff200640 1 --2- 172.21.15.67:0/962938775 >> v2:172.21.15.82:3300/0 conn(0x7fc0f80c7710 0x7fc0f8155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7fc0ff200640 1 --2- 172.21.15.67:0/962938775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f80afeb0 0x7fc0f80c71d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.316+0000 7fc0ff200640 1 -- 172.21.15.67:0/962938775 >> 172.21.15.67:0/962938775 conn(0x7fc0f80729e0 msgr2=0x7fc0f80ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7fc0ff200640 1 -- 172.21.15.67:0/962938775 shutdown_connections 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7fc0ff200640 1 -- 172.21.15.67:0/962938775 wait complete. 2024-02-26T10:38:49.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7f233ce00640 1 Processor -- start 2024-02-26T10:38:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7f5aea000640 1 Processor -- start 2024-02-26T10:38:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.318+0000 7f233ce00640 1 -- start start 2024-02-26T10:38:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fcee2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcedc157d30 0x7fcedc158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fcee2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 0x7fcedc0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fcee2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcedc0c56e0 0x7fcedc155400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fcee2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc154f50 con 0x7fcedc157d30 2024-02-26T10:38:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fcee2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0c7440 con 0x7fcedc0c56e0 2024-02-26T10:38:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fcee2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0c7720 con 0x7fcedc0bdef0 2024-02-26T10:38:49.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.319+0000 7fc0ff200640 1 Processor -- start 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f233ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 0x7f2338153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f233ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23380c56e0 0x7f23380c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f233ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2338157d30 0x7f233814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f233ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338156330 con 0x7f2338157d30 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f233ce00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23380c7a00 con 0x7f23380c56e0 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f233ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2338155b10 con 0x7f23380bdef0 2024-02-26T10:38:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7fc0ff200640 1 -- start start 2024-02-26T10:38:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7fced2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 0x7fcedc0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7fced2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 0x7fcedc0c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39640/0 (socket says 172.21.15.67:39640) 2024-02-26T10:38:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7fced2a00640 1 -- 172.21.15.67:0/2803390589 learned_addr learned my addr 172.21.15.67:0/2803390589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7fcedb400640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.67:3300/0 conn(0x7fcedc157d30 0x7fcedc158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7f432ce00640 1 -- 172.21.15.67:0/2073698195 >> v1:172.21.15.82:6789/0 conn(0x7f4328070f10 legacy=0x7f4328154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7f432ce00640 1 -- 172.21.15.67:0/2073698195 >> v1:172.21.15.67:6789/0 conn(0x7f432806ead0 legacy=0x7f4328154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7fcedbe00640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.82:3300/0 conn(0x7fcedc0c56e0 0x7fcedc155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.320+0000 7f2335e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 0x7f2338153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.317+0000 7f432ce00640 1 -- 172.21.15.67:0/2073698195 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328151870 con 0x7f43281397c0 2024-02-26T10:38:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7f2335e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 0x7f2338153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39644/0 (socket says 172.21.15.67:39644) 2024-02-26T10:38:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7f2335e00640 1 -- 172.21.15.67:0/2414781998 learned_addr learned my addr 172.21.15.67:0/2414781998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7fced2a00640 1 -- 172.21.15.67:0/2803390589 >> v2:172.21.15.82:3300/0 conn(0x7fcedc0c56e0 msgr2=0x7fcedc155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7fced2a00640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.82:3300/0 conn(0x7fcedc0c56e0 0x7fcedc155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7fced2a00640 1 -- 172.21.15.67:0/2803390589 >> v2:172.21.15.67:3300/0 conn(0x7fcedc157d30 msgr2=0x7fcedc158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7f2336800640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2338157d30 0x7f233814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7fced2a00640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.67:3300/0 conn(0x7fcedc157d30 0x7fcedc158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f2335400640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23380c56e0 0x7f23380c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 0x7fc0f81eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f80afeb0 0x7fc0f8150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0f80c7710 0x7fc0f81544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7fc0ff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80c8910 con 0x7fc0f806da40 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7fc0ff200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80acb50 con 0x7fc0f80afeb0 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7fc0ff200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f80bde90 con 0x7fc0f80c7710 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f432ce00640 1 --2- 172.21.15.67:0/2073698195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 0x7f43280729e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f431c0098a0 tx=0x7f431c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8297b821631839a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7fced2a00640 1 -- 172.21.15.67:0/2803390589 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0ad2a0 con 0x7fcedc0bdef0 2024-02-26T10:38:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7f5aea000640 1 -- start start 2024-02-26T10:38:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.321+0000 7fcedb400640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.67:3300/0 conn(0x7fcedc157d30 0x7fcedc158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5aea000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ae4072fe0 0x7f5ae406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 0x7fc0f81eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 0x7fc0f81eb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33876/0 (socket says 172.21.15.67:33876) 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7fc0f7e00640 1 -- 172.21.15.67:0/1202459843 learned_addr learned my addr 172.21.15.67:0/1202459843 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5aea000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ae4159210 0x7f5ae4159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5aea000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae415c8a0 con 0x7f5ae4156810 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5aea000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae415cb80 con 0x7f5ae4159210 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5aea000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae415ce60 con 0x7f5ae4072fe0 2024-02-26T10:38:49.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7fc0f7400640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f80afeb0 0x7fc0f8150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7fc0fd200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0f80c7710 0x7fc0f81544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f2335e00640 1 -- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23380c56e0 msgr2=0x7f23380c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f2335e00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23380c56e0 0x7f23380c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f2335e00640 1 -- 172.21.15.67:0/2414781998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2338157d30 msgr2=0x7f233814ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f2335e00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2338157d30 0x7f233814ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.322+0000 7f2335e00640 1 -- 172.21.15.67:0/2414781998 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23381ea820 con 0x7f23380bdef0 2024-02-26T10:38:49.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5ae3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ae4159210 0x7f5ae4159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f4327400640 1 -- 172.21.15.67:0/2073698195 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0317c0 con 0x7f43281397c0 2024-02-26T10:38:49.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5ae3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ae4159210 0x7f5ae4159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33892/0 (socket says 172.21.15.67:33892) 2024-02-26T10:38:49.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f5ae3e00640 1 -- 172.21.15.67:0/3070106494 learned_addr learned my addr 172.21.15.67:0/3070106494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f4327400640 1 -- 172.21.15.67:0/2073698195 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c037070 con 0x7f43281397c0 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7f4327400640 1 -- 172.21.15.67:0/2073698195 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c031b80 con 0x7f43281397c0 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fc0f7e00640 1 -- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0f80c7710 msgr2=0x7fc0f81544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fc0f7e00640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0f80c7710 0x7fc0f81544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fc0f7e00640 1 -- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f80afeb0 msgr2=0x7fc0f8150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fc0f7e00640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f80afeb0 0x7fc0f8150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fc0f7e00640 1 -- 172.21.15.67:0/1202459843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f81eaca0 con 0x7fc0f806da40 2024-02-26T10:38:49.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fbaf1a00640 1 Processor -- start 2024-02-26T10:38:49.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f2335e00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 0x7f2338153720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f23280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.323+0000 7fced2a00640 1 --2- 172.21.15.67:0/2803390589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 0x7fcedc0c51a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ad4d0 tx=0x7fcecc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ced96013070e2bb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f432f800640 1 -- 172.21.15.67:0/2073698195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 msgr2=0x7f43280729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f432f800640 1 --2- 172.21.15.67:0/2073698195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 0x7f43280729e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f431c0098a0 tx=0x7f431c02b860 comp rx=0 tx=0).stop 2024-02-26T10:38:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fcedaa00640 1 -- 172.21.15.67:0/2803390589 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0317c0 con 0x7fcedc0bdef0 2024-02-26T10:38:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fcedaa00640 1 -- 172.21.15.67:0/2803390589 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc037070 con 0x7fcedc0bdef0 2024-02-26T10:38:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fcedaa00640 1 -- 172.21.15.67:0/2803390589 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc031b80 con 0x7fcedc0bdef0 2024-02-26T10:38:49.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328031780 con 0x7f23380bdef0 2024-02-26T10:38:49.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328031920 con 0x7f23380bdef0 2024-02-26T10:38:49.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328031cd0 con 0x7f23380bdef0 2024-02-26T10:38:49.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7f5adbe00640 1 --2- 172.21.15.67:0/3070106494 >> v2:172.21.15.82:3300/0 conn(0x7f5ae4072fe0 0x7f5ae406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f432f800640 1 -- 172.21.15.67:0/2073698195 shutdown_connections 2024-02-26T10:38:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f432f800640 1 --2- 172.21.15.67:0/2073698195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43281397c0 0x7f43280729e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f432f800640 1 -- 172.21.15.67:0/2073698195 >> 172.21.15.67:0/2073698195 conn(0x7f432806d680 msgr2=0x7f43280a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7f432f800640 1 -- 172.21.15.67:0/2073698195 shutdown_connections 2024-02-26T10:38:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7fcee2800640 1 -- 172.21.15.67:0/2803390589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 msgr2=0x7fcedc0c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7fcee2800640 1 --2- 172.21.15.67:0/2803390589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 0x7fcedc0c51a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ad4d0 tx=0x7fcecc02b860 comp rx=0 tx=0).stop 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7fc0f7e00640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 0x7fc0f81eb740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0e802bf90 tx=0x7fc0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8035030 con 0x7fc0f806da40 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8003e50 con 0x7fc0f806da40 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8035030 con 0x7fc0f806da40 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f8151480 con 0x7fc0f806da40 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23381eab00 con 0x7f23380bdef0 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23381eb0c0 con 0x7f23380bdef0 2024-02-26T10:38:49.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 -- 172.21.15.67:0/2803390589 shutdown_connections 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.82:3300/0 conn(0x7fcedc0c56e0 0x7fcedc155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 --2- 172.21.15.67:0/2803390589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc0bdef0 0x7fcedc0c51a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 --2- 172.21.15.67:0/2803390589 >> v2:172.21.15.67:3300/0 conn(0x7fcedc157d30 0x7fcedc158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 -- 172.21.15.67:0/2803390589 >> 172.21.15.67:0/2803390589 conn(0x7fcedc072980 msgr2=0x7fcedc0ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f5adbe00640 1 -- 172.21.15.67:0/3070106494 >> v2:172.21.15.67:3300/0 conn(0x7f5ae4159210 msgr2=0x7f5ae4159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f5adbe00640 1 --2- 172.21.15.67:0/3070106494 >> v2:172.21.15.67:3300/0 conn(0x7f5ae4159210 0x7f5ae4159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f5adbe00640 1 -- 172.21.15.67:0/3070106494 >> v1:172.21.15.82:6789/0 conn(0x7f5ae4156810 legacy=0x7f5ae40c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.325+0000 7f5adbe00640 1 -- 172.21.15.67:0/3070106494 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae415d140 con 0x7f5ae4072fe0 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.324+0000 7fbaf1a00640 1 -- start start 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fbaf1a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbaec156810 0x7fbaec06e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fbaf1a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbaec159210 0x7fbaec159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fbaf1a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaec15c8a0 con 0x7fbaec072fe0 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fbaf1a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec15cb80 con 0x7fbaec159210 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fbaf1a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec15ce60 con 0x7fbaec156810 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.327+0000 7f5adbe00640 1 --2- 172.21.15.67:0/3070106494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4072fe0 0x7f5ae406e660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ae406bcc0 tx=0x7f5acc02b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a6f8566159763e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.327+0000 7ff1f2c00640 1 Processor -- start 2024-02-26T10:38:49.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.327+0000 7fbaeaa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbaec156810 0x7fbaec06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.327+0000 7fbae2000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbaec072fe0 0x7fbaec0c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:48314/0 (socket says 172.21.15.67:48314) 2024-02-26T10:38:49.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.327+0000 7fbae2000640 1 -- 172.21.15.67:0/914623183 learned_addr learned my addr 172.21.15.67:0/914623183 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:49.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7ff1f2c00640 1 -- start start 2024-02-26T10:38:49.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7f432f800640 1 -- 172.21.15.67:0/2073698195 wait complete. 2024-02-26T10:38:49.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 -- 172.21.15.67:0/2803390589 shutdown_connections 2024-02-26T10:38:49.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.326+0000 7fcee2800640 1 -- 172.21.15.67:0/2803390589 wait complete. 2024-02-26T10:38:49.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.329+0000 7fcee2800640 1 Processor -- start 2024-02-26T10:38:49.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.329+0000 7fcee2800640 1 -- start start 2024-02-26T10:38:49.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7f5ae2a00640 1 -- 172.21.15.67:0/3070106494 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc037070 con 0x7f5ae4072fe0 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7f5ae2a00640 1 -- 172.21.15.67:0/3070106494 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc030710 con 0x7f5ae4072fe0 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7f5ae2a00640 1 -- 172.21.15.67:0/3070106494 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc030ae0 con 0x7f5ae4072fe0 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.329+0000 7ff1f2c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec0bdef0 0x7ff1ec158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.329+0000 7ff1ebe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff1ec157d30 0x7ff1ec0c51a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54600/0 (socket says 172.21.15.67:54600) 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.329+0000 7ff1ebe00640 1 -- 172.21.15.67:0/1322519944 learned_addr learned my addr 172.21.15.67:0/1322519944 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.329+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec154f50 con 0x7ff1ec157d30 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.330+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0c7440 con 0x7ff1ec155400 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.330+0000 7ff1eb400640 1 --2- 172.21.15.67:0/1322519944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec0bdef0 0x7ff1ec158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7fbaeb400640 1 --2- 172.21.15.67:0/914623183 >> v2:172.21.15.67:3300/0 conn(0x7fbaec159210 0x7fbaec159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7fbaeaa00640 1 -- 172.21.15.67:0/914623183 >> v2:172.21.15.67:3300/0 conn(0x7fbaec159210 msgr2=0x7fbaec159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7fbaeaa00640 1 --2- 172.21.15.67:0/914623183 >> v2:172.21.15.67:3300/0 conn(0x7fbaec159210 0x7fbaec159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7fbaeaa00640 1 -- 172.21.15.67:0/914623183 >> v1:172.21.15.82:6789/0 conn(0x7fbaec072fe0 legacy=0x7fbaec0c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.328+0000 7fbaeaa00640 1 -- 172.21.15.67:0/914623183 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec15d140 con 0x7fbaec156810 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.330+0000 7fbaeb400640 1 --2- 172.21.15.67:0/914623183 >> v2:172.21.15.67:3300/0 conn(0x7fbaec159210 0x7fbaec159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.330+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c7720 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.330+0000 7fbaeaa00640 1 --2- 172.21.15.67:0/914623183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec156810 0x7fbaec06e660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0c3060 tx=0x7fbadc02be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd3f7c94b1584e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f5aea000640 1 -- 172.21.15.67:0/3070106494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4072fe0 msgr2=0x7f5ae406e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f5aea000640 1 --2- 172.21.15.67:0/3070106494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4072fe0 0x7f5ae406e660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ae406bcc0 tx=0x7f5acc02b680 comp rx=0 tx=0).stop 2024-02-26T10:38:49.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2328032410 con 0x7f23380bdef0 2024-02-26T10:38:49.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7ff1eb400640 1 -- 172.21.15.67:0/1322519944 >> v1:172.21.15.82:6789/0 conn(0x7ff1ec155400 legacy=0x7ff1ec155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7ff1eb400640 1 -- 172.21.15.67:0/1322519944 >> v1:172.21.15.67:6789/0 conn(0x7ff1ec157d30 legacy=0x7ff1ec0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:49.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7fbaea000640 1 -- 172.21.15.67:0/914623183 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc039070 con 0x7fbaec156810 2024-02-26T10:38:49.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f5aea000640 1 -- 172.21.15.67:0/3070106494 shutdown_connections 2024-02-26T10:38:49.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f5aea000640 1 --2- 172.21.15.67:0/3070106494 >> v2:172.21.15.67:3300/0 conn(0x7f5ae4159210 0x7f5ae4159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f5aea000640 1 --2- 172.21.15.67:0/3070106494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4072fe0 0x7f5ae406e660 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7f5aea000640 1 -- 172.21.15.67:0/3070106494 >> 172.21.15.67:0/3070106494 conn(0x7f5ae40a84a0 msgr2=0x7f5ae40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7fcee2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcedc0bdef0 0x7fcedc0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7fbaea000640 1 -- 172.21.15.67:0/914623183 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc02cc20 con 0x7fbaec156810 2024-02-26T10:38:49.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7fbaea000640 1 -- 172.21.15.67:0/914623183 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc034640 con 0x7fbaec156810 2024-02-26T10:38:49.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7fcee2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc0c56e0 0x7fcedc1e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.331+0000 7ff1eb400640 1 -- 172.21.15.67:0/1322519944 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0ad2a0 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcee2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 0x7fcedc0c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcee2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc156330 con 0x7fcedc0bdef0 2024-02-26T10:38:49.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcee2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0c7a00 con 0x7fcedc0c56e0 2024-02-26T10:38:49.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcee2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc155940 con 0x7fcedc157d30 2024-02-26T10:38:49.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7ff1eb400640 1 --2- 172.21.15.67:0/1322519944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec0bdef0 0x7ff1ec158e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80097d0 tx=0x7ff1d802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442940f2d02a2a1c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7f5aea000640 1 -- 172.21.15.67:0/3070106494 shutdown_connections 2024-02-26T10:38:49.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f81eb220 con 0x7fc0f806da40 2024-02-26T10:38:49.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 0x7fcedc0c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fbaf1a00640 1 -- 172.21.15.67:0/914623183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec156810 msgr2=0x7fbaec06e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcedbe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 0x7fcedc0c18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39678/0 (socket says 172.21.15.67:39678) 2024-02-26T10:38:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fcedbe00640 1 -- 172.21.15.67:0/2089615423 learned_addr learned my addr 172.21.15.67:0/2089615423 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.332+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/914623183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec156810 0x7fbaec06e660 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0c3060 tx=0x7fbadc02be70 comp rx=0 tx=0).stop 2024-02-26T10:38:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fced2a00640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc0c56e0 0x7fcedc1e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7f5aea000640 1 -- 172.21.15.67:0/3070106494 wait complete. 2024-02-26T10:38:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7f432f800640 1 Processor -- start 2024-02-26T10:38:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fcedb400640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcedc0bdef0 0x7fcedc0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fbaf1a00640 1 -- 172.21.15.67:0/914623183 shutdown_connections 2024-02-26T10:38:49.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/914623183 >> v2:172.21.15.67:3300/0 conn(0x7fbaec159210 0x7fbaec159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/914623183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec156810 0x7fbaec06e660 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fbaf1a00640 1 -- 172.21.15.67:0/914623183 >> 172.21.15.67:0/914623183 conn(0x7fbaec0a84a0 msgr2=0x7fbaec0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7f432f800640 1 -- start start 2024-02-26T10:38:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7f232e000640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231406b500 0x7f231406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fbaf1a00640 1 -- 172.21.15.67:0/914623183 shutdown_connections 2024-02-26T10:38:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fcedbe00640 1 -- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc0c56e0 msgr2=0x7fcedc1e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fbaf1a00640 1 -- 172.21.15.67:0/914623183 wait complete. 2024-02-26T10:38:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.333+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0e8038020 con 0x7fc0f806da40 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7fcedbe00640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc0c56e0 0x7fcedc1e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7fcedbe00640 1 -- 172.21.15.67:0/2089615423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcedc0bdef0 msgr2=0x7fcedc0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7fcedbe00640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcedc0bdef0 0x7fcedc0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7fcedbe00640 1 -- 172.21.15.67:0/2089615423 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc1df5e0 con 0x7fcedc157d30 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7fcedb400640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcedc0bdef0 0x7fcedc0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23280ac420 con 0x7f23380bdef0 2024-02-26T10:38:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7ff1eaa00640 1 -- 172.21.15.67:0/1322519944 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80315a0 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7ff1eaa00640 1 -- 172.21.15.67:0/1322519944 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d8037070 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.334+0000 7ff1eaa00640 1 -- 172.21.15.67:0/1322519944 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8031960 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7fcedbe00640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 0x7fcedc0c18f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fced4007e60 tx=0x7fced400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f2335400640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231406b500 0x7f231406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f8067c90 con 0x7fc0f806da40 2024-02-26T10:38:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23381eb2c0 con 0x7f23380bdef0 2024-02-26T10:38:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f5aea000640 1 Processor -- start 2024-02-26T10:38:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f432f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f432806ead0 0x7f43280c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f432f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4328070f10 0x7f43281535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f5aea000640 1 -- start start 2024-02-26T10:38:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7f432f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 0x7f4328153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328150ef0 con 0x7f4328070f10 2024-02-26T10:38:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328151e30 con 0x7f43281397c0 2024-02-26T10:38:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4328151070 con 0x7f432806ead0 2024-02-26T10:38:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.335+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4014620 con 0x7fcedc157d30 2024-02-26T10:38:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec0bdef0 msgr2=0x7ff1ec158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1322519944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec0bdef0 0x7ff1ec158e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80097d0 tx=0x7ff1d802b840 comp rx=0 tx=0).stop 2024-02-26T10:38:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f432806ead0 0x7f43280c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4019070 con 0x7fcedc157d30 2024-02-26T10:38:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 0x7f4328153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 shutdown_connections 2024-02-26T10:38:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1322519944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec0bdef0 0x7ff1ec158e10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 >> 172.21.15.67:0/1322519944 conn(0x7ff1ec072980 msgr2=0x7ff1ec0ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 0x7f4328153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39060/0 (socket says 172.21.15.67:39060) 2024-02-26T10:38:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f432d800640 1 -- 172.21.15.67:0/503803763 learned_addr learned my addr 172.21.15.67:0/503803763 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc1de540 con 0x7fcedc157d30 2024-02-26T10:38:49.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1deb00 con 0x7fcedc157d30 2024-02-26T10:38:49.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7f4327e00640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4328070f10 0x7f43281535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4014a20 con 0x7fcedc157d30 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5aea000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae4072fe0 0x7f5ae40c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5aea000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 0x7f5ae40a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5aea000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ae4159210 0x7f5ae40a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5aea000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae406ec40 con 0x7f5ae4159210 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5aea000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae415d660 con 0x7f5ae4156810 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5aea000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4159ba0 con 0x7f5ae4072fe0 2024-02-26T10:38:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 0x7f5ae40a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 0x7f5ae40a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39062/0 (socket says 172.21.15.67:39062) 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5adbe00640 1 -- 172.21.15.67:0/2181587041 learned_addr learned my addr 172.21.15.67:0/2181587041 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.337+0000 7f5ae3400640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae4072fe0 0x7f5ae40c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f2335400640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231406b500 0x7f231406d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f233806d7c0 tx=0x7f2320008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5adbe00640 1 -- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae4072fe0 msgr2=0x7f5ae40c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5adbe00640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae4072fe0 0x7f5ae40c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5adbe00640 1 -- 172.21.15.67:0/2181587041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ae4159210 msgr2=0x7f5ae40a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5ae3e00640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ae4159210 0x7f5ae40a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5adbe00640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ae4159210 0x7f5ae40a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5adbe00640 1 -- 172.21.15.67:0/2181587041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae415a130 con 0x7f5ae4156810 2024-02-26T10:38:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb0005290 con 0x7fcedc157d30 2024-02-26T10:38:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f5ae3e00640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ae4159210 0x7f5ae40a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f432d800640 1 -- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f432806ead0 msgr2=0x7f43280c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f432d800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f432806ead0 0x7f43280c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f432d800640 1 -- 172.21.15.67:0/503803763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4328070f10 msgr2=0x7f43281535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f432d800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4328070f10 0x7f43281535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7f432d800640 1 -- 172.21.15.67:0/503803763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43281331a0 con 0x7f43281397c0 2024-02-26T10:38:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f5adbe00640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 0x7f5ae40a9910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.336+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 shutdown_connections 2024-02-26T10:38:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1322519944 wait complete. 2024-02-26T10:38:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.338+0000 7fbaf1a00640 1 Processor -- start 2024-02-26T10:38:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f432d800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 0x7f4328153b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f432000c7b0 tx=0x7f432000cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7fbaf1a00640 1 -- start start 2024-02-26T10:38:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc041020 con 0x7f5ae4156810 2024-02-26T10:38:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc03c890 con 0x7f5ae4156810 2024-02-26T10:38:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7fc0efe00640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0d806b6d0 0x7fc0d806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320004b60 con 0x7f43281397c0 2024-02-26T10:38:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc037040 con 0x7f5ae4156810 2024-02-26T10:38:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae41efd00 con 0x7f5ae4156810 2024-02-26T10:38:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.340+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0e80af170 con 0x7fc0f806da40 2024-02-26T10:38:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae40aabb0 con 0x7f5ae4156810 2024-02-26T10:38:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.340+0000 7ff1f2c00640 1 Processor -- start 2024-02-26T10:38:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.339+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43280c5410 con 0x7f43281397c0 2024-02-26T10:38:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.340+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280c36a0 con 0x7f43281397c0 2024-02-26T10:38:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.340+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320004d00 con 0x7f43281397c0 2024-02-26T10:38:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.340+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000aa70 con 0x7f43281397c0 2024-02-26T10:38:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.340+0000 7fc0f7400640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0d806b6d0 0x7fc0d806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.341+0000 7ff1f2c00640 1 -- start start 2024-02-26T10:38:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.341+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4304005290 con 0x7f43281397c0 2024-02-26T10:38:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.341+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae4074a30 con 0x7f5ae4156810 2024-02-26T10:38:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fc0f7400640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0d806b6d0 0x7fc0d806db90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8157f70 tx=0x7fc0e4000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fbaf1a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 0x7fbaec14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fbaf1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaec156810 0x7fbaec143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fbaf1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec159210 0x7fbaec152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fbaeaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 0x7fbaec14cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fbaeaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 0x7fbaec14cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:39712/0 (socket says 172.21.15.67:39712) 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7fbaeaa00640 1 -- 172.21.15.67:0/690863992 learned_addr learned my addr 172.21.15.67:0/690863992 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7ff1f2c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 0x7ff1ec0c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.342+0000 7ff1f2c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec155400 0x7ff1ec1ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1f2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 0x7ff1ec1ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1f2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec156330 con 0x7ff1ec157d30 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1f2c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c7a00 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1f2c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c58b0 con 0x7ff1ec155400 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1f0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 0x7ff1ec1ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 0x7ff1ec0c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbae2000640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaec156810 0x7fbaec143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec06ec40 con 0x7fbaec156810 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaeb400640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec159210 0x7fbaec152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec15d660 con 0x7fbaec159210 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec159ba0 con 0x7fbaec072fe0 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaeaa00640 1 -- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec159210 msgr2=0x7fbaec152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaeaa00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec159210 0x7fbaec152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaeaa00640 1 -- 172.21.15.67:0/690863992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaec156810 msgr2=0x7fbaec143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaeaa00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaec156810 0x7fbaec143e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7fbaeaa00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec153410 con 0x7fbaec072fe0 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.344+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5acc044020 con 0x7f5ae4156810 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1ebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 0x7ff1ec0c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39078/0 (socket says 172.21.15.67:39078) 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.344+0000 7ff1ebe00640 1 -- 172.21.15.67:0/1802649493 learned_addr learned my addr 172.21.15.67:0/1802649493 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.343+0000 7ff1f0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 0x7ff1ec1ef980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33940/0 (socket says 172.21.15.67:33940) 2024-02-26T10:38:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.344+0000 7ff1eb400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec155400 0x7ff1ec1ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.344+0000 7fbaeaa00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 0x7fbaec14cd20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbadc037040 tx=0x7fbadc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1ebe00640 1 -- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec155400 msgr2=0x7ff1ec1ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1ebe00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec155400 0x7ff1ec1ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1ebe00640 1 -- 172.21.15.67:0/1802649493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 msgr2=0x7ff1ec1ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1ebe00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 0x7ff1ec1ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1ebe00640 1 -- 172.21.15.67:0/1802649493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0af400 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1eb400640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec155400 0x7ff1ec1ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1f0c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 0x7ff1ec1ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f432000ac10 con 0x7f43281397c0 2024-02-26T10:38:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7ff1ebe00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 0x7ff1ec0c32b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00b460 tx=0x7ff1dc00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7f5adb400640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ac006b4d0 0x7f5ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.345+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5acc0ac010 con 0x7f5ae4156810 2024-02-26T10:38:49.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.346+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0f8067c90 con 0x7fc0f806da40 2024-02-26T10:38:49.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.346+0000 7f5ae3400640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ac006b4d0 0x7f5ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.346+0000 7f4324c00640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f430806b500 0x7f430806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.346+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f432008aae0 con 0x7f43281397c0 2024-02-26T10:38:49.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.347+0000 7f432ce00640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f430806b500 0x7f430806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.347+0000 7f5ae3400640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ac006b4d0 0x7f5ac006d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4001860 tx=0x7f5ad4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.348+0000 7f432ce00640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f430806b500 0x7f430806d9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f431c036000 tx=0x7f431c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.353+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f232802f050 con 0x7f23380bdef0 2024-02-26T10:38:49.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.353+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc039050 con 0x7fbaec072fe0 2024-02-26T10:38:49.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.353+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec1522b0 con 0x7fbaec072fe0 2024-02-26T10:38:49.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.354+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec14c800 con 0x7fbaec072fe0 2024-02-26T10:38:49.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.354+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc03a040 con 0x7fbaec072fe0 2024-02-26T10:38:49.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.354+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc040450 con 0x7fbaec072fe0 2024-02-26T10:38:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.355+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac4005290 con 0x7fbaec072fe0 2024-02-26T10:38:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.355+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fced4014bc0 con 0x7fcedc157d30 2024-02-26T10:38:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.356+0000 7fced3e00640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb806b4d0 0x7fceb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.356+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fced408b910 con 0x7fcedc157d30 2024-02-26T10:38:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.357+0000 7fcedb400640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb806b4d0 0x7fceb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.358+0000 7fcedb400640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb806b4d0 0x7fceb806d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ac370 tx=0x7fcec4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.360+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5acc04b030 con 0x7f5ae4156810 2024-02-26T10:38:49.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.368+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fced405a800 con 0x7fcedc157d30 2024-02-26T10:38:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.370+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbadc047020 con 0x7fbaec072fe0 2024-02-26T10:38:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.371+0000 7fbae3400640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbac806b4d0 0x7fbac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.371+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbadc0ac6f0 con 0x7fbaec072fe0 2024-02-26T10:38:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.372+0000 7fbae2000640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbac806b4d0 0x7fbac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4320059950 con 0x7f43281397c0 2024-02-26T10:38:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc015020 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0118b0 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7fbae2000640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbac806b4d0 0x7fbac806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbaec147be0 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0cb290 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc011ca0 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.375+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0c8470 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.379+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec074a30 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.379+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1dc018020 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.380+0000 7ff1e3e00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff1cc06b520 0x7ff1cc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.381+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbadc07af90 con 0x7fbaec072fe0 2024-02-26T10:38:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.381+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1dc08a110 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.381+0000 7ff1eb400640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff1cc06b520 0x7ff1cc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.383+0000 7ff1eb400640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff1cc06b520 0x7ff1cc06d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8036000 tx=0x7ff1d803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.394+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1dc01f020 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.689+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8518005580 con 0x7f8538007210 2024-02-26T10:38:49.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.690+0000 7f8536000640 1 -- 172.21.15.67:0/4224462326 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f853005c890 con 0x7f8538007210 2024-02-26T10:38:49.692 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-02-26T10:38:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.693+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852006b5a0 msgr2=0x7f852006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.693+0000 7f8544e00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852006b5a0 0x7f852006da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85400736f0 tx=0x7f85400ab000 comp rx=0 tx=0).stop 2024-02-26T10:38:49.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.693+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 msgr2=0x7f85380c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.693+0000 7f8544e00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 0x7f85380c7550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f853000c820 tx=0x7f853000cce0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 shutdown_connections 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f853800f880 0x7f85380dd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f853800e740 0x7f85380e21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852006b5a0 0x7f852006da60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 --2- 172.21.15.67:0/4224462326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8538007210 0x7f85380c7550 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 >> 172.21.15.67:0/4224462326 conn(0x7f853804c670 msgr2=0x7f85380e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.694+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 shutdown_connections 2024-02-26T10:38:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.695+0000 7f8544e00640 1 -- 172.21.15.67:0/4224462326 wait complete. 2024-02-26T10:38:49.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-02-26T10:38:49.722 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.761+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff1ec0c6230 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.762+0000 7ff1e3e00640 1 -- 172.21.15.67:0/1802649493 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1dc00d040 con 0x7ff1ec0bdef0 2024-02-26T10:38:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:60129542150 2024-02-26T10:38:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff1cc06b520 msgr2=0x7ff1cc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff1cc06b520 0x7ff1cc06d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8036000 tx=0x7ff1d803d040 comp rx=0 tx=0).stop 2024-02-26T10:38:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 msgr2=0x7ff1ec0c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 0x7ff1ec0c32b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00b460 tx=0x7ff1dc00b920 comp rx=0 tx=0).stop 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 shutdown_connections 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ec157d30 0x7ff1ec1ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1ec155400 0x7ff1ec1ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff1cc06b520 0x7ff1cc06d9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 --2- 172.21.15.67:0/1802649493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ec0bdef0 0x7ff1ec0c32b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 >> 172.21.15.67:0/1802649493 conn(0x7ff1ec072980 msgr2=0x7ff1ec154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 shutdown_connections 2024-02-26T10:38:49.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.764+0000 7ff1f2c00640 1 -- 172.21.15.67:0/1802649493 wait complete. 2024-02-26T10:38:49.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-02-26T10:38:49.783 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.787+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc0f8074a30 con 0x7fc0f806da40 2024-02-26T10:38:49.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.788+0000 7fc0efe00640 1 -- 172.21.15.67:0/1202459843 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc0e8079800 con 0x7fc0f806da40 2024-02-26T10:38:49.791 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:49.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.789+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5ae40717d0 con 0x7f5ae4156810 2024-02-26T10:38:49.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.790+0000 7f5adb400640 1 -- 172.21.15.67:0/2181587041 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ae415a130 con 0x7f5ae4156810 2024-02-26T10:38:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.790+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0d806b6d0 msgr2=0x7fc0d806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.790+0000 7fc0ff200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0d806b6d0 0x7fc0d806db90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8157f70 tx=0x7fc0e4000ed0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 msgr2=0x7fc0f81eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 0x7fc0f81eb740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0e802bf90 tx=0x7fc0e80046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 shutdown_connections 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0f80c7710 0x7fc0f81544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0d806b6d0 0x7fc0d806db90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f80afeb0 0x7fc0f8150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 --2- 172.21.15.67:0/1202459843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0f806da40 0x7fc0f81eb740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 >> 172.21.15.67:0/1202459843 conn(0x7fc0f80729e0 msgr2=0x7fc0f81561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 shutdown_connections 2024-02-26T10:38:49.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.791+0000 7fc0ff200640 1 -- 172.21.15.67:0/1202459843 wait complete. 2024-02-26T10:38:49.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.793+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ac006b4d0 msgr2=0x7f5ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.793+0000 7f5aea000640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ac006b4d0 0x7f5ac006d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4001860 tx=0x7f5ad4009040 comp rx=0 tx=0).stop 2024-02-26T10:38:49.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.793+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 msgr2=0x7f5ae40a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.793+0000 7f5aea000640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 0x7f5ae40a9910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc0046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f23380acef0 con 0x7f23380bdef0 2024-02-26T10:38:49.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 shutdown_connections 2024-02-26T10:38:49.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ae4159210 0x7f5ae40a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ac006b4d0 0x7f5ac006d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae4156810 0x7f5ae40a9910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 --2- 172.21.15.67:0/2181587041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae4072fe0 0x7f5ae40c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 >> 172.21.15.67:0/2181587041 conn(0x7f5ae40a84a0 msgr2=0x7f5ae40c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 shutdown_connections 2024-02-26T10:38:49.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f5aea000640 1 -- 172.21.15.67:0/2181587041 wait complete. 2024-02-26T10:38:49.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.794+0000 7f232e000640 1 -- 172.21.15.67:0/2414781998 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f232807ac40 con 0x7f23380bdef0 2024-02-26T10:38:49.797 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.797+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231406b500 msgr2=0x7f231406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231406b500 0x7f231406d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f233806d7c0 tx=0x7f2320008040 comp rx=0 tx=0).stop 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 msgr2=0x7f2338153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 0x7f2338153720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f23280046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 shutdown_connections 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2338157d30 0x7f233814ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231406b500 0x7f231406d9c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23380c56e0 0x7f23380c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 --2- 172.21.15.67:0/2414781998 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23380bdef0 0x7f2338153720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 >> 172.21.15.67:0/2414781998 conn(0x7f2338072980 msgr2=0x7f233806c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 shutdown_connections 2024-02-26T10:38:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.798+0000 7f233ce00640 1 -- 172.21.15.67:0/2414781998 wait complete. 2024-02-26T10:38:49.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.805+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4304005850 con 0x7f43281397c0 2024-02-26T10:38:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.807+0000 7f4324c00640 1 -- 172.21.15.67:0/503803763 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4320059250 con 0x7f43281397c0 2024-02-26T10:38:49.809 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f430806b500 msgr2=0x7f430806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f430806b500 0x7f430806d9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f431c036000 tx=0x7f431c03d040 comp rx=0 tx=0).stop 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 msgr2=0x7f4328153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 0x7f4328153b10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f432000c7b0 tx=0x7f432000cc70 comp rx=0 tx=0).stop 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 shutdown_connections 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f430806b500 0x7f430806d9c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43281397c0 0x7f4328153b10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4328070f10 0x7f43281535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 --2- 172.21.15.67:0/503803763 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f432806ead0 0x7f43280c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.810+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 >> 172.21.15.67:0/503803763 conn(0x7f432806d680 msgr2=0x7f43280c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.811+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 shutdown_connections 2024-02-26T10:38:49.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.811+0000 7f432f800640 1 -- 172.21.15.67:0/503803763 wait complete. 2024-02-26T10:38:49.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.813+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fceb0005850 con 0x7fcedc157d30 2024-02-26T10:38:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.814+0000 7fced3e00640 1 -- 172.21.15.67:0/2089615423 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fced405a100 con 0x7fcedc157d30 2024-02-26T10:38:49.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2024-02-26T10:38:49.817 INFO:teuthology.orchestra.run.smithi067.stdout:60129542151 2024-02-26T10:38:49.817 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-02-26T10:38:49.818 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb806b4d0 msgr2=0x7fceb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb806b4d0 0x7fceb806d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0ac370 tx=0x7fcec4008040 comp rx=0 tx=0).stop 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 msgr2=0x7fcedc0c18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 0x7fcedc0c18f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fced4007e60 tx=0x7fced400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 shutdown_connections 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcedc157d30 0x7fcedc0c18f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb806b4d0 0x7fceb806d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcedc0c56e0 0x7fcedc1e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 --2- 172.21.15.67:0/2089615423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcedc0bdef0 0x7fcedc0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 >> 172.21.15.67:0/2089615423 conn(0x7fcedc072980 msgr2=0x7fcedc06cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 shutdown_connections 2024-02-26T10:38:49.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.817+0000 7fcee2800640 1 -- 172.21.15.67:0/2089615423 wait complete. 2024-02-26T10:38:49.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.818+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbac4005850 con 0x7fbaec072fe0 2024-02-26T10:38:49.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.819+0000 7fbae3400640 1 -- 172.21.15.67:0/690863992 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbadc07f030 con 0x7fbaec072fe0 2024-02-26T10:38:49.821 INFO:teuthology.orchestra.run.smithi067.stdout:60129542149 2024-02-26T10:38:49.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2024-02-26T10:38:49.823 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbac806b4d0 msgr2=0x7fbac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbac806b4d0 0x7fbac806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbaec147be0 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 msgr2=0x7fbaec14cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 0x7fbaec14cd20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbadc037040 tx=0x7fbadc004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 shutdown_connections 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbac806b4d0 0x7fbac806d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec159210 0x7fbaec152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaec156810 0x7fbaec143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 --2- 172.21.15.67:0/690863992 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbaec072fe0 0x7fbaec14cd20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 >> 172.21.15.67:0/690863992 conn(0x7fbaec0a84a0 msgr2=0x7fbaec155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 shutdown_connections 2024-02-26T10:38:49.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:49.823+0000 7fbaf1a00640 1 -- 172.21.15.67:0/690863992 wait complete. 2024-02-26T10:38:49.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2024-02-26T10:38:49.833 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.2 2024-02-26T10:38:49.839 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-02-26T10:38:49.844 DEBUG:teuthology.parallel:result is None 2024-02-26T10:38:49.844 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-26T10:38:49.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.102+0000 7f560a600640 1 Processor -- start 2024-02-26T10:38:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.102+0000 7f560a600640 1 -- start start 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.103+0000 7f560a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f560406ac20 0x7f56040a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.103+0000 7f560a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406b000 0x7f560406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.103+0000 7f560a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560406b460 con 0x7f5604153610 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.103+0000 7f560a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56040a9a80 con 0x7f560406ac20 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.103+0000 7f560a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56040a9d60 con 0x7f560406b000 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f560406ac20 0x7f56040a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5604153610 0x7f56040abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56520/0 (socket says 172.21.15.67:56520) 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609c00640 1 -- 172.21.15.67:0/232262723 learned_addr learned my addr 172.21.15.67:0/232262723 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5603e00640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406b000 0x7f560406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609200640 1 -- 172.21.15.67:0/232262723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406b000 msgr2=0x7f560406db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609200640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406b000 0x7f560406db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609200640 1 -- 172.21.15.67:0/232262723 >> v1:172.21.15.67:6789/0 conn(0x7f5604153610 legacy=0x7f56040abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5609200640 1 -- 172.21.15.67:0/232262723 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56040ab750 con 0x7f560406ac20 2024-02-26T10:38:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.104+0000 7f5603e00640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406b000 0x7f560406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.105+0000 7f5609200640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f560406ac20 0x7f56040a94a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55f80097a0 tx=0x7f55f802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7debd92c17ded8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.105+0000 7f5603400640 1 -- 172.21.15.67:0/232262723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8035020 con 0x7f560406ac20 2024-02-26T10:38:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f5603400640 1 -- 172.21.15.67:0/232262723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f8031810 con 0x7f560406ac20 2024-02-26T10:38:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f5603400640 1 -- 172.21.15.67:0/232262723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8031be0 con 0x7f560406ac20 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 -- 172.21.15.67:0/232262723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f560406ac20 msgr2=0x7f56040a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f560406ac20 0x7f56040a94a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55f80097a0 tx=0x7f55f802b8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 -- 172.21.15.67:0/232262723 shutdown_connections 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406b000 0x7f560406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 --2- 172.21.15.67:0/232262723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f560406ac20 0x7f56040a94a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 -- 172.21.15.67:0/232262723 >> 172.21.15.67:0/232262723 conn(0x7f56040c63f0 msgr2=0x7f56040c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 -- 172.21.15.67:0/232262723 shutdown_connections 2024-02-26T10:38:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.106+0000 7f560a600640 1 -- 172.21.15.67:0/232262723 wait complete. 2024-02-26T10:38:50.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.108+0000 7f560a600640 1 Processor -- start 2024-02-26T10:38:50.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.108+0000 7f560a600640 1 -- start start 2024-02-26T10:38:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f560a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406ac20 0x7f56040c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f560a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560406b000 0x7f560414c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f560a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 0x7f560414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f560a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5604150410 con 0x7f5604153610 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f560a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5604153aa0 con 0x7f560406b000 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f560a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5604150590 con 0x7f560406ac20 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f5609c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 0x7f560414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f5609c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 0x7f560414c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44368/0 (socket says 172.21.15.67:44368) 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f5609c00640 1 -- 172.21.15.67:0/2622084887 learned_addr learned my addr 172.21.15.67:0/2622084887 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f5609200640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406ac20 0x7f56040c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.109+0000 7f5603e00640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560406b000 0x7f560414c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5609c00640 1 -- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406ac20 msgr2=0x7f56040c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5609c00640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406ac20 0x7f56040c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5609c00640 1 -- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560406b000 msgr2=0x7f560414c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5609c00640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560406b000 0x7f560414c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5609c00640 1 -- 172.21.15.67:0/2622084887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56041ee5b0 con 0x7f5604153610 2024-02-26T10:38:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5603e00640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560406b000 0x7f560414c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5609c00640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 0x7f560414c560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00ca80 tx=0x7f55fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc017070 con 0x7f5604153610 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc00ac30 con 0x7f5604153610 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc012600 con 0x7f5604153610 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56041ee890 con 0x7f5604153610 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.110+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56041eee50 con 0x7f5604153610 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.111+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56040c0830 con 0x7f5604153610 2024-02-26T10:38:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.112+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55fc01a070 con 0x7f5604153610 2024-02-26T10:38:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.114+0000 7f5600c00640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f55e406b4d0 0x7f55e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.114+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55fc089e50 con 0x7f5604153610 2024-02-26T10:38:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.114+0000 7f5609200640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f55e406b4d0 0x7f55e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.115+0000 7f5609200640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f55e406b4d0 0x7f55e406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55f8036000 tx=0x7f55f803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.118+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56040c0830 con 0x7f5604153610 2024-02-26T10:38:50.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.309+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56040a94a0 con 0x7f55e406b4d0 2024-02-26T10:38:50.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.328+0000 7f5600c00640 1 -- 172.21.15.67:0/2622084887 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251839 (secure 0 0 0) 0x7f55f80867d0 con 0x7f55e406b4d0 2024-02-26T10:38:50.334 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f55e406b4d0 msgr2=0x7f55e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f55e406b4d0 0x7f55e406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55f8036000 tx=0x7f55f803c040 comp rx=0 tx=0).stop 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 msgr2=0x7f560414c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 0x7f560414c560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00ca80 tx=0x7f55fc00cf40 comp rx=0 tx=0).stop 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 shutdown_connections 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5604153610 0x7f560414c560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f55e406b4d0 0x7f55e406d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560406b000 0x7f560414c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 --2- 172.21.15.67:0/2622084887 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f560406ac20 0x7f56040c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.334+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 >> 172.21.15.67:0/2622084887 conn(0x7f56040c63f0 msgr2=0x7f56040c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:50.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.335+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 shutdown_connections 2024-02-26T10:38:50.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.335+0000 7f560a600640 1 -- 172.21.15.67:0/2622084887 wait complete. 2024-02-26T10:38:50.355 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-02-26T10:38:48.324846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219204,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755516,"statfs":{"total":773094113280,"available":772869648384,"internally_reserved":0,"allocated":2957312,"data_stored":1779452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004650"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237702+0000","last_change":"2024-02-26T10:38:35.196643+0000","last_active":"2024-02-26T10:38:37.237702+0000","last_peered":"2024-02-26T10:38:37.237702+0000","last_clean":"2024-02-26T10:38:37.237702+0000","last_became_active":"2024-02-26T10:38:35.195966+0000","last_became_peered":"2024-02-26T10:38:35.195966+0000","last_unstale":"2024-02-26T10:38:37.237702+0000","last_undegraded":"2024-02-26T10:38:37.237702+0000","last_fullsized":"2024-02-26T10:38:37.237702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174389+0000","last_change":"2024-02-26T10:38:35.189849+0000","last_active":"2024-02-26T10:38:36.174389+0000","last_peered":"2024-02-26T10:38:36.174389+0000","last_clean":"2024-02-26T10:38:36.174389+0000","last_became_active":"2024-02-26T10:38:35.189290+0000","last_became_peered":"2024-02-26T10:38:35.189290+0000","last_unstale":"2024-02-26T10:38:36.174389+0000","last_undegraded":"2024-02-26T10:38:36.174389+0000","last_fullsized":"2024-02-26T10:38:36.174389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178614+0000","last_change":"2024-02-26T10:38:35.184041+0000","last_active":"2024-02-26T10:38:36.178614+0000","last_peered":"2024-02-26T10:38:36.178614+0000","last_clean":"2024-02-26T10:38:36.178614+0000","last_became_active":"2024-02-26T10:38:35.181755+0000","last_became_peered":"2024-02-26T10:38:35.181755+0000","last_unstale":"2024-02-26T10:38:36.178614+0000","last_undegraded":"2024-02-26T10:38:36.178614+0000","last_fullsized":"2024-02-26T10:38:36.178614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235190+0000","last_change":"2024-02-26T10:38:35.199057+0000","last_active":"2024-02-26T10:38:37.235190+0000","last_peered":"2024-02-26T10:38:37.235190+0000","last_clean":"2024-02-26T10:38:37.235190+0000","last_became_active":"2024-02-26T10:38:35.198578+0000","last_became_peered":"2024-02-26T10:38:35.198578+0000","last_unstale":"2024-02-26T10:38:37.235190+0000","last_undegraded":"2024-02-26T10:38:37.235190+0000","last_fullsized":"2024-02-26T10:38:37.235190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235490+0000","last_change":"2024-02-26T10:38:35.188780+0000","last_active":"2024-02-26T10:38:37.235490+0000","last_peered":"2024-02-26T10:38:37.235490+0000","last_clean":"2024-02-26T10:38:37.235490+0000","last_became_active":"2024-02-26T10:38:35.187459+0000","last_became_peered":"2024-02-26T10:38:35.187459+0000","last_unstale":"2024-02-26T10:38:37.235490+0000","last_undegraded":"2024-02-26T10:38:37.235490+0000","last_fullsized":"2024-02-26T10:38:37.235490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235076+0000","last_change":"2024-02-26T10:38:35.204704+0000","last_active":"2024-02-26T10:38:37.235076+0000","last_peered":"2024-02-26T10:38:37.235076+0000","last_clean":"2024-02-26T10:38:37.235076+0000","last_became_active":"2024-02-26T10:38:35.204329+0000","last_became_peered":"2024-02-26T10:38:35.204329+0000","last_unstale":"2024-02-26T10:38:37.235076+0000","last_undegraded":"2024-02-26T10:38:37.235076+0000","last_fullsized":"2024-02-26T10:38:37.235076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237988+0000","last_change":"2024-02-26T10:38:35.193948+0000","last_active":"2024-02-26T10:38:37.237988+0000","last_peered":"2024-02-26T10:38:37.237988+0000","last_clean":"2024-02-26T10:38:37.237988+0000","last_became_active":"2024-02-26T10:38:35.192130+0000","last_became_peered":"2024-02-26T10:38:35.192130+0000","last_unstale":"2024-02-26T10:38:37.237988+0000","last_undegraded":"2024-02-26T10:38:37.237988+0000","last_fullsized":"2024-02-26T10:38:37.237988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:56:35.875738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234899+0000","last_change":"2024-02-26T10:38:35.205622+0000","last_active":"2024-02-26T10:38:37.234899+0000","last_peered":"2024-02-26T10:38:37.234899+0000","last_clean":"2024-02-26T10:38:37.234899+0000","last_became_active":"2024-02-26T10:38:35.205359+0000","last_became_peered":"2024-02-26T10:38:35.205359+0000","last_unstale":"2024-02-26T10:38:37.234899+0000","last_undegraded":"2024-02-26T10:38:37.234899+0000","last_fullsized":"2024-02-26T10:38:37.234899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237849+0000","last_change":"2024-02-26T10:38:35.202549+0000","last_active":"2024-02-26T10:38:37.237849+0000","last_peered":"2024-02-26T10:38:37.237849+0000","last_clean":"2024-02-26T10:38:37.237849+0000","last_became_active":"2024-02-26T10:38:35.201568+0000","last_became_peered":"2024-02-26T10:38:35.201568+0000","last_unstale":"2024-02-26T10:38:37.237849+0000","last_undegraded":"2024-02-26T10:38:37.237849+0000","last_fullsized":"2024-02-26T10:38:37.237849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182530+0000","last_change":"2024-02-26T10:38:35.206569+0000","last_active":"2024-02-26T10:38:36.182530+0000","last_peered":"2024-02-26T10:38:36.182530+0000","last_clean":"2024-02-26T10:38:36.182530+0000","last_became_active":"2024-02-26T10:38:35.206285+0000","last_became_peered":"2024-02-26T10:38:35.206285+0000","last_unstale":"2024-02-26T10:38:36.182530+0000","last_undegraded":"2024-02-26T10:38:36.182530+0000","last_fullsized":"2024-02-26T10:38:36.182530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182446+0000","last_change":"2024-02-26T10:38:35.209104+0000","last_active":"2024-02-26T10:38:36.182446+0000","last_peered":"2024-02-26T10:38:36.182446+0000","last_clean":"2024-02-26T10:38:36.182446+0000","last_became_active":"2024-02-26T10:38:35.208841+0000","last_became_peered":"2024-02-26T10:38:35.208841+0000","last_unstale":"2024-02-26T10:38:36.182446+0000","last_undegraded":"2024-02-26T10:38:36.182446+0000","last_fullsized":"2024-02-26T10:38:36.182446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175483+0000","last_change":"2024-02-26T10:38:35.198390+0000","last_active":"2024-02-26T10:38:36.175483+0000","last_peered":"2024-02-26T10:38:36.175483+0000","last_clean":"2024-02-26T10:38:36.175483+0000","last_became_active":"2024-02-26T10:38:35.197895+0000","last_became_peered":"2024-02-26T10:38:35.197895+0000","last_unstale":"2024-02-26T10:38:36.175483+0000","last_undegraded":"2024-02-26T10:38:36.175483+0000","last_fullsized":"2024-02-26T10:38:36.175483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:01.842347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.192330+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.192046+0000","last_became_peered":"2024-02-26T10:38:35.192046+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177330+0000","last_change":"2024-02-26T10:38:35.181873+0000","last_active":"2024-02-26T10:38:36.177330+0000","last_peered":"2024-02-26T10:38:36.177330+0000","last_clean":"2024-02-26T10:38:36.177330+0000","last_became_active":"2024-02-26T10:38:35.180543+0000","last_became_peered":"2024-02-26T10:38:35.180543+0000","last_unstale":"2024-02-26T10:38:36.177330+0000","last_undegraded":"2024-02-26T10:38:36.177330+0000","last_fullsized":"2024-02-26T10:38:36.177330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:04:47.351870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173628+0000","last_change":"2024-02-26T10:38:35.186385+0000","last_active":"2024-02-26T10:38:36.173628+0000","last_peered":"2024-02-26T10:38:36.173628+0000","last_clean":"2024-02-26T10:38:36.173628+0000","last_became_active":"2024-02-26T10:38:35.186103+0000","last_became_peered":"2024-02-26T10:38:35.186103+0000","last_unstale":"2024-02-26T10:38:36.173628+0000","last_undegraded":"2024-02-26T10:38:36.173628+0000","last_fullsized":"2024-02-26T10:38:36.173628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234734+0000","last_change":"2024-02-26T10:38:35.203847+0000","last_active":"2024-02-26T10:38:37.234734+0000","last_peered":"2024-02-26T10:38:37.234734+0000","last_clean":"2024-02-26T10:38:37.234734+0000","last_became_active":"2024-02-26T10:38:35.201819+0000","last_became_peered":"2024-02-26T10:38:35.201819+0000","last_unstale":"2024-02-26T10:38:37.234734+0000","last_undegraded":"2024-02-26T10:38:37.234734+0000","last_fullsized":"2024-02-26T10:38:37.234734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237593+0000","last_change":"2024-02-26T10:38:35.205600+0000","last_active":"2024-02-26T10:38:37.237593+0000","last_peered":"2024-02-26T10:38:37.237593+0000","last_clean":"2024-02-26T10:38:37.237593+0000","last_became_active":"2024-02-26T10:38:35.205272+0000","last_became_peered":"2024-02-26T10:38:35.205272+0000","last_unstale":"2024-02-26T10:38:37.237593+0000","last_undegraded":"2024-02-26T10:38:37.237593+0000","last_fullsized":"2024-02-26T10:38:37.237593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237554+0000","last_change":"2024-02-26T10:38:35.191539+0000","last_active":"2024-02-26T10:38:37.237554+0000","last_peered":"2024-02-26T10:38:37.237554+0000","last_clean":"2024-02-26T10:38:37.237554+0000","last_became_active":"2024-02-26T10:38:35.191258+0000","last_became_peered":"2024-02-26T10:38:35.191258+0000","last_unstale":"2024-02-26T10:38:37.237554+0000","last_undegraded":"2024-02-26T10:38:37.237554+0000","last_fullsized":"2024-02-26T10:38:37.237554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182184+0000","last_change":"2024-02-26T10:38:35.197155+0000","last_active":"2024-02-26T10:38:36.182184+0000","last_peered":"2024-02-26T10:38:36.182184+0000","last_clean":"2024-02-26T10:38:36.182184+0000","last_became_active":"2024-02-26T10:38:35.192752+0000","last_became_peered":"2024-02-26T10:38:35.192752+0000","last_unstale":"2024-02-26T10:38:36.182184+0000","last_undegraded":"2024-02-26T10:38:36.182184+0000","last_fullsized":"2024-02-26T10:38:36.182184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178148+0000","last_change":"2024-02-26T10:38:35.192588+0000","last_active":"2024-02-26T10:38:36.178148+0000","last_peered":"2024-02-26T10:38:36.178148+0000","last_clean":"2024-02-26T10:38:36.178148+0000","last_became_active":"2024-02-26T10:38:35.192244+0000","last_became_peered":"2024-02-26T10:38:35.192244+0000","last_unstale":"2024-02-26T10:38:36.178148+0000","last_undegraded":"2024-02-26T10:38:36.178148+0000","last_fullsized":"2024-02-26T10:38:36.178148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235257+0000","last_change":"2024-02-26T10:38:35.193557+0000","last_active":"2024-02-26T10:38:37.235257+0000","last_peered":"2024-02-26T10:38:37.235257+0000","last_clean":"2024-02-26T10:38:37.235257+0000","last_became_active":"2024-02-26T10:38:35.192814+0000","last_became_peered":"2024-02-26T10:38:35.192814+0000","last_unstale":"2024-02-26T10:38:37.235257+0000","last_undegraded":"2024-02-26T10:38:37.235257+0000","last_fullsized":"2024-02-26T10:38:37.235257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235534+0000","last_change":"2024-02-26T10:38:35.188741+0000","last_active":"2024-02-26T10:38:37.235534+0000","last_peered":"2024-02-26T10:38:37.235534+0000","last_clean":"2024-02-26T10:38:37.235534+0000","last_became_active":"2024-02-26T10:38:35.188296+0000","last_became_peered":"2024-02-26T10:38:35.188296+0000","last_unstale":"2024-02-26T10:38:37.235534+0000","last_undegraded":"2024-02-26T10:38:37.235534+0000","last_fullsized":"2024-02-26T10:38:37.235534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235999+0000","last_change":"2024-02-26T10:38:35.193665+0000","last_active":"2024-02-26T10:38:37.235999+0000","last_peered":"2024-02-26T10:38:37.235999+0000","last_clean":"2024-02-26T10:38:37.235999+0000","last_became_active":"2024-02-26T10:38:35.193314+0000","last_became_peered":"2024-02-26T10:38:35.193314+0000","last_unstale":"2024-02-26T10:38:37.235999+0000","last_undegraded":"2024-02-26T10:38:37.235999+0000","last_fullsized":"2024-02-26T10:38:37.235999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236084+0000","last_change":"2024-02-26T10:38:35.186570+0000","last_active":"2024-02-26T10:38:37.236084+0000","last_peered":"2024-02-26T10:38:37.236084+0000","last_clean":"2024-02-26T10:38:37.236084+0000","last_became_active":"2024-02-26T10:38:35.186248+0000","last_became_peered":"2024-02-26T10:38:35.186248+0000","last_unstale":"2024-02-26T10:38:37.236084+0000","last_undegraded":"2024-02-26T10:38:37.236084+0000","last_fullsized":"2024-02-26T10:38:37.236084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182707+0000","last_change":"2024-02-26T10:38:35.197545+0000","last_active":"2024-02-26T10:38:36.182707+0000","last_peered":"2024-02-26T10:38:36.182707+0000","last_clean":"2024-02-26T10:38:36.182707+0000","last_became_active":"2024-02-26T10:38:35.197220+0000","last_became_peered":"2024-02-26T10:38:35.197220+0000","last_unstale":"2024-02-26T10:38:36.182707+0000","last_undegraded":"2024-02-26T10:38:36.182707+0000","last_fullsized":"2024-02-26T10:38:36.182707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173665+0000","last_change":"2024-02-26T10:38:35.174943+0000","last_active":"2024-02-26T10:38:36.173665+0000","last_peered":"2024-02-26T10:38:36.173665+0000","last_clean":"2024-02-26T10:38:36.173665+0000","last_became_active":"2024-02-26T10:38:35.174618+0000","last_became_peered":"2024-02-26T10:38:35.174618+0000","last_unstale":"2024-02-26T10:38:36.173665+0000","last_undegraded":"2024-02-26T10:38:36.173665+0000","last_fullsized":"2024-02-26T10:38:36.173665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235860+0000","last_change":"2024-02-26T10:38:35.188806+0000","last_active":"2024-02-26T10:38:37.235860+0000","last_peered":"2024-02-26T10:38:37.235860+0000","last_clean":"2024-02-26T10:38:37.235860+0000","last_became_active":"2024-02-26T10:38:35.188539+0000","last_became_peered":"2024-02-26T10:38:35.188539+0000","last_unstale":"2024-02-26T10:38:37.235860+0000","last_undegraded":"2024-02-26T10:38:37.235860+0000","last_fullsized":"2024-02-26T10:38:37.235860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178458+0000","last_change":"2024-02-26T10:38:35.183491+0000","last_active":"2024-02-26T10:38:36.178458+0000","last_peered":"2024-02-26T10:38:36.178458+0000","last_clean":"2024-02-26T10:38:36.178458+0000","last_became_active":"2024-02-26T10:38:35.183153+0000","last_became_peered":"2024-02-26T10:38:35.183153+0000","last_unstale":"2024-02-26T10:38:36.178458+0000","last_undegraded":"2024-02-26T10:38:36.178458+0000","last_fullsized":"2024-02-26T10:38:36.178458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235539+0000","last_change":"2024-02-26T10:38:35.185342+0000","last_active":"2024-02-26T10:38:37.235539+0000","last_peered":"2024-02-26T10:38:37.235539+0000","last_clean":"2024-02-26T10:38:37.235539+0000","last_became_active":"2024-02-26T10:38:35.185067+0000","last_became_peered":"2024-02-26T10:38:35.185067+0000","last_unstale":"2024-02-26T10:38:37.235539+0000","last_undegraded":"2024-02-26T10:38:37.235539+0000","last_fullsized":"2024-02-26T10:38:37.235539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181690+0000","last_change":"2024-02-26T10:38:35.191702+0000","last_active":"2024-02-26T10:38:36.181690+0000","last_peered":"2024-02-26T10:38:36.181690+0000","last_clean":"2024-02-26T10:38:36.181690+0000","last_became_active":"2024-02-26T10:38:35.191416+0000","last_became_peered":"2024-02-26T10:38:35.191416+0000","last_unstale":"2024-02-26T10:38:36.181690+0000","last_undegraded":"2024-02-26T10:38:36.181690+0000","last_fullsized":"2024-02-26T10:38:36.181690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175512+0000","last_change":"2024-02-26T10:38:35.169056+0000","last_active":"2024-02-26T10:38:36.175512+0000","last_peered":"2024-02-26T10:38:36.175512+0000","last_clean":"2024-02-26T10:38:36.175512+0000","last_became_active":"2024-02-26T10:38:35.168640+0000","last_became_peered":"2024-02-26T10:38:35.168640+0000","last_unstale":"2024-02-26T10:38:36.175512+0000","last_undegraded":"2024-02-26T10:38:36.175512+0000","last_fullsized":"2024-02-26T10:38:36.175512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:44.237255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235514+0000","last_change":"2024-02-26T10:38:34.169112+0000","last_active":"2024-02-26T10:38:37.235514+0000","last_peered":"2024-02-26T10:38:37.235514+0000","last_clean":"2024-02-26T10:38:37.235514+0000","last_became_active":"2024-02-26T10:38:34.168796+0000","last_became_peered":"2024-02-26T10:38:34.168796+0000","last_unstale":"2024-02-26T10:38:37.235514+0000","last_undegraded":"2024-02-26T10:38:37.235514+0000","last_fullsized":"2024-02-26T10:38:37.235514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:29:24.719556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178388+0000","last_change":"2024-02-26T10:38:35.187182+0000","last_active":"2024-02-26T10:38:36.178388+0000","last_peered":"2024-02-26T10:38:36.178388+0000","last_clean":"2024-02-26T10:38:36.178388+0000","last_became_active":"2024-02-26T10:38:35.185661+0000","last_became_peered":"2024-02-26T10:38:35.185661+0000","last_unstale":"2024-02-26T10:38:36.178388+0000","last_undegraded":"2024-02-26T10:38:36.178388+0000","last_fullsized":"2024-02-26T10:38:36.178388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181759+0000","last_change":"2024-02-26T10:38:34.179842+0000","last_active":"2024-02-26T10:38:36.181759+0000","last_peered":"2024-02-26T10:38:36.181759+0000","last_clean":"2024-02-26T10:38:36.181759+0000","last_became_active":"2024-02-26T10:38:34.179535+0000","last_became_peered":"2024-02-26T10:38:34.179535+0000","last_unstale":"2024-02-26T10:38:36.181759+0000","last_undegraded":"2024-02-26T10:38:36.181759+0000","last_fullsized":"2024-02-26T10:38:36.181759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235171+0000","last_change":"2024-02-26T10:38:34.167613+0000","last_active":"2024-02-26T10:38:37.235171+0000","last_peered":"2024-02-26T10:38:37.235171+0000","last_clean":"2024-02-26T10:38:37.235171+0000","last_became_active":"2024-02-26T10:38:34.167252+0000","last_became_peered":"2024-02-26T10:38:34.167252+0000","last_unstale":"2024-02-26T10:38:37.235171+0000","last_undegraded":"2024-02-26T10:38:37.235171+0000","last_fullsized":"2024-02-26T10:38:37.235171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235040+0000","last_change":"2024-02-26T10:38:35.185939+0000","last_active":"2024-02-26T10:38:37.235040+0000","last_peered":"2024-02-26T10:38:37.235040+0000","last_clean":"2024-02-26T10:38:37.235040+0000","last_became_active":"2024-02-26T10:38:35.183226+0000","last_became_peered":"2024-02-26T10:38:35.183226+0000","last_unstale":"2024-02-26T10:38:37.235040+0000","last_undegraded":"2024-02-26T10:38:37.235040+0000","last_fullsized":"2024-02-26T10:38:37.235040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236134+0000","last_change":"2024-02-26T10:38:34.186454+0000","last_active":"2024-02-26T10:38:37.236134+0000","last_peered":"2024-02-26T10:38:37.236134+0000","last_clean":"2024-02-26T10:38:37.236134+0000","last_became_active":"2024-02-26T10:38:34.185676+0000","last_became_peered":"2024-02-26T10:38:34.185676+0000","last_unstale":"2024-02-26T10:38:37.236134+0000","last_undegraded":"2024-02-26T10:38:37.236134+0000","last_fullsized":"2024-02-26T10:38:37.236134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173408+0000","last_change":"2024-02-26T10:38:35.176977+0000","last_active":"2024-02-26T10:38:36.173408+0000","last_peered":"2024-02-26T10:38:36.173408+0000","last_clean":"2024-02-26T10:38:36.173408+0000","last_became_active":"2024-02-26T10:38:35.175624+0000","last_became_peered":"2024-02-26T10:38:35.175624+0000","last_unstale":"2024-02-26T10:38:36.173408+0000","last_undegraded":"2024-02-26T10:38:36.173408+0000","last_fullsized":"2024-02-26T10:38:36.173408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182463+0000","last_change":"2024-02-26T10:38:34.188391+0000","last_active":"2024-02-26T10:38:36.182463+0000","last_peered":"2024-02-26T10:38:36.182463+0000","last_clean":"2024-02-26T10:38:36.182463+0000","last_became_active":"2024-02-26T10:38:34.188077+0000","last_became_peered":"2024-02-26T10:38:34.188077+0000","last_unstale":"2024-02-26T10:38:36.182463+0000","last_undegraded":"2024-02-26T10:38:36.182463+0000","last_fullsized":"2024-02-26T10:38:36.182463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236526+0000","last_change":"2024-02-26T10:38:35.199765+0000","last_active":"2024-02-26T10:38:37.236526+0000","last_peered":"2024-02-26T10:38:37.236526+0000","last_clean":"2024-02-26T10:38:37.236526+0000","last_became_active":"2024-02-26T10:38:35.199478+0000","last_became_peered":"2024-02-26T10:38:35.199478+0000","last_unstale":"2024-02-26T10:38:37.236526+0000","last_undegraded":"2024-02-26T10:38:37.236526+0000","last_fullsized":"2024-02-26T10:38:37.236526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236673+0000","last_change":"2024-02-26T10:38:34.189159+0000","last_active":"2024-02-26T10:38:37.236673+0000","last_peered":"2024-02-26T10:38:37.236673+0000","last_clean":"2024-02-26T10:38:37.236673+0000","last_became_active":"2024-02-26T10:38:34.187774+0000","last_became_peered":"2024-02-26T10:38:34.187774+0000","last_unstale":"2024-02-26T10:38:37.236673+0000","last_undegraded":"2024-02-26T10:38:37.236673+0000","last_fullsized":"2024-02-26T10:38:37.236673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178929+0000","last_change":"2024-02-26T10:38:35.198495+0000","last_active":"2024-02-26T10:38:36.178929+0000","last_peered":"2024-02-26T10:38:36.178929+0000","last_clean":"2024-02-26T10:38:36.178929+0000","last_became_active":"2024-02-26T10:38:35.198049+0000","last_became_peered":"2024-02-26T10:38:35.198049+0000","last_unstale":"2024-02-26T10:38:36.178929+0000","last_undegraded":"2024-02-26T10:38:36.178929+0000","last_fullsized":"2024-02-26T10:38:36.178929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173743+0000","last_change":"2024-02-26T10:38:34.182673+0000","last_active":"2024-02-26T10:38:36.173743+0000","last_peered":"2024-02-26T10:38:36.173743+0000","last_clean":"2024-02-26T10:38:36.173743+0000","last_became_active":"2024-02-26T10:38:34.182369+0000","last_became_peered":"2024-02-26T10:38:34.182369+0000","last_unstale":"2024-02-26T10:38:36.173743+0000","last_undegraded":"2024-02-26T10:38:36.173743+0000","last_fullsized":"2024-02-26T10:38:36.173743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175617+0000","last_change":"2024-02-26T10:38:35.193526+0000","last_active":"2024-02-26T10:38:36.175617+0000","last_peered":"2024-02-26T10:38:36.175617+0000","last_clean":"2024-02-26T10:38:36.175617+0000","last_became_active":"2024-02-26T10:38:35.193192+0000","last_became_peered":"2024-02-26T10:38:35.193192+0000","last_unstale":"2024-02-26T10:38:36.175617+0000","last_undegraded":"2024-02-26T10:38:36.175617+0000","last_fullsized":"2024-02-26T10:38:36.175617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:08.969223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182341+0000","last_change":"2024-02-26T10:38:34.178005+0000","last_active":"2024-02-26T10:38:36.182341+0000","last_peered":"2024-02-26T10:38:36.182341+0000","last_clean":"2024-02-26T10:38:36.182341+0000","last_became_active":"2024-02-26T10:38:34.177684+0000","last_became_peered":"2024-02-26T10:38:34.177684+0000","last_unstale":"2024-02-26T10:38:36.182341+0000","last_undegraded":"2024-02-26T10:38:36.182341+0000","last_fullsized":"2024-02-26T10:38:36.182341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.196333+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.191241+0000","last_became_peered":"2024-02-26T10:38:35.191241+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178541+0000","last_change":"2024-02-26T10:38:34.178253+0000","last_active":"2024-02-26T10:38:36.178541+0000","last_peered":"2024-02-26T10:38:36.178541+0000","last_clean":"2024-02-26T10:38:36.178541+0000","last_became_active":"2024-02-26T10:38:34.177880+0000","last_became_peered":"2024-02-26T10:38:34.177880+0000","last_unstale":"2024-02-26T10:38:36.178541+0000","last_undegraded":"2024-02-26T10:38:36.178541+0000","last_fullsized":"2024-02-26T10:38:36.178541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177760+0000","last_change":"2024-02-26T10:38:35.196164+0000","last_active":"2024-02-26T10:38:36.177760+0000","last_peered":"2024-02-26T10:38:36.177760+0000","last_clean":"2024-02-26T10:38:36.177760+0000","last_became_active":"2024-02-26T10:38:35.195846+0000","last_became_peered":"2024-02-26T10:38:35.195846+0000","last_unstale":"2024-02-26T10:38:36.177760+0000","last_undegraded":"2024-02-26T10:38:36.177760+0000","last_fullsized":"2024-02-26T10:38:36.177760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:36.847842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235392+0000","last_change":"2024-02-26T10:38:34.180900+0000","last_active":"2024-02-26T10:38:37.235392+0000","last_peered":"2024-02-26T10:38:37.235392+0000","last_clean":"2024-02-26T10:38:37.235392+0000","last_became_active":"2024-02-26T10:38:34.180316+0000","last_became_peered":"2024-02-26T10:38:34.180316+0000","last_unstale":"2024-02-26T10:38:37.235392+0000","last_undegraded":"2024-02-26T10:38:37.235392+0000","last_fullsized":"2024-02-26T10:38:37.235392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235901+0000","last_change":"2024-02-26T10:38:35.188962+0000","last_active":"2024-02-26T10:38:37.235901+0000","last_peered":"2024-02-26T10:38:37.235901+0000","last_clean":"2024-02-26T10:38:37.235901+0000","last_became_active":"2024-02-26T10:38:35.188506+0000","last_became_peered":"2024-02-26T10:38:35.188506+0000","last_unstale":"2024-02-26T10:38:37.235901+0000","last_undegraded":"2024-02-26T10:38:37.235901+0000","last_fullsized":"2024-02-26T10:38:37.235901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181772+0000","last_change":"2024-02-26T10:38:34.180239+0000","last_active":"2024-02-26T10:38:36.181772+0000","last_peered":"2024-02-26T10:38:36.181772+0000","last_clean":"2024-02-26T10:38:36.181772+0000","last_became_active":"2024-02-26T10:38:34.179109+0000","last_became_peered":"2024-02-26T10:38:34.179109+0000","last_unstale":"2024-02-26T10:38:36.181772+0000","last_undegraded":"2024-02-26T10:38:36.181772+0000","last_fullsized":"2024-02-26T10:38:36.181772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181887+0000","last_change":"2024-02-26T10:38:35.196600+0000","last_active":"2024-02-26T10:38:36.181887+0000","last_peered":"2024-02-26T10:38:36.181887+0000","last_clean":"2024-02-26T10:38:36.181887+0000","last_became_active":"2024-02-26T10:38:35.192063+0000","last_became_peered":"2024-02-26T10:38:35.192063+0000","last_unstale":"2024-02-26T10:38:36.181887+0000","last_undegraded":"2024-02-26T10:38:36.181887+0000","last_fullsized":"2024-02-26T10:38:36.181887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237658+0000","last_change":"2024-02-26T10:38:35.190544+0000","last_active":"2024-02-26T10:38:37.237658+0000","last_peered":"2024-02-26T10:38:37.237658+0000","last_clean":"2024-02-26T10:38:37.237658+0000","last_became_active":"2024-02-26T10:38:35.190066+0000","last_became_peered":"2024-02-26T10:38:35.190066+0000","last_unstale":"2024-02-26T10:38:37.237658+0000","last_undegraded":"2024-02-26T10:38:37.237658+0000","last_fullsized":"2024-02-26T10:38:37.237658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175919+0000","last_change":"2024-02-26T10:38:34.165622+0000","last_active":"2024-02-26T10:38:36.175919+0000","last_peered":"2024-02-26T10:38:36.175919+0000","last_clean":"2024-02-26T10:38:36.175919+0000","last_became_active":"2024-02-26T10:38:34.165165+0000","last_became_peered":"2024-02-26T10:38:34.165165+0000","last_unstale":"2024-02-26T10:38:36.175919+0000","last_undegraded":"2024-02-26T10:38:36.175919+0000","last_fullsized":"2024-02-26T10:38:36.175919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:02.556621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236130+0000","last_change":"2024-02-26T10:38:35.191334+0000","last_active":"2024-02-26T10:38:37.236130+0000","last_peered":"2024-02-26T10:38:37.236130+0000","last_clean":"2024-02-26T10:38:37.236130+0000","last_became_active":"2024-02-26T10:38:35.187187+0000","last_became_peered":"2024-02-26T10:38:35.187187+0000","last_unstale":"2024-02-26T10:38:37.236130+0000","last_undegraded":"2024-02-26T10:38:37.236130+0000","last_fullsized":"2024-02-26T10:38:37.236130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.238041+0000","last_change":"2024-02-26T10:38:35.189775+0000","last_active":"2024-02-26T10:38:37.238041+0000","last_peered":"2024-02-26T10:38:37.238041+0000","last_clean":"2024-02-26T10:38:37.238041+0000","last_became_active":"2024-02-26T10:38:35.189470+0000","last_became_peered":"2024-02-26T10:38:35.189470+0000","last_unstale":"2024-02-26T10:38:37.238041+0000","last_undegraded":"2024-02-26T10:38:37.238041+0000","last_fullsized":"2024-02-26T10:38:37.238041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:56:01.655512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174043+0000","last_change":"2024-02-26T10:38:34.180005+0000","last_active":"2024-02-26T10:38:36.174043+0000","last_peered":"2024-02-26T10:38:36.174043+0000","last_clean":"2024-02-26T10:38:36.174043+0000","last_became_active":"2024-02-26T10:38:34.179694+0000","last_became_peered":"2024-02-26T10:38:34.179694+0000","last_unstale":"2024-02-26T10:38:36.174043+0000","last_undegraded":"2024-02-26T10:38:36.174043+0000","last_fullsized":"2024-02-26T10:38:36.174043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236746+0000","last_change":"2024-02-26T10:38:35.190450+0000","last_active":"2024-02-26T10:38:37.236746+0000","last_peered":"2024-02-26T10:38:37.236746+0000","last_clean":"2024-02-26T10:38:37.236746+0000","last_became_active":"2024-02-26T10:38:35.187303+0000","last_became_peered":"2024-02-26T10:38:35.187303+0000","last_unstale":"2024-02-26T10:38:37.236746+0000","last_undegraded":"2024-02-26T10:38:37.236746+0000","last_fullsized":"2024-02-26T10:38:37.236746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235843+0000","last_change":"2024-02-26T10:38:35.196564+0000","last_active":"2024-02-26T10:38:37.235843+0000","last_peered":"2024-02-26T10:38:37.235843+0000","last_clean":"2024-02-26T10:38:37.235843+0000","last_became_active":"2024-02-26T10:38:35.196286+0000","last_became_peered":"2024-02-26T10:38:35.196286+0000","last_unstale":"2024-02-26T10:38:37.235843+0000","last_undegraded":"2024-02-26T10:38:37.235843+0000","last_fullsized":"2024-02-26T10:38:37.235843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235369+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:37.235369+0000","last_peered":"2024-02-26T10:38:37.235369+0000","last_clean":"2024-02-26T10:38:37.235369+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:37.235369+0000","last_undegraded":"2024-02-26T10:38:37.235369+0000","last_fullsized":"2024-02-26T10:38:37.235369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234740+0000","last_change":"2024-02-26T10:38:34.181165+0000","last_active":"2024-02-26T10:38:37.234740+0000","last_peered":"2024-02-26T10:38:37.234740+0000","last_clean":"2024-02-26T10:38:37.234740+0000","last_became_active":"2024-02-26T10:38:34.180374+0000","last_became_peered":"2024-02-26T10:38:34.180374+0000","last_unstale":"2024-02-26T10:38:37.234740+0000","last_undegraded":"2024-02-26T10:38:37.234740+0000","last_fullsized":"2024-02-26T10:38:37.234740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235370+0000","last_change":"2024-02-26T10:38:35.205573+0000","last_active":"2024-02-26T10:38:37.235370+0000","last_peered":"2024-02-26T10:38:37.235370+0000","last_clean":"2024-02-26T10:38:37.235370+0000","last_became_active":"2024-02-26T10:38:35.205295+0000","last_became_peered":"2024-02-26T10:38:35.205295+0000","last_unstale":"2024-02-26T10:38:37.235370+0000","last_undegraded":"2024-02-26T10:38:37.235370+0000","last_fullsized":"2024-02-26T10:38:37.235370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234753+0000","last_change":"2024-02-26T10:38:34.177423+0000","last_active":"2024-02-26T10:38:37.234753+0000","last_peered":"2024-02-26T10:38:37.234753+0000","last_clean":"2024-02-26T10:38:37.234753+0000","last_became_active":"2024-02-26T10:38:34.175531+0000","last_became_peered":"2024-02-26T10:38:34.175531+0000","last_unstale":"2024-02-26T10:38:37.234753+0000","last_undegraded":"2024-02-26T10:38:37.234753+0000","last_fullsized":"2024-02-26T10:38:37.234753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.176766+0000","last_change":"2024-02-26T10:38:35.182828+0000","last_active":"2024-02-26T10:38:36.176766+0000","last_peered":"2024-02-26T10:38:36.176766+0000","last_clean":"2024-02-26T10:38:36.176766+0000","last_became_active":"2024-02-26T10:38:35.182453+0000","last_became_peered":"2024-02-26T10:38:35.182453+0000","last_unstale":"2024-02-26T10:38:36.176766+0000","last_undegraded":"2024-02-26T10:38:36.176766+0000","last_fullsized":"2024-02-26T10:38:36.176766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:15.325827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173393+0000","last_change":"2024-02-26T10:38:35.180693+0000","last_active":"2024-02-26T10:38:36.173393+0000","last_peered":"2024-02-26T10:38:36.173393+0000","last_clean":"2024-02-26T10:38:36.173393+0000","last_became_active":"2024-02-26T10:38:35.178787+0000","last_became_peered":"2024-02-26T10:38:35.178787+0000","last_unstale":"2024-02-26T10:38:36.173393+0000","last_undegraded":"2024-02-26T10:38:36.173393+0000","last_fullsized":"2024-02-26T10:38:36.173393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235683+0000","last_change":"2024-02-26T10:38:34.184572+0000","last_active":"2024-02-26T10:38:37.235683+0000","last_peered":"2024-02-26T10:38:37.235683+0000","last_clean":"2024-02-26T10:38:37.235683+0000","last_became_active":"2024-02-26T10:38:34.183650+0000","last_became_peered":"2024-02-26T10:38:34.183650+0000","last_unstale":"2024-02-26T10:38:37.235683+0000","last_undegraded":"2024-02-26T10:38:37.235683+0000","last_fullsized":"2024-02-26T10:38:37.235683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175481+0000","last_change":"2024-02-26T10:38:35.186511+0000","last_active":"2024-02-26T10:38:36.175481+0000","last_peered":"2024-02-26T10:38:36.175481+0000","last_clean":"2024-02-26T10:38:36.175481+0000","last_became_active":"2024-02-26T10:38:35.185321+0000","last_became_peered":"2024-02-26T10:38:35.185321+0000","last_unstale":"2024-02-26T10:38:36.175481+0000","last_undegraded":"2024-02-26T10:38:36.175481+0000","last_fullsized":"2024-02-26T10:38:36.175481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:18.018830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174062+0000","last_change":"2024-02-26T10:38:34.165105+0000","last_active":"2024-02-26T10:38:36.174062+0000","last_peered":"2024-02-26T10:38:36.174062+0000","last_clean":"2024-02-26T10:38:36.174062+0000","last_became_active":"2024-02-26T10:38:34.164756+0000","last_became_peered":"2024-02-26T10:38:34.164756+0000","last_unstale":"2024-02-26T10:38:36.174062+0000","last_undegraded":"2024-02-26T10:38:36.174062+0000","last_fullsized":"2024-02-26T10:38:36.174062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237188+0000","last_change":"2024-02-26T10:38:35.191275+0000","last_active":"2024-02-26T10:38:37.237188+0000","last_peered":"2024-02-26T10:38:37.237188+0000","last_clean":"2024-02-26T10:38:37.237188+0000","last_became_active":"2024-02-26T10:38:35.190875+0000","last_became_peered":"2024-02-26T10:38:35.190875+0000","last_unstale":"2024-02-26T10:38:37.237188+0000","last_undegraded":"2024-02-26T10:38:37.237188+0000","last_fullsized":"2024-02-26T10:38:37.237188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237699+0000","last_change":"2024-02-26T10:38:34.176727+0000","last_active":"2024-02-26T10:38:37.237699+0000","last_peered":"2024-02-26T10:38:37.237699+0000","last_clean":"2024-02-26T10:38:37.237699+0000","last_became_active":"2024-02-26T10:38:34.176422+0000","last_became_peered":"2024-02-26T10:38:34.176422+0000","last_unstale":"2024-02-26T10:38:37.237699+0000","last_undegraded":"2024-02-26T10:38:37.237699+0000","last_fullsized":"2024-02-26T10:38:37.237699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174128+0000","last_change":"2024-02-26T10:38:35.174940+0000","last_active":"2024-02-26T10:38:36.174128+0000","last_peered":"2024-02-26T10:38:36.174128+0000","last_clean":"2024-02-26T10:38:36.174128+0000","last_became_active":"2024-02-26T10:38:35.174609+0000","last_became_peered":"2024-02-26T10:38:35.174609+0000","last_unstale":"2024-02-26T10:38:36.174128+0000","last_undegraded":"2024-02-26T10:38:36.174128+0000","last_fullsized":"2024-02-26T10:38:36.174128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174557+0000","last_change":"2024-02-26T10:38:34.176149+0000","last_active":"2024-02-26T10:38:36.174557+0000","last_peered":"2024-02-26T10:38:36.174557+0000","last_clean":"2024-02-26T10:38:36.174557+0000","last_became_active":"2024-02-26T10:38:34.175800+0000","last_became_peered":"2024-02-26T10:38:34.175800+0000","last_unstale":"2024-02-26T10:38:36.174557+0000","last_undegraded":"2024-02-26T10:38:36.174557+0000","last_fullsized":"2024-02-26T10:38:36.174557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236113+0000","last_change":"2024-02-26T10:38:35.191410+0000","last_active":"2024-02-26T10:38:37.236113+0000","last_peered":"2024-02-26T10:38:37.236113+0000","last_clean":"2024-02-26T10:38:37.236113+0000","last_became_active":"2024-02-26T10:38:35.190954+0000","last_became_peered":"2024-02-26T10:38:35.190954+0000","last_unstale":"2024-02-26T10:38:37.236113+0000","last_undegraded":"2024-02-26T10:38:37.236113+0000","last_fullsized":"2024-02-26T10:38:37.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182629+0000","last_change":"2024-02-26T10:38:34.185447+0000","last_active":"2024-02-26T10:38:36.182629+0000","last_peered":"2024-02-26T10:38:36.182629+0000","last_clean":"2024-02-26T10:38:36.182629+0000","last_became_active":"2024-02-26T10:38:34.185054+0000","last_became_peered":"2024-02-26T10:38:34.185054+0000","last_unstale":"2024-02-26T10:38:36.182629+0000","last_undegraded":"2024-02-26T10:38:36.182629+0000","last_fullsized":"2024-02-26T10:38:36.182629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179198+0000","last_change":"2024-02-26T10:38:35.181510+0000","last_active":"2024-02-26T10:38:36.179198+0000","last_peered":"2024-02-26T10:38:36.179198+0000","last_clean":"2024-02-26T10:38:36.179198+0000","last_became_active":"2024-02-26T10:38:35.180830+0000","last_became_peered":"2024-02-26T10:38:35.180830+0000","last_unstale":"2024-02-26T10:38:36.179198+0000","last_undegraded":"2024-02-26T10:38:36.179198+0000","last_fullsized":"2024-02-26T10:38:36.179198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234630+0000","last_change":"2024-02-26T10:38:34.174303+0000","last_active":"2024-02-26T10:38:37.234630+0000","last_peered":"2024-02-26T10:38:37.234630+0000","last_clean":"2024-02-26T10:38:37.234630+0000","last_became_active":"2024-02-26T10:38:34.174006+0000","last_became_peered":"2024-02-26T10:38:34.174006+0000","last_unstale":"2024-02-26T10:38:37.234630+0000","last_undegraded":"2024-02-26T10:38:37.234630+0000","last_fullsized":"2024-02-26T10:38:37.234630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175416+0000","last_change":"2024-02-26T10:38:35.176832+0000","last_active":"2024-02-26T10:38:36.175416+0000","last_peered":"2024-02-26T10:38:36.175416+0000","last_clean":"2024-02-26T10:38:36.175416+0000","last_became_active":"2024-02-26T10:38:35.176305+0000","last_became_peered":"2024-02-26T10:38:35.176305+0000","last_unstale":"2024-02-26T10:38:36.175416+0000","last_undegraded":"2024-02-26T10:38:36.175416+0000","last_fullsized":"2024-02-26T10:38:36.175416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:06.286416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173704+0000","last_change":"2024-02-26T10:38:34.169425+0000","last_active":"2024-02-26T10:38:36.173704+0000","last_peered":"2024-02-26T10:38:36.173704+0000","last_clean":"2024-02-26T10:38:36.173704+0000","last_became_active":"2024-02-26T10:38:34.168217+0000","last_became_peered":"2024-02-26T10:38:34.168217+0000","last_unstale":"2024-02-26T10:38:36.173704+0000","last_undegraded":"2024-02-26T10:38:36.173704+0000","last_fullsized":"2024-02-26T10:38:36.173704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236029+0000","last_change":"2024-02-26T10:38:35.172836+0000","last_active":"2024-02-26T10:38:37.236029+0000","last_peered":"2024-02-26T10:38:37.236029+0000","last_clean":"2024-02-26T10:38:37.236029+0000","last_became_active":"2024-02-26T10:38:35.172554+0000","last_became_peered":"2024-02-26T10:38:35.172554+0000","last_unstale":"2024-02-26T10:38:37.236029+0000","last_undegraded":"2024-02-26T10:38:37.236029+0000","last_fullsized":"2024-02-26T10:38:37.236029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173693+0000","last_change":"2024-02-26T10:38:34.166377+0000","last_active":"2024-02-26T10:38:36.173693+0000","last_peered":"2024-02-26T10:38:36.173693+0000","last_clean":"2024-02-26T10:38:36.173693+0000","last_became_active":"2024-02-26T10:38:34.165963+0000","last_became_peered":"2024-02-26T10:38:34.165963+0000","last_unstale":"2024-02-26T10:38:36.173693+0000","last_undegraded":"2024-02-26T10:38:36.173693+0000","last_fullsized":"2024-02-26T10:38:36.173693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235575+0000","last_change":"2024-02-26T10:38:35.185120+0000","last_active":"2024-02-26T10:38:37.235575+0000","last_peered":"2024-02-26T10:38:37.235575+0000","last_clean":"2024-02-26T10:38:37.235575+0000","last_became_active":"2024-02-26T10:38:35.184832+0000","last_became_peered":"2024-02-26T10:38:35.184832+0000","last_unstale":"2024-02-26T10:38:37.235575+0000","last_undegraded":"2024-02-26T10:38:37.235575+0000","last_fullsized":"2024-02-26T10:38:37.235575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236461+0000","last_change":"2024-02-26T10:38:34.180225+0000","last_active":"2024-02-26T10:38:37.236461+0000","last_peered":"2024-02-26T10:38:37.236461+0000","last_clean":"2024-02-26T10:38:37.236461+0000","last_became_active":"2024-02-26T10:38:34.179909+0000","last_became_peered":"2024-02-26T10:38:34.179909+0000","last_unstale":"2024-02-26T10:38:37.236461+0000","last_undegraded":"2024-02-26T10:38:37.236461+0000","last_fullsized":"2024-02-26T10:38:37.236461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:05:58.590708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182262+0000","last_change":"2024-02-26T10:38:35.188975+0000","last_active":"2024-02-26T10:38:36.182262+0000","last_peered":"2024-02-26T10:38:36.182262+0000","last_clean":"2024-02-26T10:38:36.182262+0000","last_became_active":"2024-02-26T10:38:35.188608+0000","last_became_peered":"2024-02-26T10:38:35.188608+0000","last_unstale":"2024-02-26T10:38:36.182262+0000","last_undegraded":"2024-02-26T10:38:36.182262+0000","last_fullsized":"2024-02-26T10:38:36.182262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236117+0000","last_change":"2024-02-26T10:38:35.196289+0000","last_active":"2024-02-26T10:38:37.236117+0000","last_peered":"2024-02-26T10:38:37.236117+0000","last_clean":"2024-02-26T10:38:37.236117+0000","last_became_active":"2024-02-26T10:38:35.194751+0000","last_became_peered":"2024-02-26T10:38:35.194751+0000","last_unstale":"2024-02-26T10:38:37.236117+0000","last_undegraded":"2024-02-26T10:38:37.236117+0000","last_fullsized":"2024-02-26T10:38:37.236117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236713+0000","last_change":"2024-02-26T10:38:34.186760+0000","last_active":"2024-02-26T10:38:37.236713+0000","last_peered":"2024-02-26T10:38:37.236713+0000","last_clean":"2024-02-26T10:38:37.236713+0000","last_became_active":"2024-02-26T10:38:34.186428+0000","last_became_peered":"2024-02-26T10:38:34.186428+0000","last_unstale":"2024-02-26T10:38:37.236713+0000","last_undegraded":"2024-02-26T10:38:37.236713+0000","last_fullsized":"2024-02-26T10:38:37.236713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236518+0000","last_change":"2024-02-26T10:38:35.202616+0000","last_active":"2024-02-26T10:38:37.236518+0000","last_peered":"2024-02-26T10:38:37.236518+0000","last_clean":"2024-02-26T10:38:37.236518+0000","last_became_active":"2024-02-26T10:38:35.201498+0000","last_became_peered":"2024-02-26T10:38:35.201498+0000","last_unstale":"2024-02-26T10:38:37.236518+0000","last_undegraded":"2024-02-26T10:38:37.236518+0000","last_fullsized":"2024-02-26T10:38:37.236518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235996+0000","last_change":"2024-02-26T10:38:34.186745+0000","last_active":"2024-02-26T10:38:37.235996+0000","last_peered":"2024-02-26T10:38:37.235996+0000","last_clean":"2024-02-26T10:38:37.235996+0000","last_became_active":"2024-02-26T10:38:34.186470+0000","last_became_peered":"2024-02-26T10:38:34.186470+0000","last_unstale":"2024-02-26T10:38:37.235996+0000","last_undegraded":"2024-02-26T10:38:37.235996+0000","last_fullsized":"2024-02-26T10:38:37.235996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178984+0000","last_change":"2024-02-26T10:38:35.206276+0000","last_active":"2024-02-26T10:38:36.178984+0000","last_peered":"2024-02-26T10:38:36.178984+0000","last_clean":"2024-02-26T10:38:36.178984+0000","last_became_active":"2024-02-26T10:38:35.205932+0000","last_became_peered":"2024-02-26T10:38:35.205932+0000","last_unstale":"2024-02-26T10:38:36.178984+0000","last_undegraded":"2024-02-26T10:38:36.178984+0000","last_fullsized":"2024-02-26T10:38:36.178984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235907+0000","last_change":"2024-02-26T10:38:34.165440+0000","last_active":"2024-02-26T10:38:37.235907+0000","last_peered":"2024-02-26T10:38:37.235907+0000","last_clean":"2024-02-26T10:38:37.235907+0000","last_became_active":"2024-02-26T10:38:34.165164+0000","last_became_peered":"2024-02-26T10:38:34.165164+0000","last_unstale":"2024-02-26T10:38:37.235907+0000","last_undegraded":"2024-02-26T10:38:37.235907+0000","last_fullsized":"2024-02-26T10:38:37.235907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235740+0000","last_change":"2024-02-26T10:38:35.199753+0000","last_active":"2024-02-26T10:38:37.235740+0000","last_peered":"2024-02-26T10:38:37.235740+0000","last_clean":"2024-02-26T10:38:37.235740+0000","last_became_active":"2024-02-26T10:38:35.199470+0000","last_became_peered":"2024-02-26T10:38:35.199470+0000","last_unstale":"2024-02-26T10:38:37.235740+0000","last_undegraded":"2024-02-26T10:38:37.235740+0000","last_fullsized":"2024-02-26T10:38:37.235740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179016+0000","last_change":"2024-02-26T10:38:34.184020+0000","last_active":"2024-02-26T10:38:36.179016+0000","last_peered":"2024-02-26T10:38:36.179016+0000","last_clean":"2024-02-26T10:38:36.179016+0000","last_became_active":"2024-02-26T10:38:34.183497+0000","last_became_peered":"2024-02-26T10:38:34.183497+0000","last_unstale":"2024-02-26T10:38:36.179016+0000","last_undegraded":"2024-02-26T10:38:36.179016+0000","last_fullsized":"2024-02-26T10:38:36.179016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:25:08.410476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":74327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":819200,"data_stored":665243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:50.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:38:50.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.617+0000 7fcf7be00640 1 Processor -- start 2024-02-26T10:38:50.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.617+0000 7fcf7be00640 1 -- start start 2024-02-26T10:38:50.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcf7c0c2310 0x7fcf7c153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c155f10 con 0x7fcf7c06ac20 2024-02-26T10:38:50.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c06b470 con 0x7fcf7c0c2310 2024-02-26T10:38:50.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c0c28d0 con 0x7fcf7c06b000 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcf7c0c2310 0x7fcf7c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcf7c0c2310 0x7fcf7c153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47636/0 (socket says 172.21.15.67:47636) 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.619+0000 7fcf7a000640 1 -- 172.21.15.67:0/4186736550 learned_addr learned my addr 172.21.15.67:0/4186736550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.620+0000 7fcf7a000640 1 -- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 msgr2=0x7fcf7c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.620+0000 7fcf7a000640 1 --2- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.620+0000 7fcf7a000640 1 -- 172.21.15.67:0/4186736550 >> v1:172.21.15.82:6789/0 conn(0x7fcf7c06ac20 legacy=0x7fcf7c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.620+0000 7fcf7a000640 1 -- 172.21.15.67:0/4186736550 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c155150 con 0x7fcf7c0c2310 2024-02-26T10:38:50.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.621+0000 7fcf7a000640 1 --2- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c153630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800bb10 tx=0x7fcf6802e030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e878eceb1449c66 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf79600640 1 -- 172.21.15.67:0/4186736550 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6803b070 con 0x7fcf7c0c2310 2024-02-26T10:38:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf79600640 1 -- 172.21.15.67:0/4186736550 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6802ede0 con 0x7fcf7c0c2310 2024-02-26T10:38:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf79600640 1 -- 172.21.15.67:0/4186736550 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68036660 con 0x7fcf7c0c2310 2024-02-26T10:38:50.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf7be00640 1 -- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 msgr2=0x7fcf7c153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf7be00640 1 --2- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c153630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800bb10 tx=0x7fcf6802e030 comp rx=0 tx=0).stop 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf7be00640 1 -- 172.21.15.67:0/4186736550 shutdown_connections 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf7be00640 1 --2- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.622+0000 7fcf7be00640 1 --2- 172.21.15.67:0/4186736550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c153630 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.623+0000 7fcf7be00640 1 -- 172.21.15.67:0/4186736550 >> 172.21.15.67:0/4186736550 conn(0x7fcf7c0c63f0 msgr2=0x7fcf7c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.623+0000 7fcf7be00640 1 -- 172.21.15.67:0/4186736550 shutdown_connections 2024-02-26T10:38:50.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.623+0000 7fcf7be00640 1 -- 172.21.15.67:0/4186736550 wait complete. 2024-02-26T10:38:50.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.625+0000 7fcf7be00640 1 Processor -- start 2024-02-26T10:38:50.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.625+0000 7fcf7be00640 1 -- start start 2024-02-26T10:38:50.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c1edd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1559f0 con 0x7fcf7c06ac20 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c155710 con 0x7fcf7c0c2310 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c148850 con 0x7fcf7c06b000 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44384/0 (socket says 172.21.15.67:44384) 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.626+0000 7fcf7aa00640 1 -- 172.21.15.67:0/3262531434 learned_addr learned my addr 172.21.15.67:0/3262531434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7a000640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7b400640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c1edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7aa00640 1 -- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 msgr2=0x7fcf7c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7aa00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7aa00640 1 -- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 msgr2=0x7fcf7c1edd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7aa00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c1edd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7aa00640 1 -- 172.21.15.67:0/3262531434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c1eec00 con 0x7fcf7c06ac20 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7aa00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c1539c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c430 tx=0x7fcf6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:50.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c017070 con 0x7fcf7c06ac20 2024-02-26T10:38:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c0052d0 con 0x7fcf7c06ac20 2024-02-26T10:38:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c003830 con 0x7fcf7c06ac20 2024-02-26T10:38:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.627+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c154830 con 0x7fcf7c06ac20 2024-02-26T10:38:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.628+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c0aade0 con 0x7fcf7c06ac20 2024-02-26T10:38:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.629+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c074a30 con 0x7fcf7c06ac20 2024-02-26T10:38:50.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.630+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf6c01a070 con 0x7fcf7c06ac20 2024-02-26T10:38:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.630+0000 7fcf72a00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:50.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.631+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf6c0894c0 con 0x7fcf7c06ac20 2024-02-26T10:38:50.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.631+0000 7fcf7a000640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:50.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.632+0000 7fcf7a000640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf68038000 tx=0x7fcf68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:50.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.635+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf6c0583d0 con 0x7fcf7c06ac20 2024-02-26T10:38:50.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.824+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf7c0c20f0 con 0x7fcf5c06b4d0 2024-02-26T10:38:50.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.841+0000 7fcf72a00640 1 -- 172.21.15.67:0/3262531434 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251838 (secure 0 0 0) 0x7fcf7c0c20f0 con 0x7fcf5c06b4d0 2024-02-26T10:38:50.845 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:50.848 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.847+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5c06b4d0 msgr2=0x7fcf5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.847+0000 7fcf7be00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf68038000 tx=0x7fcf68008040 comp rx=0 tx=0).stop 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.847+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 msgr2=0x7fcf7c1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.847+0000 7fcf7be00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c1539c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c430 tx=0x7fcf6c00c8f0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 shutdown_connections 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf5c06b4d0 0x7fcf5c06d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c0c2310 0x7fcf7c1edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf7c06b000 0x7fcf7c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 --2- 172.21.15.67:0/3262531434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf7c06ac20 0x7fcf7c1539c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 >> 172.21.15.67:0/3262531434 conn(0x7fcf7c0c63f0 msgr2=0x7fcf7c145840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 shutdown_connections 2024-02-26T10:38:50.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:50.848+0000 7fcf7be00640 1 -- 172.21.15.67:0/3262531434 wait complete. 2024-02-26T10:38:50.868 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-02-26T10:38:50.327607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219204,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755516,"statfs":{"total":773094113280,"available":772869648384,"internally_reserved":0,"allocated":2957312,"data_stored":1779452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":44,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006731"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237702+0000","last_change":"2024-02-26T10:38:35.196643+0000","last_active":"2024-02-26T10:38:37.237702+0000","last_peered":"2024-02-26T10:38:37.237702+0000","last_clean":"2024-02-26T10:38:37.237702+0000","last_became_active":"2024-02-26T10:38:35.195966+0000","last_became_peered":"2024-02-26T10:38:35.195966+0000","last_unstale":"2024-02-26T10:38:37.237702+0000","last_undegraded":"2024-02-26T10:38:37.237702+0000","last_fullsized":"2024-02-26T10:38:37.237702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174389+0000","last_change":"2024-02-26T10:38:35.189849+0000","last_active":"2024-02-26T10:38:36.174389+0000","last_peered":"2024-02-26T10:38:36.174389+0000","last_clean":"2024-02-26T10:38:36.174389+0000","last_became_active":"2024-02-26T10:38:35.189290+0000","last_became_peered":"2024-02-26T10:38:35.189290+0000","last_unstale":"2024-02-26T10:38:36.174389+0000","last_undegraded":"2024-02-26T10:38:36.174389+0000","last_fullsized":"2024-02-26T10:38:36.174389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178614+0000","last_change":"2024-02-26T10:38:35.184041+0000","last_active":"2024-02-26T10:38:36.178614+0000","last_peered":"2024-02-26T10:38:36.178614+0000","last_clean":"2024-02-26T10:38:36.178614+0000","last_became_active":"2024-02-26T10:38:35.181755+0000","last_became_peered":"2024-02-26T10:38:35.181755+0000","last_unstale":"2024-02-26T10:38:36.178614+0000","last_undegraded":"2024-02-26T10:38:36.178614+0000","last_fullsized":"2024-02-26T10:38:36.178614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235190+0000","last_change":"2024-02-26T10:38:35.199057+0000","last_active":"2024-02-26T10:38:37.235190+0000","last_peered":"2024-02-26T10:38:37.235190+0000","last_clean":"2024-02-26T10:38:37.235190+0000","last_became_active":"2024-02-26T10:38:35.198578+0000","last_became_peered":"2024-02-26T10:38:35.198578+0000","last_unstale":"2024-02-26T10:38:37.235190+0000","last_undegraded":"2024-02-26T10:38:37.235190+0000","last_fullsized":"2024-02-26T10:38:37.235190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235490+0000","last_change":"2024-02-26T10:38:35.188780+0000","last_active":"2024-02-26T10:38:37.235490+0000","last_peered":"2024-02-26T10:38:37.235490+0000","last_clean":"2024-02-26T10:38:37.235490+0000","last_became_active":"2024-02-26T10:38:35.187459+0000","last_became_peered":"2024-02-26T10:38:35.187459+0000","last_unstale":"2024-02-26T10:38:37.235490+0000","last_undegraded":"2024-02-26T10:38:37.235490+0000","last_fullsized":"2024-02-26T10:38:37.235490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235076+0000","last_change":"2024-02-26T10:38:35.204704+0000","last_active":"2024-02-26T10:38:37.235076+0000","last_peered":"2024-02-26T10:38:37.235076+0000","last_clean":"2024-02-26T10:38:37.235076+0000","last_became_active":"2024-02-26T10:38:35.204329+0000","last_became_peered":"2024-02-26T10:38:35.204329+0000","last_unstale":"2024-02-26T10:38:37.235076+0000","last_undegraded":"2024-02-26T10:38:37.235076+0000","last_fullsized":"2024-02-26T10:38:37.235076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237988+0000","last_change":"2024-02-26T10:38:35.193948+0000","last_active":"2024-02-26T10:38:37.237988+0000","last_peered":"2024-02-26T10:38:37.237988+0000","last_clean":"2024-02-26T10:38:37.237988+0000","last_became_active":"2024-02-26T10:38:35.192130+0000","last_became_peered":"2024-02-26T10:38:35.192130+0000","last_unstale":"2024-02-26T10:38:37.237988+0000","last_undegraded":"2024-02-26T10:38:37.237988+0000","last_fullsized":"2024-02-26T10:38:37.237988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:56:35.875738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234899+0000","last_change":"2024-02-26T10:38:35.205622+0000","last_active":"2024-02-26T10:38:37.234899+0000","last_peered":"2024-02-26T10:38:37.234899+0000","last_clean":"2024-02-26T10:38:37.234899+0000","last_became_active":"2024-02-26T10:38:35.205359+0000","last_became_peered":"2024-02-26T10:38:35.205359+0000","last_unstale":"2024-02-26T10:38:37.234899+0000","last_undegraded":"2024-02-26T10:38:37.234899+0000","last_fullsized":"2024-02-26T10:38:37.234899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237849+0000","last_change":"2024-02-26T10:38:35.202549+0000","last_active":"2024-02-26T10:38:37.237849+0000","last_peered":"2024-02-26T10:38:37.237849+0000","last_clean":"2024-02-26T10:38:37.237849+0000","last_became_active":"2024-02-26T10:38:35.201568+0000","last_became_peered":"2024-02-26T10:38:35.201568+0000","last_unstale":"2024-02-26T10:38:37.237849+0000","last_undegraded":"2024-02-26T10:38:37.237849+0000","last_fullsized":"2024-02-26T10:38:37.237849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182530+0000","last_change":"2024-02-26T10:38:35.206569+0000","last_active":"2024-02-26T10:38:36.182530+0000","last_peered":"2024-02-26T10:38:36.182530+0000","last_clean":"2024-02-26T10:38:36.182530+0000","last_became_active":"2024-02-26T10:38:35.206285+0000","last_became_peered":"2024-02-26T10:38:35.206285+0000","last_unstale":"2024-02-26T10:38:36.182530+0000","last_undegraded":"2024-02-26T10:38:36.182530+0000","last_fullsized":"2024-02-26T10:38:36.182530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182446+0000","last_change":"2024-02-26T10:38:35.209104+0000","last_active":"2024-02-26T10:38:36.182446+0000","last_peered":"2024-02-26T10:38:36.182446+0000","last_clean":"2024-02-26T10:38:36.182446+0000","last_became_active":"2024-02-26T10:38:35.208841+0000","last_became_peered":"2024-02-26T10:38:35.208841+0000","last_unstale":"2024-02-26T10:38:36.182446+0000","last_undegraded":"2024-02-26T10:38:36.182446+0000","last_fullsized":"2024-02-26T10:38:36.182446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175483+0000","last_change":"2024-02-26T10:38:35.198390+0000","last_active":"2024-02-26T10:38:36.175483+0000","last_peered":"2024-02-26T10:38:36.175483+0000","last_clean":"2024-02-26T10:38:36.175483+0000","last_became_active":"2024-02-26T10:38:35.197895+0000","last_became_peered":"2024-02-26T10:38:35.197895+0000","last_unstale":"2024-02-26T10:38:36.175483+0000","last_undegraded":"2024-02-26T10:38:36.175483+0000","last_fullsized":"2024-02-26T10:38:36.175483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:01.842347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.192330+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.192046+0000","last_became_peered":"2024-02-26T10:38:35.192046+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177330+0000","last_change":"2024-02-26T10:38:35.181873+0000","last_active":"2024-02-26T10:38:36.177330+0000","last_peered":"2024-02-26T10:38:36.177330+0000","last_clean":"2024-02-26T10:38:36.177330+0000","last_became_active":"2024-02-26T10:38:35.180543+0000","last_became_peered":"2024-02-26T10:38:35.180543+0000","last_unstale":"2024-02-26T10:38:36.177330+0000","last_undegraded":"2024-02-26T10:38:36.177330+0000","last_fullsized":"2024-02-26T10:38:36.177330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:04:47.351870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173628+0000","last_change":"2024-02-26T10:38:35.186385+0000","last_active":"2024-02-26T10:38:36.173628+0000","last_peered":"2024-02-26T10:38:36.173628+0000","last_clean":"2024-02-26T10:38:36.173628+0000","last_became_active":"2024-02-26T10:38:35.186103+0000","last_became_peered":"2024-02-26T10:38:35.186103+0000","last_unstale":"2024-02-26T10:38:36.173628+0000","last_undegraded":"2024-02-26T10:38:36.173628+0000","last_fullsized":"2024-02-26T10:38:36.173628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234734+0000","last_change":"2024-02-26T10:38:35.203847+0000","last_active":"2024-02-26T10:38:37.234734+0000","last_peered":"2024-02-26T10:38:37.234734+0000","last_clean":"2024-02-26T10:38:37.234734+0000","last_became_active":"2024-02-26T10:38:35.201819+0000","last_became_peered":"2024-02-26T10:38:35.201819+0000","last_unstale":"2024-02-26T10:38:37.234734+0000","last_undegraded":"2024-02-26T10:38:37.234734+0000","last_fullsized":"2024-02-26T10:38:37.234734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237593+0000","last_change":"2024-02-26T10:38:35.205600+0000","last_active":"2024-02-26T10:38:37.237593+0000","last_peered":"2024-02-26T10:38:37.237593+0000","last_clean":"2024-02-26T10:38:37.237593+0000","last_became_active":"2024-02-26T10:38:35.205272+0000","last_became_peered":"2024-02-26T10:38:35.205272+0000","last_unstale":"2024-02-26T10:38:37.237593+0000","last_undegraded":"2024-02-26T10:38:37.237593+0000","last_fullsized":"2024-02-26T10:38:37.237593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:47:16.600627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233375+0000","last_change":"2024-02-26T10:38:35.191840+0000","last_active":"2024-02-26T10:38:37.233375+0000","last_peered":"2024-02-26T10:38:37.233375+0000","last_clean":"2024-02-26T10:38:37.233375+0000","last_became_active":"2024-02-26T10:38:35.191507+0000","last_became_peered":"2024-02-26T10:38:35.191507+0000","last_unstale":"2024-02-26T10:38:37.233375+0000","last_undegraded":"2024-02-26T10:38:37.233375+0000","last_fullsized":"2024-02-26T10:38:37.233375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:54:41.301863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237554+0000","last_change":"2024-02-26T10:38:35.191539+0000","last_active":"2024-02-26T10:38:37.237554+0000","last_peered":"2024-02-26T10:38:37.237554+0000","last_clean":"2024-02-26T10:38:37.237554+0000","last_became_active":"2024-02-26T10:38:35.191258+0000","last_became_peered":"2024-02-26T10:38:35.191258+0000","last_unstale":"2024-02-26T10:38:37.237554+0000","last_undegraded":"2024-02-26T10:38:37.237554+0000","last_fullsized":"2024-02-26T10:38:37.237554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182184+0000","last_change":"2024-02-26T10:38:35.197155+0000","last_active":"2024-02-26T10:38:36.182184+0000","last_peered":"2024-02-26T10:38:36.182184+0000","last_clean":"2024-02-26T10:38:36.182184+0000","last_became_active":"2024-02-26T10:38:35.192752+0000","last_became_peered":"2024-02-26T10:38:35.192752+0000","last_unstale":"2024-02-26T10:38:36.182184+0000","last_undegraded":"2024-02-26T10:38:36.182184+0000","last_fullsized":"2024-02-26T10:38:36.182184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178148+0000","last_change":"2024-02-26T10:38:35.192588+0000","last_active":"2024-02-26T10:38:36.178148+0000","last_peered":"2024-02-26T10:38:36.178148+0000","last_clean":"2024-02-26T10:38:36.178148+0000","last_became_active":"2024-02-26T10:38:35.192244+0000","last_became_peered":"2024-02-26T10:38:35.192244+0000","last_unstale":"2024-02-26T10:38:36.178148+0000","last_undegraded":"2024-02-26T10:38:36.178148+0000","last_fullsized":"2024-02-26T10:38:36.178148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235257+0000","last_change":"2024-02-26T10:38:35.193557+0000","last_active":"2024-02-26T10:38:37.235257+0000","last_peered":"2024-02-26T10:38:37.235257+0000","last_clean":"2024-02-26T10:38:37.235257+0000","last_became_active":"2024-02-26T10:38:35.192814+0000","last_became_peered":"2024-02-26T10:38:35.192814+0000","last_unstale":"2024-02-26T10:38:37.235257+0000","last_undegraded":"2024-02-26T10:38:37.235257+0000","last_fullsized":"2024-02-26T10:38:37.235257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235534+0000","last_change":"2024-02-26T10:38:35.188741+0000","last_active":"2024-02-26T10:38:37.235534+0000","last_peered":"2024-02-26T10:38:37.235534+0000","last_clean":"2024-02-26T10:38:37.235534+0000","last_became_active":"2024-02-26T10:38:35.188296+0000","last_became_peered":"2024-02-26T10:38:35.188296+0000","last_unstale":"2024-02-26T10:38:37.235534+0000","last_undegraded":"2024-02-26T10:38:37.235534+0000","last_fullsized":"2024-02-26T10:38:37.235534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235999+0000","last_change":"2024-02-26T10:38:35.193665+0000","last_active":"2024-02-26T10:38:37.235999+0000","last_peered":"2024-02-26T10:38:37.235999+0000","last_clean":"2024-02-26T10:38:37.235999+0000","last_became_active":"2024-02-26T10:38:35.193314+0000","last_became_peered":"2024-02-26T10:38:35.193314+0000","last_unstale":"2024-02-26T10:38:37.235999+0000","last_undegraded":"2024-02-26T10:38:37.235999+0000","last_fullsized":"2024-02-26T10:38:37.235999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236084+0000","last_change":"2024-02-26T10:38:35.186570+0000","last_active":"2024-02-26T10:38:37.236084+0000","last_peered":"2024-02-26T10:38:37.236084+0000","last_clean":"2024-02-26T10:38:37.236084+0000","last_became_active":"2024-02-26T10:38:35.186248+0000","last_became_peered":"2024-02-26T10:38:35.186248+0000","last_unstale":"2024-02-26T10:38:37.236084+0000","last_undegraded":"2024-02-26T10:38:37.236084+0000","last_fullsized":"2024-02-26T10:38:37.236084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182707+0000","last_change":"2024-02-26T10:38:35.197545+0000","last_active":"2024-02-26T10:38:36.182707+0000","last_peered":"2024-02-26T10:38:36.182707+0000","last_clean":"2024-02-26T10:38:36.182707+0000","last_became_active":"2024-02-26T10:38:35.197220+0000","last_became_peered":"2024-02-26T10:38:35.197220+0000","last_unstale":"2024-02-26T10:38:36.182707+0000","last_undegraded":"2024-02-26T10:38:36.182707+0000","last_fullsized":"2024-02-26T10:38:36.182707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173665+0000","last_change":"2024-02-26T10:38:35.174943+0000","last_active":"2024-02-26T10:38:36.173665+0000","last_peered":"2024-02-26T10:38:36.173665+0000","last_clean":"2024-02-26T10:38:36.173665+0000","last_became_active":"2024-02-26T10:38:35.174618+0000","last_became_peered":"2024-02-26T10:38:35.174618+0000","last_unstale":"2024-02-26T10:38:36.173665+0000","last_undegraded":"2024-02-26T10:38:36.173665+0000","last_fullsized":"2024-02-26T10:38:36.173665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235860+0000","last_change":"2024-02-26T10:38:35.188806+0000","last_active":"2024-02-26T10:38:37.235860+0000","last_peered":"2024-02-26T10:38:37.235860+0000","last_clean":"2024-02-26T10:38:37.235860+0000","last_became_active":"2024-02-26T10:38:35.188539+0000","last_became_peered":"2024-02-26T10:38:35.188539+0000","last_unstale":"2024-02-26T10:38:37.235860+0000","last_undegraded":"2024-02-26T10:38:37.235860+0000","last_fullsized":"2024-02-26T10:38:37.235860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178458+0000","last_change":"2024-02-26T10:38:35.183491+0000","last_active":"2024-02-26T10:38:36.178458+0000","last_peered":"2024-02-26T10:38:36.178458+0000","last_clean":"2024-02-26T10:38:36.178458+0000","last_became_active":"2024-02-26T10:38:35.183153+0000","last_became_peered":"2024-02-26T10:38:35.183153+0000","last_unstale":"2024-02-26T10:38:36.178458+0000","last_undegraded":"2024-02-26T10:38:36.178458+0000","last_fullsized":"2024-02-26T10:38:36.178458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235539+0000","last_change":"2024-02-26T10:38:35.185342+0000","last_active":"2024-02-26T10:38:37.235539+0000","last_peered":"2024-02-26T10:38:37.235539+0000","last_clean":"2024-02-26T10:38:37.235539+0000","last_became_active":"2024-02-26T10:38:35.185067+0000","last_became_peered":"2024-02-26T10:38:35.185067+0000","last_unstale":"2024-02-26T10:38:37.235539+0000","last_undegraded":"2024-02-26T10:38:37.235539+0000","last_fullsized":"2024-02-26T10:38:37.235539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232756+0000","last_change":"2024-02-26T10:38:35.184916+0000","last_active":"2024-02-26T10:38:37.232756+0000","last_peered":"2024-02-26T10:38:37.232756+0000","last_clean":"2024-02-26T10:38:37.232756+0000","last_became_active":"2024-02-26T10:38:35.184582+0000","last_became_peered":"2024-02-26T10:38:35.184582+0000","last_unstale":"2024-02-26T10:38:37.232756+0000","last_undegraded":"2024-02-26T10:38:37.232756+0000","last_fullsized":"2024-02-26T10:38:37.232756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:01.111739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181690+0000","last_change":"2024-02-26T10:38:35.191702+0000","last_active":"2024-02-26T10:38:36.181690+0000","last_peered":"2024-02-26T10:38:36.181690+0000","last_clean":"2024-02-26T10:38:36.181690+0000","last_became_active":"2024-02-26T10:38:35.191416+0000","last_became_peered":"2024-02-26T10:38:35.191416+0000","last_unstale":"2024-02-26T10:38:36.181690+0000","last_undegraded":"2024-02-26T10:38:36.181690+0000","last_fullsized":"2024-02-26T10:38:36.181690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:36:38.515074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175512+0000","last_change":"2024-02-26T10:38:35.169056+0000","last_active":"2024-02-26T10:38:36.175512+0000","last_peered":"2024-02-26T10:38:36.175512+0000","last_clean":"2024-02-26T10:38:36.175512+0000","last_became_active":"2024-02-26T10:38:35.168640+0000","last_became_peered":"2024-02-26T10:38:35.168640+0000","last_unstale":"2024-02-26T10:38:36.175512+0000","last_undegraded":"2024-02-26T10:38:36.175512+0000","last_fullsized":"2024-02-26T10:38:36.175512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:44.237255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235514+0000","last_change":"2024-02-26T10:38:34.169112+0000","last_active":"2024-02-26T10:38:37.235514+0000","last_peered":"2024-02-26T10:38:37.235514+0000","last_clean":"2024-02-26T10:38:37.235514+0000","last_became_active":"2024-02-26T10:38:34.168796+0000","last_became_peered":"2024-02-26T10:38:34.168796+0000","last_unstale":"2024-02-26T10:38:37.235514+0000","last_undegraded":"2024-02-26T10:38:37.235514+0000","last_fullsized":"2024-02-26T10:38:37.235514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:29:24.719556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178388+0000","last_change":"2024-02-26T10:38:35.187182+0000","last_active":"2024-02-26T10:38:36.178388+0000","last_peered":"2024-02-26T10:38:36.178388+0000","last_clean":"2024-02-26T10:38:36.178388+0000","last_became_active":"2024-02-26T10:38:35.185661+0000","last_became_peered":"2024-02-26T10:38:35.185661+0000","last_unstale":"2024-02-26T10:38:36.178388+0000","last_undegraded":"2024-02-26T10:38:36.178388+0000","last_fullsized":"2024-02-26T10:38:36.178388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181759+0000","last_change":"2024-02-26T10:38:34.179842+0000","last_active":"2024-02-26T10:38:36.181759+0000","last_peered":"2024-02-26T10:38:36.181759+0000","last_clean":"2024-02-26T10:38:36.181759+0000","last_became_active":"2024-02-26T10:38:34.179535+0000","last_became_peered":"2024-02-26T10:38:34.179535+0000","last_unstale":"2024-02-26T10:38:36.181759+0000","last_undegraded":"2024-02-26T10:38:36.181759+0000","last_fullsized":"2024-02-26T10:38:36.181759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235171+0000","last_change":"2024-02-26T10:38:34.167613+0000","last_active":"2024-02-26T10:38:37.235171+0000","last_peered":"2024-02-26T10:38:37.235171+0000","last_clean":"2024-02-26T10:38:37.235171+0000","last_became_active":"2024-02-26T10:38:34.167252+0000","last_became_peered":"2024-02-26T10:38:34.167252+0000","last_unstale":"2024-02-26T10:38:37.235171+0000","last_undegraded":"2024-02-26T10:38:37.235171+0000","last_fullsized":"2024-02-26T10:38:37.235171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235040+0000","last_change":"2024-02-26T10:38:35.185939+0000","last_active":"2024-02-26T10:38:37.235040+0000","last_peered":"2024-02-26T10:38:37.235040+0000","last_clean":"2024-02-26T10:38:37.235040+0000","last_became_active":"2024-02-26T10:38:35.183226+0000","last_became_peered":"2024-02-26T10:38:35.183226+0000","last_unstale":"2024-02-26T10:38:37.235040+0000","last_undegraded":"2024-02-26T10:38:37.235040+0000","last_fullsized":"2024-02-26T10:38:37.235040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:02.482956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236134+0000","last_change":"2024-02-26T10:38:34.186454+0000","last_active":"2024-02-26T10:38:37.236134+0000","last_peered":"2024-02-26T10:38:37.236134+0000","last_clean":"2024-02-26T10:38:37.236134+0000","last_became_active":"2024-02-26T10:38:34.185676+0000","last_became_peered":"2024-02-26T10:38:34.185676+0000","last_unstale":"2024-02-26T10:38:37.236134+0000","last_undegraded":"2024-02-26T10:38:37.236134+0000","last_fullsized":"2024-02-26T10:38:37.236134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173408+0000","last_change":"2024-02-26T10:38:35.176977+0000","last_active":"2024-02-26T10:38:36.173408+0000","last_peered":"2024-02-26T10:38:36.173408+0000","last_clean":"2024-02-26T10:38:36.173408+0000","last_became_active":"2024-02-26T10:38:35.175624+0000","last_became_peered":"2024-02-26T10:38:35.175624+0000","last_unstale":"2024-02-26T10:38:36.173408+0000","last_undegraded":"2024-02-26T10:38:36.173408+0000","last_fullsized":"2024-02-26T10:38:36.173408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182463+0000","last_change":"2024-02-26T10:38:34.188391+0000","last_active":"2024-02-26T10:38:36.182463+0000","last_peered":"2024-02-26T10:38:36.182463+0000","last_clean":"2024-02-26T10:38:36.182463+0000","last_became_active":"2024-02-26T10:38:34.188077+0000","last_became_peered":"2024-02-26T10:38:34.188077+0000","last_unstale":"2024-02-26T10:38:36.182463+0000","last_undegraded":"2024-02-26T10:38:36.182463+0000","last_fullsized":"2024-02-26T10:38:36.182463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236526+0000","last_change":"2024-02-26T10:38:35.199765+0000","last_active":"2024-02-26T10:38:37.236526+0000","last_peered":"2024-02-26T10:38:37.236526+0000","last_clean":"2024-02-26T10:38:37.236526+0000","last_became_active":"2024-02-26T10:38:35.199478+0000","last_became_peered":"2024-02-26T10:38:35.199478+0000","last_unstale":"2024-02-26T10:38:37.236526+0000","last_undegraded":"2024-02-26T10:38:37.236526+0000","last_fullsized":"2024-02-26T10:38:37.236526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:41.134017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236673+0000","last_change":"2024-02-26T10:38:34.189159+0000","last_active":"2024-02-26T10:38:37.236673+0000","last_peered":"2024-02-26T10:38:37.236673+0000","last_clean":"2024-02-26T10:38:37.236673+0000","last_became_active":"2024-02-26T10:38:34.187774+0000","last_became_peered":"2024-02-26T10:38:34.187774+0000","last_unstale":"2024-02-26T10:38:37.236673+0000","last_undegraded":"2024-02-26T10:38:37.236673+0000","last_fullsized":"2024-02-26T10:38:37.236673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178929+0000","last_change":"2024-02-26T10:38:35.198495+0000","last_active":"2024-02-26T10:38:36.178929+0000","last_peered":"2024-02-26T10:38:36.178929+0000","last_clean":"2024-02-26T10:38:36.178929+0000","last_became_active":"2024-02-26T10:38:35.198049+0000","last_became_peered":"2024-02-26T10:38:35.198049+0000","last_unstale":"2024-02-26T10:38:36.178929+0000","last_undegraded":"2024-02-26T10:38:36.178929+0000","last_fullsized":"2024-02-26T10:38:36.178929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:05:30.706442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173743+0000","last_change":"2024-02-26T10:38:34.182673+0000","last_active":"2024-02-26T10:38:36.173743+0000","last_peered":"2024-02-26T10:38:36.173743+0000","last_clean":"2024-02-26T10:38:36.173743+0000","last_became_active":"2024-02-26T10:38:34.182369+0000","last_became_peered":"2024-02-26T10:38:34.182369+0000","last_unstale":"2024-02-26T10:38:36.173743+0000","last_undegraded":"2024-02-26T10:38:36.173743+0000","last_fullsized":"2024-02-26T10:38:36.173743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175617+0000","last_change":"2024-02-26T10:38:35.193526+0000","last_active":"2024-02-26T10:38:36.175617+0000","last_peered":"2024-02-26T10:38:36.175617+0000","last_clean":"2024-02-26T10:38:36.175617+0000","last_became_active":"2024-02-26T10:38:35.193192+0000","last_became_peered":"2024-02-26T10:38:35.193192+0000","last_unstale":"2024-02-26T10:38:36.175617+0000","last_undegraded":"2024-02-26T10:38:36.175617+0000","last_fullsized":"2024-02-26T10:38:36.175617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:08.969223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182341+0000","last_change":"2024-02-26T10:38:34.178005+0000","last_active":"2024-02-26T10:38:36.182341+0000","last_peered":"2024-02-26T10:38:36.182341+0000","last_clean":"2024-02-26T10:38:36.182341+0000","last_became_active":"2024-02-26T10:38:34.177684+0000","last_became_peered":"2024-02-26T10:38:34.177684+0000","last_unstale":"2024-02-26T10:38:36.182341+0000","last_undegraded":"2024-02-26T10:38:36.182341+0000","last_fullsized":"2024-02-26T10:38:36.182341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235427+0000","last_change":"2024-02-26T10:38:35.196333+0000","last_active":"2024-02-26T10:38:37.235427+0000","last_peered":"2024-02-26T10:38:37.235427+0000","last_clean":"2024-02-26T10:38:37.235427+0000","last_became_active":"2024-02-26T10:38:35.191241+0000","last_became_peered":"2024-02-26T10:38:35.191241+0000","last_unstale":"2024-02-26T10:38:37.235427+0000","last_undegraded":"2024-02-26T10:38:37.235427+0000","last_fullsized":"2024-02-26T10:38:37.235427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:49:24.751331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178541+0000","last_change":"2024-02-26T10:38:34.178253+0000","last_active":"2024-02-26T10:38:36.178541+0000","last_peered":"2024-02-26T10:38:36.178541+0000","last_clean":"2024-02-26T10:38:36.178541+0000","last_became_active":"2024-02-26T10:38:34.177880+0000","last_became_peered":"2024-02-26T10:38:34.177880+0000","last_unstale":"2024-02-26T10:38:36.178541+0000","last_undegraded":"2024-02-26T10:38:36.178541+0000","last_fullsized":"2024-02-26T10:38:36.178541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.177760+0000","last_change":"2024-02-26T10:38:35.196164+0000","last_active":"2024-02-26T10:38:36.177760+0000","last_peered":"2024-02-26T10:38:36.177760+0000","last_clean":"2024-02-26T10:38:36.177760+0000","last_became_active":"2024-02-26T10:38:35.195846+0000","last_became_peered":"2024-02-26T10:38:35.195846+0000","last_unstale":"2024-02-26T10:38:36.177760+0000","last_undegraded":"2024-02-26T10:38:36.177760+0000","last_fullsized":"2024-02-26T10:38:36.177760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:36.847842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235392+0000","last_change":"2024-02-26T10:38:34.180900+0000","last_active":"2024-02-26T10:38:37.235392+0000","last_peered":"2024-02-26T10:38:37.235392+0000","last_clean":"2024-02-26T10:38:37.235392+0000","last_became_active":"2024-02-26T10:38:34.180316+0000","last_became_peered":"2024-02-26T10:38:34.180316+0000","last_unstale":"2024-02-26T10:38:37.235392+0000","last_undegraded":"2024-02-26T10:38:37.235392+0000","last_fullsized":"2024-02-26T10:38:37.235392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235901+0000","last_change":"2024-02-26T10:38:35.188962+0000","last_active":"2024-02-26T10:38:37.235901+0000","last_peered":"2024-02-26T10:38:37.235901+0000","last_clean":"2024-02-26T10:38:37.235901+0000","last_became_active":"2024-02-26T10:38:35.188506+0000","last_became_peered":"2024-02-26T10:38:35.188506+0000","last_unstale":"2024-02-26T10:38:37.235901+0000","last_undegraded":"2024-02-26T10:38:37.235901+0000","last_fullsized":"2024-02-26T10:38:37.235901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181772+0000","last_change":"2024-02-26T10:38:34.180239+0000","last_active":"2024-02-26T10:38:36.181772+0000","last_peered":"2024-02-26T10:38:36.181772+0000","last_clean":"2024-02-26T10:38:36.181772+0000","last_became_active":"2024-02-26T10:38:34.179109+0000","last_became_peered":"2024-02-26T10:38:34.179109+0000","last_unstale":"2024-02-26T10:38:36.181772+0000","last_undegraded":"2024-02-26T10:38:36.181772+0000","last_fullsized":"2024-02-26T10:38:36.181772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.181887+0000","last_change":"2024-02-26T10:38:35.196600+0000","last_active":"2024-02-26T10:38:36.181887+0000","last_peered":"2024-02-26T10:38:36.181887+0000","last_clean":"2024-02-26T10:38:36.181887+0000","last_became_active":"2024-02-26T10:38:35.192063+0000","last_became_peered":"2024-02-26T10:38:35.192063+0000","last_unstale":"2024-02-26T10:38:36.181887+0000","last_undegraded":"2024-02-26T10:38:36.181887+0000","last_fullsized":"2024-02-26T10:38:36.181887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:18:01.128987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232648+0000","last_change":"2024-02-26T10:38:34.159995+0000","last_active":"2024-02-26T10:38:37.232648+0000","last_peered":"2024-02-26T10:38:37.232648+0000","last_clean":"2024-02-26T10:38:37.232648+0000","last_became_active":"2024-02-26T10:38:34.159031+0000","last_became_peered":"2024-02-26T10:38:34.159031+0000","last_unstale":"2024-02-26T10:38:37.232648+0000","last_undegraded":"2024-02-26T10:38:37.232648+0000","last_fullsized":"2024-02-26T10:38:37.232648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:32:57.566992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237658+0000","last_change":"2024-02-26T10:38:35.190544+0000","last_active":"2024-02-26T10:38:37.237658+0000","last_peered":"2024-02-26T10:38:37.237658+0000","last_clean":"2024-02-26T10:38:37.237658+0000","last_became_active":"2024-02-26T10:38:35.190066+0000","last_became_peered":"2024-02-26T10:38:35.190066+0000","last_unstale":"2024-02-26T10:38:37.237658+0000","last_undegraded":"2024-02-26T10:38:37.237658+0000","last_fullsized":"2024-02-26T10:38:37.237658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175919+0000","last_change":"2024-02-26T10:38:34.165622+0000","last_active":"2024-02-26T10:38:36.175919+0000","last_peered":"2024-02-26T10:38:36.175919+0000","last_clean":"2024-02-26T10:38:36.175919+0000","last_became_active":"2024-02-26T10:38:34.165165+0000","last_became_peered":"2024-02-26T10:38:34.165165+0000","last_unstale":"2024-02-26T10:38:36.175919+0000","last_undegraded":"2024-02-26T10:38:36.175919+0000","last_fullsized":"2024-02-26T10:38:36.175919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:17:02.556621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236130+0000","last_change":"2024-02-26T10:38:35.191334+0000","last_active":"2024-02-26T10:38:37.236130+0000","last_peered":"2024-02-26T10:38:37.236130+0000","last_clean":"2024-02-26T10:38:37.236130+0000","last_became_active":"2024-02-26T10:38:35.187187+0000","last_became_peered":"2024-02-26T10:38:35.187187+0000","last_unstale":"2024-02-26T10:38:37.236130+0000","last_undegraded":"2024-02-26T10:38:37.236130+0000","last_fullsized":"2024-02-26T10:38:37.236130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233476+0000","last_change":"2024-02-26T10:38:34.162987+0000","last_active":"2024-02-26T10:38:37.233476+0000","last_peered":"2024-02-26T10:38:37.233476+0000","last_clean":"2024-02-26T10:38:37.233476+0000","last_became_active":"2024-02-26T10:38:34.160976+0000","last_became_peered":"2024-02-26T10:38:34.160976+0000","last_unstale":"2024-02-26T10:38:37.233476+0000","last_undegraded":"2024-02-26T10:38:37.233476+0000","last_fullsized":"2024-02-26T10:38:37.233476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:10.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.238041+0000","last_change":"2024-02-26T10:38:35.189775+0000","last_active":"2024-02-26T10:38:37.238041+0000","last_peered":"2024-02-26T10:38:37.238041+0000","last_clean":"2024-02-26T10:38:37.238041+0000","last_became_active":"2024-02-26T10:38:35.189470+0000","last_became_peered":"2024-02-26T10:38:35.189470+0000","last_unstale":"2024-02-26T10:38:37.238041+0000","last_undegraded":"2024-02-26T10:38:37.238041+0000","last_fullsized":"2024-02-26T10:38:37.238041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:56:01.655512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174043+0000","last_change":"2024-02-26T10:38:34.180005+0000","last_active":"2024-02-26T10:38:36.174043+0000","last_peered":"2024-02-26T10:38:36.174043+0000","last_clean":"2024-02-26T10:38:36.174043+0000","last_became_active":"2024-02-26T10:38:34.179694+0000","last_became_peered":"2024-02-26T10:38:34.179694+0000","last_unstale":"2024-02-26T10:38:36.174043+0000","last_undegraded":"2024-02-26T10:38:36.174043+0000","last_fullsized":"2024-02-26T10:38:36.174043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236746+0000","last_change":"2024-02-26T10:38:35.190450+0000","last_active":"2024-02-26T10:38:37.236746+0000","last_peered":"2024-02-26T10:38:37.236746+0000","last_clean":"2024-02-26T10:38:37.236746+0000","last_became_active":"2024-02-26T10:38:35.187303+0000","last_became_peered":"2024-02-26T10:38:35.187303+0000","last_unstale":"2024-02-26T10:38:37.236746+0000","last_undegraded":"2024-02-26T10:38:37.236746+0000","last_fullsized":"2024-02-26T10:38:37.236746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:05:44.100068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.232966+0000","last_change":"2024-02-26T10:38:34.183223+0000","last_active":"2024-02-26T10:38:37.232966+0000","last_peered":"2024-02-26T10:38:37.232966+0000","last_clean":"2024-02-26T10:38:37.232966+0000","last_became_active":"2024-02-26T10:38:34.182882+0000","last_became_peered":"2024-02-26T10:38:34.182882+0000","last_unstale":"2024-02-26T10:38:37.232966+0000","last_undegraded":"2024-02-26T10:38:37.232966+0000","last_fullsized":"2024-02-26T10:38:37.232966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:35:22.624584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235843+0000","last_change":"2024-02-26T10:38:35.196564+0000","last_active":"2024-02-26T10:38:37.235843+0000","last_peered":"2024-02-26T10:38:37.235843+0000","last_clean":"2024-02-26T10:38:37.235843+0000","last_became_active":"2024-02-26T10:38:35.196286+0000","last_became_peered":"2024-02-26T10:38:35.196286+0000","last_unstale":"2024-02-26T10:38:37.235843+0000","last_undegraded":"2024-02-26T10:38:37.235843+0000","last_fullsized":"2024-02-26T10:38:37.235843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:46.451631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235369+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:38:37.235369+0000","last_peered":"2024-02-26T10:38:37.235369+0000","last_clean":"2024-02-26T10:38:37.235369+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:38:37.235369+0000","last_undegraded":"2024-02-26T10:38:37.235369+0000","last_fullsized":"2024-02-26T10:38:37.235369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234740+0000","last_change":"2024-02-26T10:38:34.181165+0000","last_active":"2024-02-26T10:38:37.234740+0000","last_peered":"2024-02-26T10:38:37.234740+0000","last_clean":"2024-02-26T10:38:37.234740+0000","last_became_active":"2024-02-26T10:38:34.180374+0000","last_became_peered":"2024-02-26T10:38:34.180374+0000","last_unstale":"2024-02-26T10:38:37.234740+0000","last_undegraded":"2024-02-26T10:38:37.234740+0000","last_fullsized":"2024-02-26T10:38:37.234740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:55:06.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235370+0000","last_change":"2024-02-26T10:38:35.205573+0000","last_active":"2024-02-26T10:38:37.235370+0000","last_peered":"2024-02-26T10:38:37.235370+0000","last_clean":"2024-02-26T10:38:37.235370+0000","last_became_active":"2024-02-26T10:38:35.205295+0000","last_became_peered":"2024-02-26T10:38:35.205295+0000","last_unstale":"2024-02-26T10:38:37.235370+0000","last_undegraded":"2024-02-26T10:38:37.235370+0000","last_fullsized":"2024-02-26T10:38:37.235370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234753+0000","last_change":"2024-02-26T10:38:34.177423+0000","last_active":"2024-02-26T10:38:37.234753+0000","last_peered":"2024-02-26T10:38:37.234753+0000","last_clean":"2024-02-26T10:38:37.234753+0000","last_became_active":"2024-02-26T10:38:34.175531+0000","last_became_peered":"2024-02-26T10:38:34.175531+0000","last_unstale":"2024-02-26T10:38:37.234753+0000","last_undegraded":"2024-02-26T10:38:37.234753+0000","last_fullsized":"2024-02-26T10:38:37.234753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.176766+0000","last_change":"2024-02-26T10:38:35.182828+0000","last_active":"2024-02-26T10:38:36.176766+0000","last_peered":"2024-02-26T10:38:36.176766+0000","last_clean":"2024-02-26T10:38:36.176766+0000","last_became_active":"2024-02-26T10:38:35.182453+0000","last_became_peered":"2024-02-26T10:38:35.182453+0000","last_unstale":"2024-02-26T10:38:36.176766+0000","last_undegraded":"2024-02-26T10:38:36.176766+0000","last_fullsized":"2024-02-26T10:38:36.176766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:16:15.325827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.233008+0000","last_change":"2024-02-26T10:38:34.162528+0000","last_active":"2024-02-26T10:38:37.233008+0000","last_peered":"2024-02-26T10:38:37.233008+0000","last_clean":"2024-02-26T10:38:37.233008+0000","last_became_active":"2024-02-26T10:38:34.161419+0000","last_became_peered":"2024-02-26T10:38:34.161419+0000","last_unstale":"2024-02-26T10:38:37.233008+0000","last_undegraded":"2024-02-26T10:38:37.233008+0000","last_fullsized":"2024-02-26T10:38:37.233008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:12.268907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173393+0000","last_change":"2024-02-26T10:38:35.180693+0000","last_active":"2024-02-26T10:38:36.173393+0000","last_peered":"2024-02-26T10:38:36.173393+0000","last_clean":"2024-02-26T10:38:36.173393+0000","last_became_active":"2024-02-26T10:38:35.178787+0000","last_became_peered":"2024-02-26T10:38:35.178787+0000","last_unstale":"2024-02-26T10:38:36.173393+0000","last_undegraded":"2024-02-26T10:38:36.173393+0000","last_fullsized":"2024-02-26T10:38:36.173393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235683+0000","last_change":"2024-02-26T10:38:34.184572+0000","last_active":"2024-02-26T10:38:37.235683+0000","last_peered":"2024-02-26T10:38:37.235683+0000","last_clean":"2024-02-26T10:38:37.235683+0000","last_became_active":"2024-02-26T10:38:34.183650+0000","last_became_peered":"2024-02-26T10:38:34.183650+0000","last_unstale":"2024-02-26T10:38:37.235683+0000","last_undegraded":"2024-02-26T10:38:37.235683+0000","last_fullsized":"2024-02-26T10:38:37.235683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175481+0000","last_change":"2024-02-26T10:38:35.186511+0000","last_active":"2024-02-26T10:38:36.175481+0000","last_peered":"2024-02-26T10:38:36.175481+0000","last_clean":"2024-02-26T10:38:36.175481+0000","last_became_active":"2024-02-26T10:38:35.185321+0000","last_became_peered":"2024-02-26T10:38:35.185321+0000","last_unstale":"2024-02-26T10:38:36.175481+0000","last_undegraded":"2024-02-26T10:38:36.175481+0000","last_fullsized":"2024-02-26T10:38:36.175481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:18.018830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174062+0000","last_change":"2024-02-26T10:38:34.165105+0000","last_active":"2024-02-26T10:38:36.174062+0000","last_peered":"2024-02-26T10:38:36.174062+0000","last_clean":"2024-02-26T10:38:36.174062+0000","last_became_active":"2024-02-26T10:38:34.164756+0000","last_became_peered":"2024-02-26T10:38:34.164756+0000","last_unstale":"2024-02-26T10:38:36.174062+0000","last_undegraded":"2024-02-26T10:38:36.174062+0000","last_fullsized":"2024-02-26T10:38:36.174062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:55:16.827326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237188+0000","last_change":"2024-02-26T10:38:35.191275+0000","last_active":"2024-02-26T10:38:37.237188+0000","last_peered":"2024-02-26T10:38:37.237188+0000","last_clean":"2024-02-26T10:38:37.237188+0000","last_became_active":"2024-02-26T10:38:35.190875+0000","last_became_peered":"2024-02-26T10:38:35.190875+0000","last_unstale":"2024-02-26T10:38:37.237188+0000","last_undegraded":"2024-02-26T10:38:37.237188+0000","last_fullsized":"2024-02-26T10:38:37.237188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:39:28.214994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.237699+0000","last_change":"2024-02-26T10:38:34.176727+0000","last_active":"2024-02-26T10:38:37.237699+0000","last_peered":"2024-02-26T10:38:37.237699+0000","last_clean":"2024-02-26T10:38:37.237699+0000","last_became_active":"2024-02-26T10:38:34.176422+0000","last_became_peered":"2024-02-26T10:38:34.176422+0000","last_unstale":"2024-02-26T10:38:37.237699+0000","last_undegraded":"2024-02-26T10:38:37.237699+0000","last_fullsized":"2024-02-26T10:38:37.237699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:03.697301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174128+0000","last_change":"2024-02-26T10:38:35.174940+0000","last_active":"2024-02-26T10:38:36.174128+0000","last_peered":"2024-02-26T10:38:36.174128+0000","last_clean":"2024-02-26T10:38:36.174128+0000","last_became_active":"2024-02-26T10:38:35.174609+0000","last_became_peered":"2024-02-26T10:38:35.174609+0000","last_unstale":"2024-02-26T10:38:36.174128+0000","last_undegraded":"2024-02-26T10:38:36.174128+0000","last_fullsized":"2024-02-26T10:38:36.174128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.174557+0000","last_change":"2024-02-26T10:38:34.176149+0000","last_active":"2024-02-26T10:38:36.174557+0000","last_peered":"2024-02-26T10:38:36.174557+0000","last_clean":"2024-02-26T10:38:36.174557+0000","last_became_active":"2024-02-26T10:38:34.175800+0000","last_became_peered":"2024-02-26T10:38:34.175800+0000","last_unstale":"2024-02-26T10:38:36.174557+0000","last_undegraded":"2024-02-26T10:38:36.174557+0000","last_fullsized":"2024-02-26T10:38:36.174557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236113+0000","last_change":"2024-02-26T10:38:35.191410+0000","last_active":"2024-02-26T10:38:37.236113+0000","last_peered":"2024-02-26T10:38:37.236113+0000","last_clean":"2024-02-26T10:38:37.236113+0000","last_became_active":"2024-02-26T10:38:35.190954+0000","last_became_peered":"2024-02-26T10:38:35.190954+0000","last_unstale":"2024-02-26T10:38:37.236113+0000","last_undegraded":"2024-02-26T10:38:37.236113+0000","last_fullsized":"2024-02-26T10:38:37.236113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182629+0000","last_change":"2024-02-26T10:38:34.185447+0000","last_active":"2024-02-26T10:38:36.182629+0000","last_peered":"2024-02-26T10:38:36.182629+0000","last_clean":"2024-02-26T10:38:36.182629+0000","last_became_active":"2024-02-26T10:38:34.185054+0000","last_became_peered":"2024-02-26T10:38:34.185054+0000","last_unstale":"2024-02-26T10:38:36.182629+0000","last_undegraded":"2024-02-26T10:38:36.182629+0000","last_fullsized":"2024-02-26T10:38:36.182629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:10:08.425439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179198+0000","last_change":"2024-02-26T10:38:35.181510+0000","last_active":"2024-02-26T10:38:36.179198+0000","last_peered":"2024-02-26T10:38:36.179198+0000","last_clean":"2024-02-26T10:38:36.179198+0000","last_became_active":"2024-02-26T10:38:35.180830+0000","last_became_peered":"2024-02-26T10:38:35.180830+0000","last_unstale":"2024-02-26T10:38:36.179198+0000","last_undegraded":"2024-02-26T10:38:36.179198+0000","last_fullsized":"2024-02-26T10:38:36.179198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.234630+0000","last_change":"2024-02-26T10:38:34.174303+0000","last_active":"2024-02-26T10:38:37.234630+0000","last_peered":"2024-02-26T10:38:37.234630+0000","last_clean":"2024-02-26T10:38:37.234630+0000","last_became_active":"2024-02-26T10:38:34.174006+0000","last_became_peered":"2024-02-26T10:38:34.174006+0000","last_unstale":"2024-02-26T10:38:37.234630+0000","last_undegraded":"2024-02-26T10:38:37.234630+0000","last_fullsized":"2024-02-26T10:38:37.234630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:41:07.777719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.175416+0000","last_change":"2024-02-26T10:38:35.176832+0000","last_active":"2024-02-26T10:38:36.175416+0000","last_peered":"2024-02-26T10:38:36.175416+0000","last_clean":"2024-02-26T10:38:36.175416+0000","last_became_active":"2024-02-26T10:38:35.176305+0000","last_became_peered":"2024-02-26T10:38:35.176305+0000","last_unstale":"2024-02-26T10:38:36.175416+0000","last_undegraded":"2024-02-26T10:38:36.175416+0000","last_fullsized":"2024-02-26T10:38:36.175416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:06.286416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173704+0000","last_change":"2024-02-26T10:38:34.169425+0000","last_active":"2024-02-26T10:38:36.173704+0000","last_peered":"2024-02-26T10:38:36.173704+0000","last_clean":"2024-02-26T10:38:36.173704+0000","last_became_active":"2024-02-26T10:38:34.168217+0000","last_became_peered":"2024-02-26T10:38:34.168217+0000","last_unstale":"2024-02-26T10:38:36.173704+0000","last_undegraded":"2024-02-26T10:38:36.173704+0000","last_fullsized":"2024-02-26T10:38:36.173704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236029+0000","last_change":"2024-02-26T10:38:35.172836+0000","last_active":"2024-02-26T10:38:37.236029+0000","last_peered":"2024-02-26T10:38:37.236029+0000","last_clean":"2024-02-26T10:38:37.236029+0000","last_became_active":"2024-02-26T10:38:35.172554+0000","last_became_peered":"2024-02-26T10:38:35.172554+0000","last_unstale":"2024-02-26T10:38:37.236029+0000","last_undegraded":"2024-02-26T10:38:37.236029+0000","last_fullsized":"2024-02-26T10:38:37.236029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:27:50.204495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.173693+0000","last_change":"2024-02-26T10:38:34.166377+0000","last_active":"2024-02-26T10:38:36.173693+0000","last_peered":"2024-02-26T10:38:36.173693+0000","last_clean":"2024-02-26T10:38:36.173693+0000","last_became_active":"2024-02-26T10:38:34.165963+0000","last_became_peered":"2024-02-26T10:38:34.165963+0000","last_unstale":"2024-02-26T10:38:36.173693+0000","last_undegraded":"2024-02-26T10:38:36.173693+0000","last_fullsized":"2024-02-26T10:38:36.173693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:05.094912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235575+0000","last_change":"2024-02-26T10:38:35.185120+0000","last_active":"2024-02-26T10:38:37.235575+0000","last_peered":"2024-02-26T10:38:37.235575+0000","last_clean":"2024-02-26T10:38:37.235575+0000","last_became_active":"2024-02-26T10:38:35.184832+0000","last_became_peered":"2024-02-26T10:38:35.184832+0000","last_unstale":"2024-02-26T10:38:37.235575+0000","last_undegraded":"2024-02-26T10:38:37.235575+0000","last_fullsized":"2024-02-26T10:38:37.235575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:08.501176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236461+0000","last_change":"2024-02-26T10:38:34.180225+0000","last_active":"2024-02-26T10:38:37.236461+0000","last_peered":"2024-02-26T10:38:37.236461+0000","last_clean":"2024-02-26T10:38:37.236461+0000","last_became_active":"2024-02-26T10:38:34.179909+0000","last_became_peered":"2024-02-26T10:38:34.179909+0000","last_unstale":"2024-02-26T10:38:37.236461+0000","last_undegraded":"2024-02-26T10:38:37.236461+0000","last_fullsized":"2024-02-26T10:38:37.236461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:05:58.590708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.182262+0000","last_change":"2024-02-26T10:38:35.188975+0000","last_active":"2024-02-26T10:38:36.182262+0000","last_peered":"2024-02-26T10:38:36.182262+0000","last_clean":"2024-02-26T10:38:36.182262+0000","last_became_active":"2024-02-26T10:38:35.188608+0000","last_became_peered":"2024-02-26T10:38:35.188608+0000","last_unstale":"2024-02-26T10:38:36.182262+0000","last_undegraded":"2024-02-26T10:38:36.182262+0000","last_fullsized":"2024-02-26T10:38:36.182262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:41:45.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236117+0000","last_change":"2024-02-26T10:38:35.196289+0000","last_active":"2024-02-26T10:38:37.236117+0000","last_peered":"2024-02-26T10:38:37.236117+0000","last_clean":"2024-02-26T10:38:37.236117+0000","last_became_active":"2024-02-26T10:38:35.194751+0000","last_became_peered":"2024-02-26T10:38:35.194751+0000","last_unstale":"2024-02-26T10:38:37.236117+0000","last_undegraded":"2024-02-26T10:38:37.236117+0000","last_fullsized":"2024-02-26T10:38:37.236117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:21:23.591439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236713+0000","last_change":"2024-02-26T10:38:34.186760+0000","last_active":"2024-02-26T10:38:37.236713+0000","last_peered":"2024-02-26T10:38:37.236713+0000","last_clean":"2024-02-26T10:38:37.236713+0000","last_became_active":"2024-02-26T10:38:34.186428+0000","last_became_peered":"2024-02-26T10:38:34.186428+0000","last_unstale":"2024-02-26T10:38:37.236713+0000","last_undegraded":"2024-02-26T10:38:37.236713+0000","last_fullsized":"2024-02-26T10:38:37.236713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:27:00.650843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.236518+0000","last_change":"2024-02-26T10:38:35.202616+0000","last_active":"2024-02-26T10:38:37.236518+0000","last_peered":"2024-02-26T10:38:37.236518+0000","last_clean":"2024-02-26T10:38:37.236518+0000","last_became_active":"2024-02-26T10:38:35.201498+0000","last_became_peered":"2024-02-26T10:38:35.201498+0000","last_unstale":"2024-02-26T10:38:37.236518+0000","last_undegraded":"2024-02-26T10:38:37.236518+0000","last_fullsized":"2024-02-26T10:38:37.236518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:00:58.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235996+0000","last_change":"2024-02-26T10:38:34.186745+0000","last_active":"2024-02-26T10:38:37.235996+0000","last_peered":"2024-02-26T10:38:37.235996+0000","last_clean":"2024-02-26T10:38:37.235996+0000","last_became_active":"2024-02-26T10:38:34.186470+0000","last_became_peered":"2024-02-26T10:38:34.186470+0000","last_unstale":"2024-02-26T10:38:37.235996+0000","last_undegraded":"2024-02-26T10:38:37.235996+0000","last_fullsized":"2024-02-26T10:38:37.235996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:47:43.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.178984+0000","last_change":"2024-02-26T10:38:35.206276+0000","last_active":"2024-02-26T10:38:36.178984+0000","last_peered":"2024-02-26T10:38:36.178984+0000","last_clean":"2024-02-26T10:38:36.178984+0000","last_became_active":"2024-02-26T10:38:35.205932+0000","last_became_peered":"2024-02-26T10:38:35.205932+0000","last_unstale":"2024-02-26T10:38:36.178984+0000","last_undegraded":"2024-02-26T10:38:36.178984+0000","last_fullsized":"2024-02-26T10:38:36.178984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:46.169344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235907+0000","last_change":"2024-02-26T10:38:34.165440+0000","last_active":"2024-02-26T10:38:37.235907+0000","last_peered":"2024-02-26T10:38:37.235907+0000","last_clean":"2024-02-26T10:38:37.235907+0000","last_became_active":"2024-02-26T10:38:34.165164+0000","last_became_peered":"2024-02-26T10:38:34.165164+0000","last_unstale":"2024-02-26T10:38:37.235907+0000","last_undegraded":"2024-02-26T10:38:37.235907+0000","last_fullsized":"2024-02-26T10:38:37.235907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:00.856675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:37.235740+0000","last_change":"2024-02-26T10:38:35.199753+0000","last_active":"2024-02-26T10:38:37.235740+0000","last_peered":"2024-02-26T10:38:37.235740+0000","last_clean":"2024-02-26T10:38:37.235740+0000","last_became_active":"2024-02-26T10:38:35.199470+0000","last_became_peered":"2024-02-26T10:38:35.199470+0000","last_unstale":"2024-02-26T10:38:37.235740+0000","last_undegraded":"2024-02-26T10:38:37.235740+0000","last_fullsized":"2024-02-26T10:38:37.235740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:34.135370+0000","last_clean_scrub_stamp":"2024-02-26T10:38:34.135370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:50:18.606268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-02-26T10:38:36.179016+0000","last_change":"2024-02-26T10:38:34.184020+0000","last_active":"2024-02-26T10:38:36.179016+0000","last_peered":"2024-02-26T10:38:36.179016+0000","last_clean":"2024-02-26T10:38:36.179016+0000","last_became_active":"2024-02-26T10:38:34.183497+0000","last_became_peered":"2024-02-26T10:38:34.183497+0000","last_unstale":"2024-02-26T10:38:36.179016+0000","last_undegraded":"2024-02-26T10:38:36.179016+0000","last_fullsized":"2024-02-26T10:38:36.179016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:32.943866+0000","last_clean_scrub_stamp":"2024-02-26T10:38:32.943866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:25:08.410476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":74327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":74237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":74397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":819200,"data_stored":665243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:38:50.872 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-26T10:38:50.872 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-26T10:38:50.872 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-02-26T10:38:50.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-02-26T10:38:51.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.124+0000 7f8b2d800640 1 Processor -- start 2024-02-26T10:38:51.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.124+0000 7f8b2d800640 1 -- start start 2024-02-26T10:38:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.125+0000 7f8b2d800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b281568b0 0x7f8b28072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.125+0000 7f8b2d800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2806eac0 con 0x7f8b2806e580 2024-02-26T10:38:51.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.125+0000 7f8b2d800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b28152f00 con 0x7f8b281500d0 2024-02-26T10:38:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.125+0000 7f8b2d800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b281531e0 con 0x7f8b281568b0 2024-02-26T10:38:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.125+0000 7f8b27400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8b2806e580 0x7f8b281561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56530/0 (socket says 172.21.15.67:56530) 2024-02-26T10:38:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.125+0000 7f8b27400640 1 -- 172.21.15.67:0/2666617186 learned_addr learned my addr 172.21.15.67:0/2666617186 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:51.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b2ce00640 1 --2- 172.21.15.67:0/2666617186 >> v2:172.21.15.82:3300/0 conn(0x7f8b281568b0 0x7f8b28072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005116211 0 0) 0x7f8b2806eac0 con 0x7f8b2806e580 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0c003520 con 0x7f8b2806e580 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722857786 0 0) 0x7f8b0c003520 con 0x7f8b2806e580 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 >> v2:172.21.15.82:3300/0 conn(0x7f8b281568b0 msgr2=0x7f8b28072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 --2- 172.21.15.67:0/2666617186 >> v2:172.21.15.82:3300/0 conn(0x7f8b281568b0 0x7f8b28072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 >> v1:172.21.15.82:6789/0 conn(0x7f8b281500d0 legacy=0x7f8b280c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b281534c0 con 0x7f8b2806e580 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8b14001670 con 0x7f8b2806e580 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 >> v1:172.21.15.67:6789/0 conn(0x7f8b2806e580 legacy=0x7f8b281561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c0051c0 0x7f8b0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281531e0 con 0x7f8b0c004800 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c00c160 con 0x7f8b0c0039b0 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.126+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c00c360 con 0x7f8b0c0051c0 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27400640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b2ce00640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c0051c0 0x7f8b0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27e00640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27400640 1 -- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c0051c0 msgr2=0x7f8b0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27400640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c0051c0 0x7f8b0c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27400640 1 -- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c0039b0 msgr2=0x7f8b0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27400640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27400640 1 -- 172.21.15.67:0/2666617186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c003520 con 0x7f8b0c004800 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b2ce00640 1 -- 172.21.15.67:0/2666617186 reap_dead start 2024-02-26T10:38:51.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.127+0000 7f8b27e00640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c0039b0 0x7f8b0c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.128+0000 7f8b27400640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c210 tx=0x7f8b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18cc767fcbf3316 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.128+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14007cf0 con 0x7f8b0c004800 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.128+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14030070 con 0x7f8b0c004800 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.128+0000 7f8b26a00640 1 -- 172.21.15.67:0/2666617186 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14005d70 con 0x7f8b0c004800 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 -- 172.21.15.67:0/2666617186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b0c004800 msgr2=0x7f8b0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c210 tx=0x7f8b0c005830 comp rx=0 tx=0).stop 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 -- 172.21.15.67:0/2666617186 shutdown_connections 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 --2- 172.21.15.67:0/2666617186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b0c004800 0x7f8b0c004c80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 -- 172.21.15.67:0/2666617186 >> 172.21.15.67:0/2666617186 conn(0x7f8b280a84a0 msgr2=0x7f8b280a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 -- 172.21.15.67:0/2666617186 shutdown_connections 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.129+0000 7f8b2d800640 1 -- 172.21.15.67:0/2666617186 wait complete. 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.131+0000 7f8b2d800640 1 Processor -- start 2024-02-26T10:38:51.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.131+0000 7f8b2d800640 1 -- start start 2024-02-26T10:38:51.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b2d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c004800 0x7f8b28144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b2d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 0x7f8b281eb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b2d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b281eebd0 0x7f8b281419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b2d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281539e0 con 0x7f8b28149e90 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b2d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2806e1f0 con 0x7f8b281eebd0 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b2d800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280aab10 con 0x7f8b0c004800 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 0x7f8b281eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 0x7f8b281eb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44406/0 (socket says 172.21.15.67:44406) 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b27400640 1 -- 172.21.15.67:0/949809752 learned_addr learned my addr 172.21.15.67:0/949809752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.132+0000 7f8b27e00640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c004800 0x7f8b28144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b2ce00640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b281eebd0 0x7f8b281419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27400640 1 -- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c004800 msgr2=0x7f8b28144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27400640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c004800 0x7f8b28144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27400640 1 -- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b281eebd0 msgr2=0x7f8b281419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27400640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b281eebd0 0x7f8b281419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27400640 1 -- 172.21.15.67:0/949809752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280aadf0 con 0x7f8b28149e90 2024-02-26T10:38:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27e00640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c004800 0x7f8b28144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b27400640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 0x7f8b281eb360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001970 tx=0x7f8b281530d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14007cf0 con 0x7f8b28149e90 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1402cba0 con 0x7f8b28149e90 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14001c30 con 0x7f8b28149e90 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b281eae40 con 0x7f8b28149e90 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.133+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b280c3f70 con 0x7f8b28149e90 2024-02-26T10:38:51.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.134+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b280715e0 con 0x7f8b28149e90 2024-02-26T10:38:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.135+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b14007070 con 0x7f8b28149e90 2024-02-26T10:38:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.136+0000 7f8b1fe00640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b0806b500 0x7f8b0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.136+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b140ab700 con 0x7f8b28149e90 2024-02-26T10:38:51.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.137+0000 7f8b27e00640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b0806b500 0x7f8b0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.138+0000 7f8b27e00640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b0806b500 0x7f8b0806d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8b280bf7e0 tx=0x7f8b18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:51.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.141+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b280715e0 con 0x7f8b28149e90 2024-02-26T10:38:51.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.375+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8b28153b60 con 0x7f8b28149e90 2024-02-26T10:38:51.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.376+0000 7f8b1fe00640 1 -- 172.21.15.67:0/949809752 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8b28153b60 con 0x7f8b28149e90 2024-02-26T10:38:51.378 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:51.378 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b0806b500 msgr2=0x7f8b0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b0806b500 0x7f8b0806d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8b280bf7e0 tx=0x7f8b18009040 comp rx=0 tx=0).stop 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 msgr2=0x7f8b281eb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 0x7f8b281eb360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001970 tx=0x7f8b281530d0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 shutdown_connections 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b0806b500 0x7f8b0806d9c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b281eebd0 0x7f8b281419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b28149e90 0x7f8b281eb360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 --2- 172.21.15.67:0/949809752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b0c004800 0x7f8b28144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.378+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 >> 172.21.15.67:0/949809752 conn(0x7f8b280a84a0 msgr2=0x7f8b2806b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.379+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 shutdown_connections 2024-02-26T10:38:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.379+0000 7f8b2d800640 1 -- 172.21.15.67:0/949809752 wait complete. 2024-02-26T10:38:51.400 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-02-26T10:38:51.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:51.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.612+0000 7f2c86600640 1 Processor -- start 2024-02-26T10:38:51.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.613+0000 7f2c86600640 1 -- start start 2024-02-26T10:38:51.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.614+0000 7f2c86600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c800c8510 0x7f2c800c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.614+0000 7f2c86600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c80152760 con 0x7f2c80152380 2024-02-26T10:38:51.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.614+0000 7f2c86600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8006d9a0 con 0x7f2c800c7020 2024-02-26T10:38:51.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.614+0000 7f2c86600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800c7550 con 0x7f2c800c8510 2024-02-26T10:38:51.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.614+0000 7f2c85200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2c80152380 0x7f2c80155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56534/0 (socket says 172.21.15.67:56534) 2024-02-26T10:38:51.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.614+0000 7f2c85200640 1 -- 172.21.15.67:0/2998152378 learned_addr learned my addr 172.21.15.67:0/2998152378 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:51.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c85c00640 1 --2- 172.21.15.67:0/2998152378 >> v2:172.21.15.67:3300/0 conn(0x7f2c800c8510 0x7f2c800c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c7f400640 1 -- 172.21.15.67:0/2998152378 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151600409 0 0) 0x7f2c80152760 con 0x7f2c80152380 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c7f400640 1 -- 172.21.15.67:0/2998152378 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c64003520 con 0x7f2c80152380 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c7f400640 1 -- 172.21.15.67:0/2998152378 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317023559 0 0) 0x7f2c64003520 con 0x7f2c80152380 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c85c00640 1 -- 172.21.15.67:0/2998152378 >> v1:172.21.15.82:6789/0 conn(0x7f2c800c7020 legacy=0x7f2c8006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c85c00640 1 -- 172.21.15.67:0/2998152378 >> v1:172.21.15.67:6789/0 conn(0x7f2c80152380 legacy=0x7f2c80155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.615+0000 7f2c85c00640 1 -- 172.21.15.67:0/2998152378 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800c7830 con 0x7f2c800c8510 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.616+0000 7f2c85c00640 1 --2- 172.21.15.67:0/2998152378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c800c4f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c780097a0 tx=0x7f2c7802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3180a03121532c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:51.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.616+0000 7f2c7f400640 1 -- 172.21.15.67:0/2998152378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78037070 con 0x7f2c800c8510 2024-02-26T10:38:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c7f400640 1 -- 172.21.15.67:0/2998152378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78030510 con 0x7f2c800c8510 2024-02-26T10:38:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c7f400640 1 -- 172.21.15.67:0/2998152378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c700019f0 con 0x7f2c800c8510 2024-02-26T10:38:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 -- 172.21.15.67:0/2998152378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 msgr2=0x7f2c800c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 --2- 172.21.15.67:0/2998152378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c800c4f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c780097a0 tx=0x7f2c7802b710 comp rx=0 tx=0).stop 2024-02-26T10:38:51.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 -- 172.21.15.67:0/2998152378 shutdown_connections 2024-02-26T10:38:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 --2- 172.21.15.67:0/2998152378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c800c4f90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 -- 172.21.15.67:0/2998152378 >> 172.21.15.67:0/2998152378 conn(0x7f2c800ac1a0 msgr2=0x7f2c800ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 -- 172.21.15.67:0/2998152378 shutdown_connections 2024-02-26T10:38:51.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.617+0000 7f2c86600640 1 -- 172.21.15.67:0/2998152378 wait complete. 2024-02-26T10:38:51.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.619+0000 7f2c86600640 1 Processor -- start 2024-02-26T10:38:51.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.619+0000 7f2c86600640 1 -- start start 2024-02-26T10:38:51.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c86600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c800c7020 0x7f2c80158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c86600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c801533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c86600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c80152380 0x7f2c80153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c86600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800c54d0 con 0x7f2c800c8510 2024-02-26T10:38:51.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c86600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8006cd70 con 0x7f2c80152380 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c86600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800c5650 con 0x7f2c800c7020 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c801533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c801533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44416/0 (socket says 172.21.15.67:44416) 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c7fe00640 1 -- 172.21.15.67:0/647850912 learned_addr learned my addr 172.21.15.67:0/647850912 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c85200640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c800c7020 0x7f2c80158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c7fe00640 1 -- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c800c7020 msgr2=0x7f2c80158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c7fe00640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c800c7020 0x7f2c80158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c7fe00640 1 -- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c80152380 msgr2=0x7f2c80153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.620+0000 7f2c85c00640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c80152380 0x7f2c80153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c7fe00640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c80152380 0x7f2c80153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c7fe00640 1 -- 172.21.15.67:0/647850912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800c0a90 con 0x7f2c800c8510 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c85c00640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c80152380 0x7f2c80153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:51.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c7fe00640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c801533e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c460 tx=0x7f2c6800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68016020 con 0x7f2c800c8510 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68012870 con 0x7f2c800c8510 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68012c60 con 0x7f2c800c8510 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c80156d40 con 0x7f2c800c8510 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.621+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c80158eb0 con 0x7f2c800c8510 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.622+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8006ac20 con 0x7f2c800c8510 2024-02-26T10:38:51.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.623+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c68019020 con 0x7f2c800c8510 2024-02-26T10:38:51.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.624+0000 7f2c7cc00640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:51.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.624+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c68089f30 con 0x7f2c800c8510 2024-02-26T10:38:51.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.625+0000 7f2c85200640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:51.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.626+0000 7f2c85200640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2c80152d50 tx=0x7f2c70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:51.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.628+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c8006ac20 con 0x7f2c800c8510 2024-02-26T10:38:51.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.821+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c8006bf80 con 0x7f2c800c8510 2024-02-26T10:38:51.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.823+0000 7f2c7cc00640 1 -- 172.21.15.67:0/647850912 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f2c8006bf80 con 0x7f2c800c8510 2024-02-26T10:38:51.825 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:51.825 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:51.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2c5c06b4d0 msgr2=0x7f2c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2c80152d50 tx=0x7f2c70009040 comp rx=0 tx=0).stop 2024-02-26T10:38:51.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 msgr2=0x7f2c801533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c801533e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c460 tx=0x7f2c6800c920 comp rx=0 tx=0).stop 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 shutdown_connections 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c80152380 0x7f2c80153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c800c8510 0x7f2c801533e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 --2- 172.21.15.67:0/647850912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c800c7020 0x7f2c80158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 >> 172.21.15.67:0/647850912 conn(0x7f2c800ac1a0 msgr2=0x7f2c801506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 shutdown_connections 2024-02-26T10:38:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:51.826+0000 7f2c86600640 1 -- 172.21.15.67:0/647850912 wait complete. 2024-02-26T10:38:51.845 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:38:51.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:38:52.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.050+0000 7fbc40a00640 1 Processor -- start 2024-02-26T10:38:52.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.051+0000 7fbc40a00640 1 -- start start 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.052+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c0a9430 0x7fbc3c0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.052+0000 7fbc40a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc3c06e2d0 0x7fbc3c0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.052+0000 7fbc40a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3c150780 con 0x7fbc3c1503a0 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.052+0000 7fbc40a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3c0a98c0 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.052+0000 7fbc40a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0a9ba0 con 0x7fbc3c0a9430 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc3c06e2d0 0x7fbc3c0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc3c06e2d0 0x7fbc3c0c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44424/0 (socket says 172.21.15.67:44424) 2024-02-26T10:38:52.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 -- 172.21.15.67:0/3168106042 learned_addr learned my addr 172.21.15.67:0/3168106042 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:52.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc39e00640 1 --2- 172.21.15.67:0/3168106042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c0a9430 0x7fbc3c0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 -- 172.21.15.67:0/3168106042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c0a9430 msgr2=0x7fbc3c0ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:52.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 --2- 172.21.15.67:0/3168106042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c0a9430 0x7fbc3c0ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 -- 172.21.15.67:0/3168106042 >> v1:172.21.15.82:6789/0 conn(0x7fbc3c1503a0 legacy=0x7fbc3c1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:52.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.053+0000 7fbc3b200640 1 -- 172.21.15.67:0/3168106042 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c0a9e80 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.054+0000 7fbc3b200640 1 --2- 172.21.15.67:0/3168106042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c0c2e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009e50 tx=0x7fbc3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8badca0c4ea2e649 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc39400640 1 -- 172.21.15.67:0/3168106042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3402ca70 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc39400640 1 -- 172.21.15.67:0/3168106042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3402cc10 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc39400640 1 -- 172.21.15.67:0/3168106042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc340343f0 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc40a00640 1 -- 172.21.15.67:0/3168106042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 msgr2=0x7fbc3c0c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc40a00640 1 --2- 172.21.15.67:0/3168106042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c0c2e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009e50 tx=0x7fbc3402bed0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc40a00640 1 -- 172.21.15.67:0/3168106042 shutdown_connections 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc40a00640 1 --2- 172.21.15.67:0/3168106042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c0c2e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc40a00640 1 --2- 172.21.15.67:0/3168106042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c0a9430 0x7fbc3c0ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.055+0000 7fbc40a00640 1 -- 172.21.15.67:0/3168106042 >> 172.21.15.67:0/3168106042 conn(0x7fbc3c0c6380 msgr2=0x7fbc3c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.056+0000 7fbc40a00640 1 -- 172.21.15.67:0/3168106042 shutdown_connections 2024-02-26T10:38:52.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.056+0000 7fbc40a00640 1 -- 172.21.15.67:0/3168106042 wait complete. 2024-02-26T10:38:52.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.058+0000 7fbc40a00640 1 Processor -- start 2024-02-26T10:38:52.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.058+0000 7fbc40a00640 1 -- start start 2024-02-26T10:38:52.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1503a0 0x7fbc3c1df930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c1ec4d0 0x7fbc3c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc40a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0c0f80 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc40a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c06ddb0 con 0x7fbc3c1503a0 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc40a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c06e740 con 0x7fbc3c1ec4d0 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44440/0 (socket says 172.21.15.67:44440) 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.059+0000 7fbc3a800640 1 -- 172.21.15.67:0/1124758134 learned_addr learned my addr 172.21.15.67:0/1124758134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc39e00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1503a0 0x7fbc3c1df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3b200640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c1ec4d0 0x7fbc3c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3a800640 1 -- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c1ec4d0 msgr2=0x7fbc3c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3a800640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c1ec4d0 0x7fbc3c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3a800640 1 -- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1503a0 msgr2=0x7fbc3c1df930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3a800640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1503a0 0x7fbc3c1df930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3a800640 1 -- 172.21.15.67:0/1124758134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3c148540 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3b200640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c1ec4d0 0x7fbc3c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc3a800640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c147db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00b3f0 tx=0x7fbc2c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c015020 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.060+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c011830 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.061+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c011c20 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.061+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c1ec960 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.061+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c153570 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.062+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3c074a30 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.063+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc2c018020 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.063+0000 7fbc32a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc1806b4d0 0x7fbc1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.064+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc2c089d70 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.064+0000 7fbc39e00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc1806b4d0 0x7fbc1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.065+0000 7fbc39e00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc1806b4d0 0x7fbc1806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c148460 tx=0x7fbc24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.068+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc2c01f030 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.259+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc3c0c3ad0 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.260+0000 7fbc32a00640 1 -- 172.21.15.67:0/1124758134 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc2c0172c0 con 0x7fbc3c06e2d0 2024-02-26T10:38:52.262 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:38:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc1806b4d0 msgr2=0x7fbc1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc1806b4d0 0x7fbc1806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c148460 tx=0x7fbc24008040 comp rx=0 tx=0).stop 2024-02-26T10:38:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 msgr2=0x7fbc3c147db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c147db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00b3f0 tx=0x7fbc2c00b8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 shutdown_connections 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc3c1ec4d0 0x7fbc3c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc1806b4d0 0x7fbc1806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1503a0 0x7fbc3c1df930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 --2- 172.21.15.67:0/1124758134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc3c06e2d0 0x7fbc3c147db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 >> 172.21.15.67:0/1124758134 conn(0x7fbc3c0c6380 msgr2=0x7fbc3c0c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 shutdown_connections 2024-02-26T10:38:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.263+0000 7fbc40a00640 1 -- 172.21.15.67:0/1124758134 wait complete. 2024-02-26T10:38:52.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:38:52.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.492+0000 7f8a2ea00640 1 Processor -- start 2024-02-26T10:38:52.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.492+0000 7f8a2ea00640 1 -- start start 2024-02-26T10:38:52.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.494+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 0x7f8a280afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.494+0000 7f8a2ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2806c110 con 0x7f8a280c5290 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.494+0000 7f8a2ea00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2806d000 con 0x7f8a281562f0 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.494+0000 7f8a2ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280c72c0 con 0x7f8a280af8a0 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.494+0000 7f8a2d600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8a280c5290 0x7f8a28155880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56542/0 (socket says 172.21.15.67:56542) 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.494+0000 7f8a2d600640 1 -- 172.21.15.67:0/4092974174 learned_addr learned my addr 172.21.15.67:0/4092974174 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a2e000640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 0x7f8a280afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614899076 0 0) 0x7f8a2806c110 con 0x7f8a280c5290 2024-02-26T10:38:52.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a10003520 con 0x7f8a280c5290 2024-02-26T10:38:52.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1702853766 0 0) 0x7f8a10003520 con 0x7f8a280c5290 2024-02-26T10:38:52.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 msgr2=0x7f8a280afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 0x7f8a280afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 >> v1:172.21.15.82:6789/0 conn(0x7f8a281562f0 legacy=0x7f8a2806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280c75a0 con 0x7f8a280c5290 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8a1c001670 con 0x7f8a280c5290 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 >> v1:172.21.15.67:6789/0 conn(0x7f8a280c5290 legacy=0x7f8a28155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 0x7f8a100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a10003fb0 0x7f8a10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10004d40 0x7f8a100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280c72c0 con 0x7f8a10003fb0 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1000bbb0 con 0x7f8a10004d40 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1000bdb0 con 0x7f8a280af8a0 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.495+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a10003fb0 0x7f8a10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2d600640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 0x7f8a100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2e000640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10004d40 0x7f8a100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2cc00640 1 -- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 msgr2=0x7f8a100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:52.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280af8a0 0x7f8a100039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2cc00640 1 -- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10004d40 msgr2=0x7f8a100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:52.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10004d40 0x7f8a100051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.496+0000 7f8a2cc00640 1 -- 172.21.15.67:0/4092974174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10003520 con 0x7f8a10003fb0 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.497+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a10003fb0 0x7f8a10004800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a14001750 tx=0x7f8a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835f4f063eff3c08 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.497+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14001370 con 0x7f8a10003fb0 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.497+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a140052d0 con 0x7f8a10003fb0 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.497+0000 7f8a27e00640 1 -- 172.21.15.67:0/4092974174 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003830 con 0x7f8a10003fb0 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2ea00640 1 -- 172.21.15.67:0/4092974174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a10003fb0 msgr2=0x7f8a10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/4092974174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a10003fb0 0x7f8a10004800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a14001750 tx=0x7f8a10005830 comp rx=0 tx=0).stop 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2e000640 1 -- 172.21.15.67:0/4092974174 reap_dead start 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2ea00640 1 -- 172.21.15.67:0/4092974174 shutdown_connections 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2ea00640 1 -- 172.21.15.67:0/4092974174 >> 172.21.15.67:0/4092974174 conn(0x7f8a280ac230 msgr2=0x7f8a280ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:52.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2ea00640 1 -- 172.21.15.67:0/4092974174 shutdown_connections 2024-02-26T10:38:52.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.498+0000 7f8a2ea00640 1 -- 172.21.15.67:0/4092974174 wait complete. 2024-02-26T10:38:52.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.500+0000 7f8a2ea00640 1 Processor -- start 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.500+0000 7f8a2ea00640 1 -- start start 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.500+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10003fb0 0x7f8a2814ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.500+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 0x7f8a280adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280ae480 0x7f8a280c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280c5670 con 0x7f8a280c9f20 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2806c3d0 con 0x7f8a10003fb0 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280c57f0 con 0x7f8a280ae480 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 0x7f8a280adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 0x7f8a280adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44450/0 (socket says 172.21.15.67:44450) 2024-02-26T10:38:52.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 -- 172.21.15.67:0/1333363164 learned_addr learned my addr 172.21.15.67:0/1333363164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2d600640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10003fb0 0x7f8a2814ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2e000640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280ae480 0x7f8a280c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 -- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280ae480 msgr2=0x7f8a280c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280ae480 0x7f8a280c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 -- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10003fb0 msgr2=0x7f8a2814ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10003fb0 0x7f8a2814ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2cc00640 1 -- 172.21.15.67:0/1333363164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280abfc0 con 0x7f8a280c9f20 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.501+0000 7f8a2d600640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10003fb0 0x7f8a2814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.502+0000 7f8a2cc00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 0x7f8a280adf40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005040 tx=0x7f8a14001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.502+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14041070 con 0x7f8a280c9f20 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.502+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14000ed0 con 0x7f8a280c9f20 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.502+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a280c91b0 con 0x7f8a280c9f20 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.502+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14007a30 con 0x7f8a280c9f20 2024-02-26T10:38:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.502+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a280c2100 con 0x7f8a280c9f20 2024-02-26T10:38:52.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.503+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a28074a30 con 0x7f8a280c9f20 2024-02-26T10:38:52.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.504+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a14044070 con 0x7f8a280c9f20 2024-02-26T10:38:52.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.505+0000 7f8a25600640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a0806b500 0x7f8a0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.505+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a140abc60 con 0x7f8a280c9f20 2024-02-26T10:38:52.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.506+0000 7f8a2d600640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a0806b500 0x7f8a0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.507+0000 7f8a2d600640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a0806b500 0x7f8a0806d9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0078f0 tx=0x7f8a1c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.509+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a1407aa40 con 0x7f8a280c9f20 2024-02-26T10:38:52.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.705+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a2814f9a0 con 0x7f8a280c9f20 2024-02-26T10:38:52.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.706+0000 7f8a25600640 1 -- 172.21.15.67:0/1333363164 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a1407a340 con 0x7f8a280c9f20 2024-02-26T10:38:52.708 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:38:52.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a0806b500 msgr2=0x7f8a0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a0806b500 0x7f8a0806d9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0078f0 tx=0x7f8a1c009040 comp rx=0 tx=0).stop 2024-02-26T10:38:52.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 msgr2=0x7f8a280adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 0x7f8a280adf40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a14005040 tx=0x7f8a14001120 comp rx=0 tx=0).stop 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 shutdown_connections 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a280ae480 0x7f8a280c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a280c9f20 0x7f8a280adf40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a0806b500 0x7f8a0806d9c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 --2- 172.21.15.67:0/1333363164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a10003fb0 0x7f8a2814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 >> 172.21.15.67:0/1333363164 conn(0x7f8a280ac230 msgr2=0x7f8a280ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 shutdown_connections 2024-02-26T10:38:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.709+0000 7f8a2ea00640 1 -- 172.21.15.67:0/1333363164 wait complete. 2024-02-26T10:38:52.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:38:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.945+0000 7f37c9200640 1 Processor -- start 2024-02-26T10:38:52.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.945+0000 7f37c9200640 1 -- start start 2024-02-26T10:38:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c40ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c9200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 0x7f37c41535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c9200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c4153ae0 con 0x7f37c406e2d0 2024-02-26T10:38:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c9200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c4150780 con 0x7f37c40a9430 2024-02-26T10:38:52.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40a98c0 con 0x7f37c41503a0 2024-02-26T10:38:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 0x7f37c41535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c3400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c40ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 0x7f37c41535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47706/0 (socket says 172.21.15.67:47706) 2024-02-26T10:38:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.947+0000 7f37c2a00640 1 -- 172.21.15.67:0/4131156728 learned_addr learned my addr 172.21.15.67:0/4131156728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c2000640 1 -- 172.21.15.67:0/4131156728 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192318483 0 0) 0x7f37c4153ae0 con 0x7f37c406e2d0 2024-02-26T10:38:52.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c2000640 1 -- 172.21.15.67:0/4131156728 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a8003520 con 0x7f37c406e2d0 2024-02-26T10:38:52.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c3400640 1 -- 172.21.15.67:0/4131156728 >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 msgr2=0x7f37c41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:52.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c3400640 1 --2- 172.21.15.67:0/4131156728 >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 0x7f37c41535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c3400640 1 -- 172.21.15.67:0/4131156728 >> v1:172.21.15.82:6789/0 conn(0x7f37c406e2d0 legacy=0x7f37c40c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c3400640 1 -- 172.21.15.67:0/4131156728 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40a9ba0 con 0x7f37c41503a0 2024-02-26T10:38:52.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.948+0000 7f37c2a00640 1 --2- 172.21.15.67:0/4131156728 >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 0x7f37c41535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:38:52.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.950+0000 7f37c3400640 1 --2- 172.21.15.67:0/4131156728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c40ab480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37b40097a0 tx=0x7f37b402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea5713dc4c846ab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.950+0000 7f37c2000640 1 -- 172.21.15.67:0/4131156728 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4035020 con 0x7f37c41503a0 2024-02-26T10:38:52.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.950+0000 7f37c2000640 1 -- 172.21.15.67:0/4131156728 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b4031810 con 0x7f37c41503a0 2024-02-26T10:38:52.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.950+0000 7f37c2000640 1 -- 172.21.15.67:0/4131156728 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4031bc0 con 0x7f37c41503a0 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 -- 172.21.15.67:0/4131156728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 msgr2=0x7f37c40ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 --2- 172.21.15.67:0/4131156728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c40ab480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f37b40097a0 tx=0x7f37b402b8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 -- 172.21.15.67:0/4131156728 shutdown_connections 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 --2- 172.21.15.67:0/4131156728 >> v2:172.21.15.82:3300/0 conn(0x7f37c40a9430 0x7f37c41535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 --2- 172.21.15.67:0/4131156728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c40ab480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 -- 172.21.15.67:0/4131156728 >> 172.21.15.67:0/4131156728 conn(0x7f37c40c6380 msgr2=0x7f37c40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 -- 172.21.15.67:0/4131156728 shutdown_connections 2024-02-26T10:38:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.951+0000 7f37c9200640 1 -- 172.21.15.67:0/4131156728 wait complete. 2024-02-26T10:38:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.952+0000 7f37c9200640 1 Processor -- start 2024-02-26T10:38:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.952+0000 7f37c9200640 1 -- start start 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.953+0000 7f37c9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 0x7f37c4151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.953+0000 7f37c9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c4071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.953+0000 7f37c9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c4140390 0x7f37c4072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.953+0000 7f37c9200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c406ddb0 con 0x7f37c406e2d0 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.953+0000 7f37c9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c406dad0 con 0x7f37c4140390 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.953+0000 7f37c9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40c0f80 con 0x7f37c41503a0 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 0x7f37c4151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 0x7f37c4151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44452/0 (socket says 172.21.15.67:44452) 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 -- 172.21.15.67:0/353345706 learned_addr learned my addr 172.21.15.67:0/353345706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c2a00640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c4071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3e00640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c4140390 0x7f37c4072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 -- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 msgr2=0x7f37c4071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c4071ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 -- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c4140390 msgr2=0x7f37c4072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c4140390 0x7f37c4072530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 -- 172.21.15.67:0/353345706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c41ee4e0 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.954+0000 7f37c3400640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 0x7f37c4151d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f37b402b8f0 tx=0x7f37b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.955+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4035040 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.955+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b4039cf0 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.955+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4039ea0 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.955+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c41ee7c0 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.955+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41eed80 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.956+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c40678e0 con 0x7f37c406e2d0 2024-02-26T10:38:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.957+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37b4038020 con 0x7f37c406e2d0 2024-02-26T10:38:52.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.958+0000 7f37bb400640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37a406b4d0 0x7f37a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:52.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.958+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37b40ac310 con 0x7f37c406e2d0 2024-02-26T10:38:52.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.958+0000 7f37c2a00640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37a406b4d0 0x7f37a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.959+0000 7f37c2a00640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37a406b4d0 0x7f37a406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f37c4148350 tx=0x7f37b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:52.962+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37b407b190 con 0x7f37c406e2d0 2024-02-26T10:38:53.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.160+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37c4074a30 con 0x7f37c406e2d0 2024-02-26T10:38:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.161+0000 7f37bb400640 1 -- 172.21.15.67:0/353345706 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37b407aa90 con 0x7f37c406e2d0 2024-02-26T10:38:53.163 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:38:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37a406b4d0 msgr2=0x7f37a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37a406b4d0 0x7f37a406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f37c4148350 tx=0x7f37b0008040 comp rx=0 tx=0).stop 2024-02-26T10:38:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 msgr2=0x7f37c4151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 0x7f37c4151d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f37b402b8f0 tx=0x7f37b40046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 shutdown_connections 2024-02-26T10:38:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37a406b4d0 0x7f37a406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c4140390 0x7f37c4072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37c41503a0 0x7f37c4071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 --2- 172.21.15.67:0/353345706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37c406e2d0 0x7f37c4151d80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 >> 172.21.15.67:0/353345706 conn(0x7f37c40c6380 msgr2=0x7f37c414c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 shutdown_connections 2024-02-26T10:38:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.164+0000 7f37c9200640 1 -- 172.21.15.67:0/353345706 wait complete. 2024-02-26T10:38:53.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:38:53.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.409+0000 7fdefb200640 1 Processor -- start 2024-02-26T10:38:53.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.409+0000 7fdefb200640 1 -- start start 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.410+0000 7fdefb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc06ac20 0x7fdefc06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.410+0000 7fdefb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdefc153630 0x7fdefc1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.410+0000 7fdefb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc155f10 con 0x7fdefc0c2310 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.410+0000 7fdefb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc153aa0 con 0x7fdefc153630 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.410+0000 7fdefb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c28d0 con 0x7fdefc06ac20 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdefc153630 0x7fdefc1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdefc153630 0x7fdefc1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44460/0 (socket says 172.21.15.67:44460) 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdef9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdefc0c2310 0x7fdefc152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56554/0 (socket says 172.21.15.67:56554) 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 -- 172.21.15.67:0/1902745071 learned_addr learned my addr 172.21.15.67:0/1902745071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdef9e00640 1 --2- 172.21.15.67:0/1902745071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc06ac20 0x7fdefc06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 -- 172.21.15.67:0/1902745071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc06ac20 msgr2=0x7fdefc06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:53.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 --2- 172.21.15.67:0/1902745071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc06ac20 0x7fdefc06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 -- 172.21.15.67:0/1902745071 >> v1:172.21.15.67:6789/0 conn(0x7fdefc0c2310 legacy=0x7fdefc152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.411+0000 7fdefa800640 1 -- 172.21.15.67:0/1902745071 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc155150 con 0x7fdefc153630 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.412+0000 7fdefa800640 1 --2- 172.21.15.67:0/1902745071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc153630 0x7fdefc1524b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009870 tx=0x7fdef402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4684bc5a4ec3dc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdef8a00640 1 -- 172.21.15.67:0/1902745071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40317c0 con 0x7fdefc153630 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdef8a00640 1 -- 172.21.15.67:0/1902745071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4037070 con 0x7fdefc153630 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdef8a00640 1 -- 172.21.15.67:0/1902745071 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4031b80 con 0x7fdefc153630 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 -- 172.21.15.67:0/1902745071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc153630 msgr2=0x7fdefc1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 --2- 172.21.15.67:0/1902745071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc153630 0x7fdefc1524b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009870 tx=0x7fdef402b860 comp rx=0 tx=0).stop 2024-02-26T10:38:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 -- 172.21.15.67:0/1902745071 shutdown_connections 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 --2- 172.21.15.67:0/1902745071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc153630 0x7fdefc1524b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 --2- 172.21.15.67:0/1902745071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc06ac20 0x7fdefc06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 -- 172.21.15.67:0/1902745071 >> 172.21.15.67:0/1902745071 conn(0x7fdefc0c63f0 msgr2=0x7fdefc0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.413+0000 7fdefb200640 1 -- 172.21.15.67:0/1902745071 shutdown_connections 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.414+0000 7fdefb200640 1 -- 172.21.15.67:0/1902745071 wait complete. 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.415+0000 7fdefb200640 1 Processor -- start 2024-02-26T10:38:53.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.415+0000 7fdefb200640 1 -- start start 2024-02-26T10:38:53.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdefb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 0x7fdefc150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdefb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdefc0c2310 0x7fdefc144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdefb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc153630 0x7fdefc1db120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdefb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1559f0 con 0x7fdefc06ac20 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdefb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc155710 con 0x7fdefc0c2310 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdefb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc148850 con 0x7fdefc153630 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.416+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 0x7fdefc150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 0x7fdefc150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44466/0 (socket says 172.21.15.67:44466) 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 -- 172.21.15.67:0/2445402174 learned_addr learned my addr 172.21.15.67:0/2445402174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdefa800640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc153630 0x7fdefc1db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9400640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdefc0c2310 0x7fdefc144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 -- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc153630 msgr2=0x7fdefc1db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc153630 0x7fdefc1db120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 -- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdefc0c2310 msgr2=0x7fdefc144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdefc0c2310 0x7fdefc144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 -- 172.21.15.67:0/2445402174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc141ae0 con 0x7fdefc06ac20 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdefa800640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc153630 0x7fdefc1db120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.417+0000 7fdef9e00640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 0x7fdefc150150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007e70 tx=0x7fdeec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.418+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec014620 con 0x7fdefc06ac20 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.418+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec019070 con 0x7fdefc06ac20 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.418+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec014a00 con 0x7fdefc06ac20 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.418+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc14f570 con 0x7fdefc06ac20 2024-02-26T10:38:53.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.418+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1ec540 con 0x7fdefc06ac20 2024-02-26T10:38:53.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.419+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc074a30 con 0x7fdefc06ac20 2024-02-26T10:38:53.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.420+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdeec014ba0 con 0x7fdefc06ac20 2024-02-26T10:38:53.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.421+0000 7fdef2000640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.421+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdeec08aae0 con 0x7fdefc06ac20 2024-02-26T10:38:53.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.421+0000 7fdef9400640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.422+0000 7fdef9400640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fded806b500 0x7fded806d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdefc142530 tx=0x7fdee4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.425+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdeec0598c0 con 0x7fdefc06ac20 2024-02-26T10:38:53.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.662+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdefc0c20f0 con 0x7fdefc06ac20 2024-02-26T10:38:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.663+0000 7fdef2000640 1 -- 172.21.15.67:0/2445402174 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fdeec0591c0 con 0x7fdefc06ac20 2024-02-26T10:38:53.665 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:53.665 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:38:53.665 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:38:53.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fded806b500 msgr2=0x7fded806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fded806b500 0x7fded806d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdefc142530 tx=0x7fdee4009040 comp rx=0 tx=0).stop 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 msgr2=0x7fdefc150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 0x7fdefc150150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdeec007e70 tx=0x7fdeec00e4b0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 shutdown_connections 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdefc153630 0x7fdefc1db120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdefc0c2310 0x7fdefc144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 --2- 172.21.15.67:0/2445402174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc06ac20 0x7fdefc150150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 >> 172.21.15.67:0/2445402174 conn(0x7fdefc0c63f0 msgr2=0x7fdefc152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 shutdown_connections 2024-02-26T10:38:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.666+0000 7fdefb200640 1 -- 172.21.15.67:0/2445402174 wait complete. 2024-02-26T10:38:53.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.910+0000 7ff2a3000640 1 Processor -- start 2024-02-26T10:38:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.910+0000 7ff2a3000640 1 -- start start 2024-02-26T10:38:53.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.911+0000 7ff2a3000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff29c06ac20 0x7ff29c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.911+0000 7ff2a3000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c06b460 con 0x7ff29c06b000 2024-02-26T10:38:53.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.911+0000 7ff2a3000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c06df90 con 0x7ff29c06db00 2024-02-26T10:38:53.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.911+0000 7ff2a3000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0a9a80 con 0x7ff29c06ac20 2024-02-26T10:38:53.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.912+0000 7ff2a1200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff29c06b000 0x7ff29c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56558/0 (socket says 172.21.15.67:56558) 2024-02-26T10:38:53.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.912+0000 7ff2a1200640 1 -- 172.21.15.67:0/2432434785 learned_addr learned my addr 172.21.15.67:0/2432434785 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.912+0000 7ff2a1c00640 1 --2- 172.21.15.67:0/2432434785 >> v2:172.21.15.82:3300/0 conn(0x7ff29c06ac20 0x7ff29c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.912+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18548838 0 0) 0x7ff29c06b460 con 0x7ff29c06b000 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.912+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff280003520 con 0x7ff29c06b000 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.912+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756327948 0 0) 0x7ff280003520 con 0x7ff29c06b000 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 >> v2:172.21.15.82:3300/0 conn(0x7ff29c06ac20 msgr2=0x7ff29c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 --2- 172.21.15.67:0/2432434785 >> v2:172.21.15.82:3300/0 conn(0x7ff29c06ac20 0x7ff29c0a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 >> v1:172.21.15.82:6789/0 conn(0x7ff29c06db00 legacy=0x7ff29c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0a9d60 con 0x7ff29c06b000 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ff288001670 con 0x7ff29c06b000 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 >> v1:172.21.15.67:6789/0 conn(0x7ff29c06b000 legacy=0x7ff29c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2800039b0 0x7ff280003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff280004800 0x7ff280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0a9a80 con 0x7ff2800051c0 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28000c060 con 0x7ff280004800 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28000c260 con 0x7ff2800039b0 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a2600640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a1200640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff280004800 0x7ff280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a1c00640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2800039b0 0x7ff280003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a2600640 1 -- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2800039b0 msgr2=0x7ff280003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a2600640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2800039b0 0x7ff280003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a2600640 1 -- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff280004800 msgr2=0x7ff280004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a2600640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff280004800 0x7ff280004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.913+0000 7ff2a2600640 1 -- 172.21.15.67:0/2432434785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280003520 con 0x7ff2800051c0 2024-02-26T10:38:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.914+0000 7ff2a2600640 1 -- 172.21.15.67:0/2432434785 reap_dead start 2024-02-26T10:38:53.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff2a2600640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff294001e50 tx=0x7ff280005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d68650a2c0ba04 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294001a70 con 0x7ff2800051c0 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff294003b90 con 0x7ff2800051c0 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff29be00640 1 -- 172.21.15.67:0/2432434785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940050d0 con 0x7ff2800051c0 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff2a3000640 1 -- 172.21.15.67:0/2432434785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 msgr2=0x7ff28000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff294001e50 tx=0x7ff280005830 comp rx=0 tx=0).stop 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff2a3000640 1 -- 172.21.15.67:0/2432434785 shutdown_connections 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2432434785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff28000b9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.915+0000 7ff2a3000640 1 -- 172.21.15.67:0/2432434785 >> 172.21.15.67:0/2432434785 conn(0x7ff29c0c63f0 msgr2=0x7ff29c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.916+0000 7ff2a3000640 1 -- 172.21.15.67:0/2432434785 shutdown_connections 2024-02-26T10:38:53.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.916+0000 7ff2a3000640 1 -- 172.21.15.67:0/2432434785 wait complete. 2024-02-26T10:38:53.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.917+0000 7ff2a3000640 1 Processor -- start 2024-02-26T10:38:53.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.917+0000 7ff2a3000640 1 -- start start 2024-02-26T10:38:53.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.918+0000 7ff2a3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff29c148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.918+0000 7ff2a3000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c070c50 0x7ff29c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.918+0000 7ff2a3000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff29c0c5c70 0x7ff29c1eb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.918+0000 7ff2a3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c150410 con 0x7ff2800051c0 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.918+0000 7ff2a3000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0aba30 con 0x7ff29c070c50 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.918+0000 7ff2a3000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c150590 con 0x7ff29c0c5c70 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff29c148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff29c148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44484/0 (socket says 172.21.15.67:44484) 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 -- 172.21.15.67:0/2419511484 learned_addr learned my addr 172.21.15.67:0/2419511484 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 -- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff29c0c5c70 msgr2=0x7ff29c1eb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff29c0c5c70 0x7ff29c1eb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1200640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c070c50 0x7ff29c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 -- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c070c50 msgr2=0x7ff29c14a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c070c50 0x7ff29c14a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 -- 172.21.15.67:0/2419511484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c072560 con 0x7ff2800051c0 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.919+0000 7ff2a1c00640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff29c148a30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff28c011570 tx=0x7ff28c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:53.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.920+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c015930 con 0x7ff2800051c0 2024-02-26T10:38:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.920+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c015ad0 con 0x7ff2800051c0 2024-02-26T10:38:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.920+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c01d400 con 0x7ff2800051c0 2024-02-26T10:38:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.920+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c1eaac0 con 0x7ff2800051c0 2024-02-26T10:38:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.920+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c072800 con 0x7ff2800051c0 2024-02-26T10:38:53.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.921+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c074a30 con 0x7ff2800051c0 2024-02-26T10:38:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.922+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff28c01d5a0 con 0x7ff2800051c0 2024-02-26T10:38:53.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.923+0000 7ff299600640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff27c06b550 0x7ff27c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:53.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.923+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff28c08a890 con 0x7ff2800051c0 2024-02-26T10:38:53.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.923+0000 7ff2a1200640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff27c06b550 0x7ff27c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:53.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.925+0000 7ff2a1200640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff27c06b550 0x7ff27c06da10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0c6350 tx=0x7ff288001150 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:53.928+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff28c019030 con 0x7ff2800051c0 2024-02-26T10:38:54.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.125+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff29c06b9b0 con 0x7ff2800051c0 2024-02-26T10:38:54.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.128+0000 7ff299600640 1 -- 172.21.15.67:0/2419511484 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7ff28c059880 con 0x7ff2800051c0 2024-02-26T10:38:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:54.131 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff27c06b550 msgr2=0x7ff27c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff27c06b550 0x7ff27c06da10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0c6350 tx=0x7ff288001150 comp rx=0 tx=0).stop 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 msgr2=0x7ff29c148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff29c148a30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff28c011570 tx=0x7ff28c011a30 comp rx=0 tx=0).stop 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 shutdown_connections 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff29c0c5c70 0x7ff29c1eb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff27c06b550 0x7ff27c06da10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c070c50 0x7ff29c14a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 --2- 172.21.15.67:0/2419511484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2800051c0 0x7ff29c148a30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 >> 172.21.15.67:0/2419511484 conn(0x7ff29c0c63f0 msgr2=0x7ff29c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:54.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 shutdown_connections 2024-02-26T10:38:54.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.131+0000 7ff2a3000640 1 -- 172.21.15.67:0/2419511484 wait complete. 2024-02-26T10:38:54.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:38:54.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:38:54.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.376+0000 7f96a2800640 1 Processor -- start 2024-02-26T10:38:54.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.376+0000 7f96a2800640 1 -- start start 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.377+0000 7f96a2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f969c1591b0 0x7f969c158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.377+0000 7f96a2800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c072ce0 con 0x7f969c0c1400 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.377+0000 7f96a2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c155bf0 con 0x7f969c0c0ef0 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.377+0000 7f96a2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c073010 con 0x7f969c1591b0 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.378+0000 7f969be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f969c0c1400 0x7f969c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56572/0 (socket says 172.21.15.67:56572) 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.378+0000 7f969be00640 1 -- 172.21.15.67:0/3707846795 learned_addr learned my addr 172.21.15.67:0/3707846795 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.378+0000 7f969b400640 1 --2- 172.21.15.67:0/3707846795 >> v2:172.21.15.82:3300/0 conn(0x7f969c1591b0 0x7f969c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.378+0000 7f969aa00640 1 -- 172.21.15.67:0/3707846795 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055473609 0 0) 0x7f969c072ce0 con 0x7f969c0c1400 2024-02-26T10:38:54.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.378+0000 7f969aa00640 1 -- 172.21.15.67:0/3707846795 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9680003520 con 0x7f969c0c1400 2024-02-26T10:38:54.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.379+0000 7f969b400640 1 -- 172.21.15.67:0/3707846795 >> v1:172.21.15.82:6789/0 conn(0x7f969c0c0ef0 legacy=0x7f969c070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.379+0000 7f969b400640 1 -- 172.21.15.67:0/3707846795 >> v1:172.21.15.67:6789/0 conn(0x7f969c0c1400 legacy=0x7f969c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.379+0000 7f969aa00640 1 -- 172.21.15.67:0/3707846795 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573771654 0 0) 0x7f9680003520 con 0x7f969c0c1400 2024-02-26T10:38:54.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.379+0000 7f969b400640 1 -- 172.21.15.67:0/3707846795 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c155ed0 con 0x7f969c1591b0 2024-02-26T10:38:54.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.380+0000 7f969b400640 1 --2- 172.21.15.67:0/3707846795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c1591b0 0x7f969c158580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f968c0097a0 tx=0x7f968c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337bbcd1b364c2d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:54.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.380+0000 7f969aa00640 1 -- 172.21.15.67:0/3707846795 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c037070 con 0x7f969c1591b0 2024-02-26T10:38:54.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.380+0000 7f969aa00640 1 -- 172.21.15.67:0/3707846795 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c030510 con 0x7f969c1591b0 2024-02-26T10:38:54.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f969aa00640 1 -- 172.21.15.67:0/3707846795 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694001a10 con 0x7f969c1591b0 2024-02-26T10:38:54.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 -- 172.21.15.67:0/3707846795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c1591b0 msgr2=0x7f969c158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 --2- 172.21.15.67:0/3707846795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c1591b0 0x7f969c158580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f968c0097a0 tx=0x7f968c02b790 comp rx=0 tx=0).stop 2024-02-26T10:38:54.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 -- 172.21.15.67:0/3707846795 shutdown_connections 2024-02-26T10:38:54.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 --2- 172.21.15.67:0/3707846795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c1591b0 0x7f969c158580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 -- 172.21.15.67:0/3707846795 >> 172.21.15.67:0/3707846795 conn(0x7f969c0c6da0 msgr2=0x7f969c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:54.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 -- 172.21.15.67:0/3707846795 shutdown_connections 2024-02-26T10:38:54.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.381+0000 7f96a2800640 1 -- 172.21.15.67:0/3707846795 wait complete. 2024-02-26T10:38:54.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.383+0000 7f96a2800640 1 Processor -- start 2024-02-26T10:38:54.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.383+0000 7f96a2800640 1 -- start start 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f96a2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 0x7f969c14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f96a2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c0c1400 0x7f969c157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f96a2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f969c1591b0 0x7f969c0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f96a2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c06e560 con 0x7f969c0c0ef0 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f96a2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c14f490 con 0x7f969c0c1400 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f96a2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c06e6e0 con 0x7f969c1591b0 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f969b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 0x7f969c14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f969b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 0x7f969c14b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44498/0 (socket says 172.21.15.67:44498) 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.384+0000 7f969b400640 1 -- 172.21.15.67:0/4072829572 learned_addr learned my addr 172.21.15.67:0/4072829572 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969be00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f969c1591b0 0x7f969c0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f9692a00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c0c1400 0x7f969c157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969b400640 1 -- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f969c1591b0 msgr2=0x7f969c0ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969b400640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f969c1591b0 0x7f969c0ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969b400640 1 -- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c0c1400 msgr2=0x7f969c157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969b400640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c0c1400 0x7f969c157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969b400640 1 -- 172.21.15.67:0/4072829572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c14bf40 con 0x7f969c0c0ef0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f9692a00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c0c1400 0x7f969c157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969be00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f969c1591b0 0x7f969c0ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f969b400640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 0x7f969c14b840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f968c000c00 tx=0x7f968c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c042070 con 0x7f969c0c0ef0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c03e870 con 0x7f969c0c0ef0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c037040 con 0x7f969c0c0ef0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.385+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c1ead20 con 0x7f969c0c0ef0 2024-02-26T10:38:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.386+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c157b80 con 0x7f969c0c0ef0 2024-02-26T10:38:54.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.387+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c074a30 con 0x7f969c0c0ef0 2024-02-26T10:38:54.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.388+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f968c045070 con 0x7f969c0c0ef0 2024-02-26T10:38:54.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.389+0000 7f9693e00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f967806b4d0 0x7f967806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.389+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f968c0abfc0 con 0x7f969c0c0ef0 2024-02-26T10:38:54.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.390+0000 7f9692a00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f967806b4d0 0x7f967806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.391+0000 7f9692a00640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f967806b4d0 0x7f967806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f969c148b70 tx=0x7f9684009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:54.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.393+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f968c07af50 con 0x7f969c0c0ef0 2024-02-26T10:38:54.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.591+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f969c06de50 con 0x7f969c0c0ef0 2024-02-26T10:38:54.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.592+0000 7f9693e00640 1 -- 172.21.15.67:0/4072829572 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f968c07a850 con 0x7f969c0c0ef0 2024-02-26T10:38:54.594 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:38:54.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f967806b4d0 msgr2=0x7f967806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f967806b4d0 0x7f967806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f969c148b70 tx=0x7f9684009040 comp rx=0 tx=0).stop 2024-02-26T10:38:54.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 msgr2=0x7f969c14b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 0x7f969c14b840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f968c000c00 tx=0x7f968c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 shutdown_connections 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f969c1591b0 0x7f969c0ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f967806b4d0 0x7f967806d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c0c1400 0x7f969c157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 --2- 172.21.15.67:0/4072829572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f969c0c0ef0 0x7f969c14b840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 >> 172.21.15.67:0/4072829572 conn(0x7f969c0c6da0 msgr2=0x7f969c0c01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 shutdown_connections 2024-02-26T10:38:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.595+0000 7f96a2800640 1 -- 172.21.15.67:0/4072829572 wait complete. 2024-02-26T10:38:54.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:38:54.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.838+0000 7f65dac00640 1 Processor -- start 2024-02-26T10:38:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.839+0000 7f65dac00640 1 -- start start 2024-02-26T10:38:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.840+0000 7f65dac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d41562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.840+0000 7f65dac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65d4155880 0x7f65d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.840+0000 7f65dac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d406c110 con 0x7f65d406ce30 2024-02-26T10:38:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.840+0000 7f65dac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d40c72c0 con 0x7f65d4155880 2024-02-26T10:38:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.840+0000 7f65dac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40c75a0 con 0x7f65d40c5290 2024-02-26T10:38:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d8e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65d4155880 0x7f65d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d41562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d41562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:33420/0 (socket says 172.21.15.67:33420) 2024-02-26T10:38:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d9800640 1 -- 172.21.15.67:0/1714120303 learned_addr learned my addr 172.21.15.67:0/1714120303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d8e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65d4155880 0x7f65d406c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47772/0 (socket says 172.21.15.67:47772) 2024-02-26T10:38:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d8e00640 1 -- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 msgr2=0x7f65d41562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d8e00640 1 --2- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d41562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d8e00640 1 -- 172.21.15.67:0/1714120303 >> v1:172.21.15.82:6789/0 conn(0x7f65d406ce30 legacy=0x7f65d40af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.841+0000 7f65d8e00640 1 -- 172.21.15.67:0/1714120303 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40c7880 con 0x7f65d4155880 2024-02-26T10:38:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.843+0000 7f65d8e00640 1 --2- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d406c8f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c40097a0 tx=0x7f65c402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27cfe59164927cb2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:54.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.843+0000 7f65d3e00640 1 -- 172.21.15.67:0/1714120303 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4031780 con 0x7f65d4155880 2024-02-26T10:38:54.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.843+0000 7f65d3e00640 1 -- 172.21.15.67:0/1714120303 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4037070 con 0x7f65d4155880 2024-02-26T10:38:54.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.843+0000 7f65d3e00640 1 -- 172.21.15.67:0/1714120303 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4031b40 con 0x7f65d4155880 2024-02-26T10:38:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 -- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 msgr2=0x7f65d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 --2- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d406c8f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f65c40097a0 tx=0x7f65c402b820 comp rx=0 tx=0).stop 2024-02-26T10:38:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 -- 172.21.15.67:0/1714120303 shutdown_connections 2024-02-26T10:38:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 --2- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d406c8f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 --2- 172.21.15.67:0/1714120303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d41562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 -- 172.21.15.67:0/1714120303 >> 172.21.15.67:0/1714120303 conn(0x7f65d40ac230 msgr2=0x7f65d40ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:54.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 -- 172.21.15.67:0/1714120303 shutdown_connections 2024-02-26T10:38:54.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.844+0000 7f65dac00640 1 -- 172.21.15.67:0/1714120303 wait complete. 2024-02-26T10:38:54.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.846+0000 7f65dac00640 1 Processor -- start 2024-02-26T10:38:54.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.846+0000 7f65dac00640 1 -- start start 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65dac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d40ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65dac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d41ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65dac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 0x7f65d41eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65dac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40affb0 con 0x7f65d40c3fd0 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65dac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4155cc0 con 0x7f65d4155880 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65dac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40b0130 con 0x7f65d40c5290 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65da200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 0x7f65d41eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65da200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 0x7f65d41eaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44502/0 (socket says 172.21.15.67:44502) 2024-02-26T10:38:54.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65da200640 1 -- 172.21.15.67:0/3307394824 learned_addr learned my addr 172.21.15.67:0/3307394824 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:54.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65d9800640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d40ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.847+0000 7f65d8e00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d41ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65da200640 1 -- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 msgr2=0x7f65d40ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65da200640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d40ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65da200640 1 -- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 msgr2=0x7f65d41ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65da200640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d41ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65da200640 1 -- 172.21.15.67:0/3307394824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d41ee900 con 0x7f65d40c3fd0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65da200640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 0x7f65d41eaf20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c3f0 tx=0x7f65cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0052d0 con 0x7f65d40c3fd0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc005470 con 0x7f65d40c3fd0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007260 con 0x7f65d40c3fd0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d40c9160 con 0x7f65d40c3fd0 2024-02-26T10:38:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.848+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d40abe70 con 0x7f65d40c3fd0 2024-02-26T10:38:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.850+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d40c9770 con 0x7f65d40c3fd0 2024-02-26T10:38:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.851+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65cc007400 con 0x7f65d40c3fd0 2024-02-26T10:38:54.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.851+0000 7f65d1600640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65b806b500 0x7f65b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:54.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.852+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65cc089460 con 0x7f65d40c3fd0 2024-02-26T10:38:54.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.852+0000 7f65d9800640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65b806b500 0x7f65b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:54.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.853+0000 7f65d9800640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65b806b500 0x7f65b806d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f65d4153d30 tx=0x7f65c800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:54.856+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65cc057d20 con 0x7f65d40c3fd0 2024-02-26T10:38:55.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.054+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65d41591b0 con 0x7f65d40c3fd0 2024-02-26T10:38:55.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.055+0000 7f65d1600640 1 -- 172.21.15.67:0/3307394824 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65d41591b0 con 0x7f65d40c3fd0 2024-02-26T10:38:55.057 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:38:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65b806b500 msgr2=0x7f65b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65b806b500 0x7f65b806d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f65d4153d30 tx=0x7f65c800b040 comp rx=0 tx=0).stop 2024-02-26T10:38:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 msgr2=0x7f65d41eaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 0x7f65d41eaf20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c3f0 tx=0x7f65cc00c8b0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 shutdown_connections 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d40c3fd0 0x7f65d41eaf20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65b806b500 0x7f65b806d9c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d4155880 0x7f65d41ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 --2- 172.21.15.67:0/3307394824 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65d40c5290 0x7f65d40ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 >> 172.21.15.67:0/3307394824 conn(0x7f65d40ac230 msgr2=0x7f65d40ac610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 shutdown_connections 2024-02-26T10:38:55.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.058+0000 7f65dac00640 1 -- 172.21.15.67:0/3307394824 wait complete. 2024-02-26T10:38:55.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:38:55.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.305+0000 7f534de00640 1 Processor -- start 2024-02-26T10:38:55.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.306+0000 7f534de00640 1 -- start start 2024-02-26T10:38:55.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.307+0000 7f534de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c25a0 0x7f53480c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.307+0000 7f534de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53480c1eb0 0x7f534814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f534de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53480c1be0 con 0x7f534814f260 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f534de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53480c1750 con 0x7f53480c1eb0 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f534de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c3660 con 0x7f53480c25a0 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53480c1eb0 0x7f534814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53480c1eb0 0x7f534814ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44506/0 (socket says 172.21.15.67:44506) 2024-02-26T10:38:55.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 -- 172.21.15.67:0/3097426682 learned_addr learned my addr 172.21.15.67:0/3097426682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:55.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f534ca00640 1 --2- 172.21.15.67:0/3097426682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c25a0 0x7f53480c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 -- 172.21.15.67:0/3097426682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c25a0 msgr2=0x7f53480c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:55.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 --2- 172.21.15.67:0/3097426682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c25a0 0x7f53480c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 -- 172.21.15.67:0/3097426682 >> v1:172.21.15.82:6789/0 conn(0x7f534814f260 legacy=0x7f5348151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:55.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.308+0000 7f5347e00640 1 -- 172.21.15.67:0/3097426682 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480c0e20 con 0x7f53480c1eb0 2024-02-26T10:38:55.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.309+0000 7f5347e00640 1 --2- 172.21.15.67:0/3097426682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53480c1eb0 0x7f534814ed20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f533400bb10 tx=0x7f533402e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270e7c08b3a8df51 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:55.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f5347400640 1 -- 172.21.15.67:0/3097426682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533403b070 con 0x7f53480c1eb0 2024-02-26T10:38:55.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f5347400640 1 -- 172.21.15.67:0/3097426682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533402ede0 con 0x7f53480c1eb0 2024-02-26T10:38:55.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f5347400640 1 -- 172.21.15.67:0/3097426682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334036660 con 0x7f53480c1eb0 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f534de00640 1 -- 172.21.15.67:0/3097426682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53480c1eb0 msgr2=0x7f534814ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f534de00640 1 --2- 172.21.15.67:0/3097426682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53480c1eb0 0x7f534814ed20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f533400bb10 tx=0x7f533402e030 comp rx=0 tx=0).stop 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f534de00640 1 -- 172.21.15.67:0/3097426682 shutdown_connections 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f534de00640 1 --2- 172.21.15.67:0/3097426682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53480c1eb0 0x7f534814ed20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.310+0000 7f534de00640 1 --2- 172.21.15.67:0/3097426682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c25a0 0x7f53480c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.311+0000 7f534de00640 1 -- 172.21.15.67:0/3097426682 >> 172.21.15.67:0/3097426682 conn(0x7f53480c6380 msgr2=0x7f53480c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.311+0000 7f534de00640 1 -- 172.21.15.67:0/3097426682 shutdown_connections 2024-02-26T10:38:55.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.311+0000 7f534de00640 1 -- 172.21.15.67:0/3097426682 wait complete. 2024-02-26T10:38:55.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.312+0000 7f534de00640 1 Processor -- start 2024-02-26T10:38:55.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.312+0000 7f534de00640 1 -- start start 2024-02-26T10:38:55.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.313+0000 7f534de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c1eb0 0x7f53481ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.313+0000 7f534de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53480c25a0 0x7f53480bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.313+0000 7f534de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 0x7f53481ead70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.313+0000 7f534de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c3350 con 0x7f534814f260 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.313+0000 7f534de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c13e0 con 0x7f53480c25a0 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.313+0000 7f534de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c2980 con 0x7f53480c1eb0 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 0x7f53481ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 0x7f53481ead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44510/0 (socket says 172.21.15.67:44510) 2024-02-26T10:38:55.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 -- 172.21.15.67:0/1061867947 learned_addr learned my addr 172.21.15.67:0/1061867947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:55.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534ca00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c1eb0 0x7f53481ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f5347e00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53480c25a0 0x7f53480bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 -- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c1eb0 msgr2=0x7f53481ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c1eb0 0x7f53481ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 -- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53480c25a0 msgr2=0x7f53480bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53480c25a0 0x7f53480bfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.314+0000 7f534d400640 1 -- 172.21.15.67:0/1061867947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480c5630 con 0x7f534814f260 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.315+0000 7f534d400640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 0x7f53481ead70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f534000b990 tx=0x7f534000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.315+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340019070 con 0x7f534814f260 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.315+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534000cba0 con 0x7f534814f260 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.315+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340014440 con 0x7f534814f260 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.315+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53480c5910 con 0x7f534814f260 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.315+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53480c5ed0 con 0x7f534814f260 2024-02-26T10:38:55.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.316+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53480678e0 con 0x7f534814f260 2024-02-26T10:38:55.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.317+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f534001c070 con 0x7f534814f260 2024-02-26T10:38:55.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.318+0000 7f5344c00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f532806b4d0 0x7f532806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.318+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5340089ef0 con 0x7f534814f260 2024-02-26T10:38:55.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.319+0000 7f534ca00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f532806b4d0 0x7f532806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.320+0000 7f534ca00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f532806b4d0 0x7f532806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f533c006fd0 tx=0x7f533c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:55.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.324+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5340058e00 con 0x7f534814f260 2024-02-26T10:38:55.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.528+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5348074a30 con 0x7f534814f260 2024-02-26T10:38:55.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.528+0000 7f5344c00640 1 -- 172.21.15.67:0/1061867947 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5340058700 con 0x7f534814f260 2024-02-26T10:38:55.531 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:38:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f532806b4d0 msgr2=0x7f532806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f532806b4d0 0x7f532806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f533c006fd0 tx=0x7f533c008040 comp rx=0 tx=0).stop 2024-02-26T10:38:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 msgr2=0x7f53481ead70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 0x7f53481ead70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f534000b990 tx=0x7f534000be50 comp rx=0 tx=0).stop 2024-02-26T10:38:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 shutdown_connections 2024-02-26T10:38:55.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534814f260 0x7f53481ead70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f532806b4d0 0x7f532806d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53480c25a0 0x7f53480bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 --2- 172.21.15.67:0/1061867947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53480c1eb0 0x7f53481ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 >> 172.21.15.67:0/1061867947 conn(0x7f53480c6380 msgr2=0x7f5348070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:55.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 shutdown_connections 2024-02-26T10:38:55.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.531+0000 7f534de00640 1 -- 172.21.15.67:0/1061867947 wait complete. 2024-02-26T10:38:55.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:38:55.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.774+0000 7f0f1ca00640 1 Processor -- start 2024-02-26T10:38:55.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.774+0000 7f0f1ca00640 1 -- start start 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.775+0000 7f0f1ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f1806ac20 0x7f0f180a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.775+0000 7f0f1ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f1806b000 0x7f0f1806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.775+0000 7f0f1ca00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1806b460 con 0x7f0f18153610 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.775+0000 7f0f1ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f180a9a80 con 0x7f0f1806b000 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.775+0000 7f0f1ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180a9d60 con 0x7f0f1806ac20 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f1806b000 0x7f0f1806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f1806b000 0x7f0f1806db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44516/0 (socket says 172.21.15.67:44516) 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 -- 172.21.15.67:0/4186442167 learned_addr learned my addr 172.21.15.67:0/4186442167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f17400640 1 --2- 172.21.15.67:0/4186442167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f1806ac20 0x7f0f180a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 -- 172.21.15.67:0/4186442167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f1806ac20 msgr2=0x7f0f180a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 --2- 172.21.15.67:0/4186442167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f1806ac20 0x7f0f180a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 -- 172.21.15.67:0/4186442167 >> v1:172.21.15.82:6789/0 conn(0x7f0f18153610 legacy=0x7f0f180abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.776+0000 7f0f16a00640 1 -- 172.21.15.67:0/4186442167 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180ab750 con 0x7f0f1806b000 2024-02-26T10:38:55.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.777+0000 7f0f16a00640 1 --2- 172.21.15.67:0/4186442167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f1806db00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009e50 tx=0x7f0f0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cc9bef42f10714 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.778+0000 7f0f16000640 1 -- 172.21.15.67:0/4186442167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0402ca70 con 0x7f0f1806b000 2024-02-26T10:38:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.778+0000 7f0f16000640 1 -- 172.21.15.67:0/4186442167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0402cc10 con 0x7f0f1806b000 2024-02-26T10:38:55.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.778+0000 7f0f16000640 1 -- 172.21.15.67:0/4186442167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f040343f0 con 0x7f0f1806b000 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.778+0000 7f0f1ca00640 1 -- 172.21.15.67:0/4186442167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 msgr2=0x7f0f1806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.778+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/4186442167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f1806db00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009e50 tx=0x7f0f0402bed0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.779+0000 7f0f1ca00640 1 -- 172.21.15.67:0/4186442167 shutdown_connections 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.779+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/4186442167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f1806db00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.779+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/4186442167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f1806ac20 0x7f0f180a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.779+0000 7f0f1ca00640 1 -- 172.21.15.67:0/4186442167 >> 172.21.15.67:0/4186442167 conn(0x7f0f180c63f0 msgr2=0x7f0f180c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.779+0000 7f0f1ca00640 1 -- 172.21.15.67:0/4186442167 shutdown_connections 2024-02-26T10:38:55.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.779+0000 7f0f1ca00640 1 -- 172.21.15.67:0/4186442167 wait complete. 2024-02-26T10:38:55.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.780+0000 7f0f1ca00640 1 Processor -- start 2024-02-26T10:38:55.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.780+0000 7f0f1ca00640 1 -- start start 2024-02-26T10:38:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.781+0000 7f0f1ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1806ac20 0x7f0f181dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.781+0000 7f0f1ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f181de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.781+0000 7f0f1ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f18153610 0x7f0f181dec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.781+0000 7f0f1ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18150410 con 0x7f0f1806b000 2024-02-26T10:38:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.781+0000 7f0f1ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18153aa0 con 0x7f0f1806ac20 2024-02-26T10:38:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.781+0000 7f0f1ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18150590 con 0x7f0f18153610 2024-02-26T10:38:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f181de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f181de6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44522/0 (socket says 172.21.15.67:44522) 2024-02-26T10:38:55.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 -- 172.21.15.67:0/911578500 learned_addr learned my addr 172.21.15.67:0/911578500 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 -- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f18153610 msgr2=0x7f0f181dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f17400640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1806ac20 0x7f0f181dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f17e00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f18153610 0x7f0f181dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f18153610 0x7f0f181dec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 -- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1806ac20 msgr2=0x7f0f181dc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1806ac20 0x7f0f181dc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 -- 172.21.15.67:0/911578500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1814c590 con 0x7f0f1806b000 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.782+0000 7f0f16a00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f181de6f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f04036040 tx=0x7f0f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.783+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04043020 con 0x7f0f1806b000 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.783+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0403e450 con 0x7f0f1806b000 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.783+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1814c090 con 0x7f0f1806b000 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.783+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04009510 con 0x7f0f1806b000 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.783+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f180c0110 con 0x7f0f1806b000 2024-02-26T10:38:55.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.784+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f18074a30 con 0x7f0f1806b000 2024-02-26T10:38:55.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.785+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f04046020 con 0x7f0f1806b000 2024-02-26T10:38:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.787+0000 7f0f0f400640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ef806b500 0x7f0ef806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:55.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.787+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f040ac870 con 0x7f0f1806b000 2024-02-26T10:38:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.787+0000 7f0f17400640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ef806b500 0x7f0ef806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.788+0000 7f0f17400640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ef806b500 0x7f0ef806d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006fd0 tx=0x7f0f08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:55.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:55.791+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f0404d070 con 0x7f0f1806b000 2024-02-26T10:38:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.027+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f1806b9b0 con 0x7f0f1806b000 2024-02-26T10:38:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.028+0000 7f0f0f400640 1 -- 172.21.15.67:0/911578500 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f0f0407b650 con 0x7f0f1806b000 2024-02-26T10:38:56.031 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:56.031 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:38:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ef806b500 msgr2=0x7f0ef806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ef806b500 0x7f0ef806d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006fd0 tx=0x7f0f08008040 comp rx=0 tx=0).stop 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 msgr2=0x7f0f181de6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f181de6f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f04036040 tx=0x7f0f04004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 shutdown_connections 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f18153610 0x7f0f181dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f1806b000 0x7f0f181de6f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ef806b500 0x7f0ef806d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 --2- 172.21.15.67:0/911578500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f1806ac20 0x7f0f181dc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 >> 172.21.15.67:0/911578500 conn(0x7f0f180c63f0 msgr2=0x7f0f180abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 shutdown_connections 2024-02-26T10:38:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.031+0000 7f0f1ca00640 1 -- 172.21.15.67:0/911578500 wait complete. 2024-02-26T10:38:56.053 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:38:56.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.270+0000 7fab2e600640 1 Processor -- start 2024-02-26T10:38:56.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.270+0000 7fab2e600640 1 -- start start 2024-02-26T10:38:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2e600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab2806ac20 0x7fab2806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab28153630 0x7fab281524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab28155f10 con 0x7fab280c2310 2024-02-26T10:38:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab28153a60 con 0x7fab28153630 2024-02-26T10:38:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2e600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab280c28d0 con 0x7fab2806ac20 2024-02-26T10:38:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab28153630 0x7fab281524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab28153630 0x7fab281524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44524/0 (socket says 172.21.15.67:44524) 2024-02-26T10:38:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2dc00640 1 -- 172.21.15.67:0/2464529183 learned_addr learned my addr 172.21.15.67:0/2464529183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab27e00640 1 --1- 172.21.15.67:0/2464529183 >> v1:172.21.15.67:6789/0 conn(0x7fab280c2310 0x7fab28152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56574/0 (socket says 172.21.15.67:56574) 2024-02-26T10:38:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.272+0000 7fab2d200640 1 --2- 172.21.15.67:0/2464529183 >> v2:172.21.15.82:3300/0 conn(0x7fab2806ac20 0x7fab2806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.273+0000 7fab2dc00640 1 -- 172.21.15.67:0/2464529183 >> v2:172.21.15.82:3300/0 conn(0x7fab2806ac20 msgr2=0x7fab2806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.273+0000 7fab2dc00640 1 --2- 172.21.15.67:0/2464529183 >> v2:172.21.15.82:3300/0 conn(0x7fab2806ac20 0x7fab2806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.273+0000 7fab2dc00640 1 -- 172.21.15.67:0/2464529183 >> v1:172.21.15.67:6789/0 conn(0x7fab280c2310 legacy=0x7fab28152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.273+0000 7fab2dc00640 1 -- 172.21.15.67:0/2464529183 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28155150 con 0x7fab28153630 2024-02-26T10:38:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.273+0000 7fab27400640 1 -- 172.21.15.67:0/2464529183 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175652633 0 0) 0x7fab28155f10 con 0x7fab280c2310 2024-02-26T10:38:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.274+0000 7fab2dc00640 1 --2- 172.21.15.67:0/2464529183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281524b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab2806ee00 tx=0x7fab2002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727d8586539e2a10 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.274+0000 7fab27400640 1 -- 172.21.15.67:0/2464529183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2002c9f0 con 0x7fab28153630 2024-02-26T10:38:56.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.274+0000 7fab27400640 1 -- 172.21.15.67:0/2464529183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2002cb90 con 0x7fab28153630 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.274+0000 7fab27400640 1 -- 172.21.15.67:0/2464529183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab200343f0 con 0x7fab28153630 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 -- 172.21.15.67:0/2464529183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 msgr2=0x7fab281524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 --2- 172.21.15.67:0/2464529183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281524b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab2806ee00 tx=0x7fab2002be50 comp rx=0 tx=0).stop 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 -- 172.21.15.67:0/2464529183 shutdown_connections 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 --2- 172.21.15.67:0/2464529183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281524b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 --2- 172.21.15.67:0/2464529183 >> v2:172.21.15.82:3300/0 conn(0x7fab2806ac20 0x7fab2806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 -- 172.21.15.67:0/2464529183 >> 172.21.15.67:0/2464529183 conn(0x7fab280c63f0 msgr2=0x7fab280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 -- 172.21.15.67:0/2464529183 shutdown_connections 2024-02-26T10:38:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.275+0000 7fab2e600640 1 -- 172.21.15.67:0/2464529183 wait complete. 2024-02-26T10:38:56.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.277+0000 7fab2e600640 1 Processor -- start 2024-02-26T10:38:56.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.277+0000 7fab2e600640 1 -- start start 2024-02-26T10:38:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab2806ac20 0x7fab281443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab280c2310 0x7fab281ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab281559f0 con 0x7fab28153630 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28155710 con 0x7fab280c2310 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab28148850 con 0x7fab2806ac20 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281ee810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44526/0 (socket says 172.21.15.67:44526) 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2dc00640 1 -- 172.21.15.67:0/146283296 learned_addr learned my addr 172.21.15.67:0/146283296 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab2d200640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab2806ac20 0x7fab281443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.278+0000 7fab27e00640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab280c2310 0x7fab281ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2dc00640 1 -- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab2806ac20 msgr2=0x7fab281443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2dc00640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab2806ac20 0x7fab281443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2dc00640 1 -- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab280c2310 msgr2=0x7fab281ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2dc00640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab280c2310 0x7fab281ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2dc00640 1 -- 172.21.15.67:0/146283296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28072270 con 0x7fab28153630 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2dc00640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281ee810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab20009f40 tx=0x7fab20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20044020 con 0x7fab28153630 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20040470 con 0x7fab28153630 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab28072550 con 0x7fab28153630 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.279+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20040860 con 0x7fab28153630 2024-02-26T10:38:56.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.280+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab28072b10 con 0x7fab28153630 2024-02-26T10:38:56.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.281+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab28074a30 con 0x7fab28153630 2024-02-26T10:38:56.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.282+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab20047020 con 0x7fab28153630 2024-02-26T10:38:56.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.283+0000 7fab24c00640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab0806b4d0 0x7fab0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.283+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab200acc70 con 0x7fab28153630 2024-02-26T10:38:56.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.283+0000 7fab2d200640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab0806b4d0 0x7fab0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.284+0000 7fab2d200640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab0806b4d0 0x7fab0806d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009a30 tx=0x7fab1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:56.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.287+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab2004e030 con 0x7fab28153630 2024-02-26T10:38:56.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.493+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab28153be0 con 0x7fab28153630 2024-02-26T10:38:56.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.496+0000 7fab24c00640 1 -- 172.21.15.67:0/146283296 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fab2007bad0 con 0x7fab28153630 2024-02-26T10:38:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:56.498 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:38:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab0806b4d0 msgr2=0x7fab0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab0806b4d0 0x7fab0806d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009a30 tx=0x7fab1c008040 comp rx=0 tx=0).stop 2024-02-26T10:38:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 msgr2=0x7fab281ee810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281ee810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab20009f40 tx=0x7fab20004a40 comp rx=0 tx=0).stop 2024-02-26T10:38:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 shutdown_connections 2024-02-26T10:38:56.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab28153630 0x7fab281ee810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fab0806b4d0 0x7fab0806d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab280c2310 0x7fab281ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 --2- 172.21.15.67:0/146283296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab2806ac20 0x7fab281443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 >> 172.21.15.67:0/146283296 conn(0x7fab280c63f0 msgr2=0x7fab28145460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 shutdown_connections 2024-02-26T10:38:56.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.499+0000 7fab2e600640 1 -- 172.21.15.67:0/146283296 wait complete. 2024-02-26T10:38:56.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:38:56.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.741+0000 7f4996c00640 1 Processor -- start 2024-02-26T10:38:56.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.741+0000 7f4996c00640 1 -- start start 2024-02-26T10:38:56.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f4996c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f499014f3d0 0x7f499006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f4996c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f49900c2870 0x7f4990155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f4996c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49901556a0 0x7f4990071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f4996c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499006dd80 con 0x7f499014f3d0 2024-02-26T10:38:56.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f4996c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499006ed10 con 0x7f49900c2870 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f4996c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990155c00 con 0x7f49901556a0 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f498fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f499014f3d0 0x7f499006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f498fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f499014f3d0 0x7f499006e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44528/0 (socket says 172.21.15.67:44528) 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.743+0000 7f498fe00640 1 -- 172.21.15.67:0/795248933 learned_addr learned my addr 172.21.15.67:0/795248933 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.744+0000 7f498fe00640 1 -- 172.21.15.67:0/795248933 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49901556a0 msgr2=0x7f4990071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.744+0000 7f498fe00640 1 --2- 172.21.15.67:0/795248933 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49901556a0 0x7f4990071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.744+0000 7f498fe00640 1 -- 172.21.15.67:0/795248933 >> v2:172.21.15.82:3300/0 conn(0x7f49900c2870 msgr2=0x7f4990155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.744+0000 7f498fe00640 1 --2- 172.21.15.67:0/795248933 >> v2:172.21.15.82:3300/0 conn(0x7f49900c2870 0x7f4990155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.744+0000 7f498fe00640 1 -- 172.21.15.67:0/795248933 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990155ee0 con 0x7f499014f3d0 2024-02-26T10:38:56.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.745+0000 7f498fe00640 1 --2- 172.21.15.67:0/795248933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f499006e7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49900c1120 tx=0x7f498002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ce93f88f71a48c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:56.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.745+0000 7f498ea00640 1 -- 172.21.15.67:0/795248933 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498002e070 con 0x7f499014f3d0 2024-02-26T10:38:56.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.745+0000 7f498ea00640 1 -- 172.21.15.67:0/795248933 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980037070 con 0x7f499014f3d0 2024-02-26T10:38:56.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.745+0000 7f498ea00640 1 -- 172.21.15.67:0/795248933 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980031a00 con 0x7f499014f3d0 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 -- 172.21.15.67:0/795248933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 msgr2=0x7f499006e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 --2- 172.21.15.67:0/795248933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f499006e7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49900c1120 tx=0x7f498002b880 comp rx=0 tx=0).stop 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 -- 172.21.15.67:0/795248933 shutdown_connections 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 --2- 172.21.15.67:0/795248933 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49901556a0 0x7f4990071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 --2- 172.21.15.67:0/795248933 >> v2:172.21.15.82:3300/0 conn(0x7f49900c2870 0x7f4990155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 --2- 172.21.15.67:0/795248933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f499006e7d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 -- 172.21.15.67:0/795248933 >> 172.21.15.67:0/795248933 conn(0x7f49900a84a0 msgr2=0x7f49900a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 -- 172.21.15.67:0/795248933 shutdown_connections 2024-02-26T10:38:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.746+0000 7f4996c00640 1 -- 172.21.15.67:0/795248933 wait complete. 2024-02-26T10:38:56.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.748+0000 7f4996c00640 1 Processor -- start 2024-02-26T10:38:56.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.748+0000 7f4996c00640 1 -- start start 2024-02-26T10:38:56.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f4996c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49900c2870 0x7f49901419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f4996c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f49901506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f4996c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49901556a0 0x7f4990150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f4996c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c1810 con 0x7f499014f3d0 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f4996c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499006e2b0 con 0x7f49901556a0 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f4996c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c2cd0 con 0x7f49900c2870 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f498f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f49901506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f498f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f49901506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44534/0 (socket says 172.21.15.67:44534) 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f498f400640 1 -- 172.21.15.67:0/1054593880 learned_addr learned my addr 172.21.15.67:0/1054593880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.749+0000 7f498fe00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49900c2870 0x7f49901419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f4994c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49901556a0 0x7f4990150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f498f400640 1 -- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49900c2870 msgr2=0x7f49901419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f498f400640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49900c2870 0x7f49901419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f498f400640 1 -- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49901556a0 msgr2=0x7f4990150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:56.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f498f400640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49901556a0 0x7f4990150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:56.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f498f400640 1 -- 172.21.15.67:0/1054593880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990145560 con 0x7f499014f3d0 2024-02-26T10:38:56.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f498f400640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f49901506e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b440 tx=0x7f497c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:56.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c015020 con 0x7f499014f3d0 2024-02-26T10:38:56.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c011890 con 0x7f499014f3d0 2024-02-26T10:38:56.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c011c80 con 0x7f499014f3d0 2024-02-26T10:38:56.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4990142720 con 0x7f499014f3d0 2024-02-26T10:38:56.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.750+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49901521f0 con 0x7f499014f3d0 2024-02-26T10:38:56.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.751+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4990074a30 con 0x7f499014f3d0 2024-02-26T10:38:56.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.752+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f497c018020 con 0x7f499014f3d0 2024-02-26T10:38:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.754+0000 7f4987e00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f497006b4d0 0x7f497006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.754+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f497c08a340 con 0x7f499014f3d0 2024-02-26T10:38:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.754+0000 7f498fe00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f497006b4d0 0x7f497006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:56.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.755+0000 7f498fe00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f497006b4d0 0x7f497006d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4980036000 tx=0x7f498003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:56.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.758+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f497c01f020 con 0x7f499014f3d0 2024-02-26T10:38:56.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.993+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49900a9550 con 0x7f499014f3d0 2024-02-26T10:38:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.995+0000 7f4987e00640 1 -- 172.21.15.67:0/1054593880 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f497c00d040 con 0x7f499014f3d0 2024-02-26T10:38:56.997 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:38:56.997 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:38:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f497006b4d0 msgr2=0x7f497006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f497006b4d0 0x7f497006d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4980036000 tx=0x7f498003d040 comp rx=0 tx=0).stop 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 msgr2=0x7f49901506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f49901506e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b440 tx=0x7f497c00b900 comp rx=0 tx=0).stop 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 shutdown_connections 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f497006b4d0 0x7f497006d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49901556a0 0x7f4990150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f499014f3d0 0x7f49901506e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 --2- 172.21.15.67:0/1054593880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49900c2870 0x7f49901419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.997+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 >> 172.21.15.67:0/1054593880 conn(0x7f49900a84a0 msgr2=0x7f4990153290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.998+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 shutdown_connections 2024-02-26T10:38:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:56.998+0000 7f4996c00640 1 -- 172.21.15.67:0/1054593880 wait complete. 2024-02-26T10:38:57.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-26T10:38:34.632128+0000', 'modified': '2024-02-26T10:38:37.689492+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/1820988808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 1820988808}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 1820988808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T10:38:57.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-26T10:38:57.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-26T10:38:57.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.650+0000 7f268da00640 1 Processor -- start 2024-02-26T10:38:57.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.651+0000 7f268da00640 1 -- start start 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f268da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2688152380 0x7f26880c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f268da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26880c7020 0x7f2688155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f268da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f268806d290 0x7f26880c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f268da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2688152760 con 0x7f2688152380 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f268da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2688155cd0 con 0x7f268806d290 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f268da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880c7550 con 0x7f26880c7020 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f2687e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2688152380 0x7f26880c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f2687e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2688152380 0x7f26880c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44550/0 (socket says 172.21.15.67:44550) 2024-02-26T10:38:57.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.652+0000 7f2687e00640 1 -- 172.21.15.67:0/711282900 learned_addr learned my addr 172.21.15.67:0/711282900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f2687e00640 1 -- 172.21.15.67:0/711282900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26880c7020 msgr2=0x7f2688155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f268d000640 1 --2- 172.21.15.67:0/711282900 >> v2:172.21.15.82:3300/0 conn(0x7f268806d290 0x7f26880c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f2687400640 1 --2- 172.21.15.67:0/711282900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26880c7020 0x7f2688155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f2687e00640 1 --2- 172.21.15.67:0/711282900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26880c7020 0x7f2688155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f2687e00640 1 -- 172.21.15.67:0/711282900 >> v2:172.21.15.82:3300/0 conn(0x7f268806d290 msgr2=0x7f26880c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f2687e00640 1 --2- 172.21.15.67:0/711282900 >> v2:172.21.15.82:3300/0 conn(0x7f268806d290 0x7f26880c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.653+0000 7f2687e00640 1 -- 172.21.15.67:0/711282900 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880c7830 con 0x7f2688152380 2024-02-26T10:38:57.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.654+0000 7f2687e00640 1 --2- 172.21.15.67:0/711282900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2688152380 0x7f26880c8510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f268806dd70 tx=0x7f267002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbcde6fde6f625f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.654+0000 7f2686a00640 1 -- 172.21.15.67:0/711282900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267002e070 con 0x7f2688152380 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.654+0000 7f2686a00640 1 -- 172.21.15.67:0/711282900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670037070 con 0x7f2688152380 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.654+0000 7f2686a00640 1 -- 172.21.15.67:0/711282900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670031a00 con 0x7f2688152380 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 -- 172.21.15.67:0/711282900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2688152380 msgr2=0x7f26880c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 --2- 172.21.15.67:0/711282900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2688152380 0x7f26880c8510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f268806dd70 tx=0x7f267002b880 comp rx=0 tx=0).stop 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 -- 172.21.15.67:0/711282900 shutdown_connections 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 --2- 172.21.15.67:0/711282900 >> v2:172.21.15.82:3300/0 conn(0x7f268806d290 0x7f26880c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 --2- 172.21.15.67:0/711282900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26880c7020 0x7f2688155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 --2- 172.21.15.67:0/711282900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2688152380 0x7f26880c8510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 -- 172.21.15.67:0/711282900 >> 172.21.15.67:0/711282900 conn(0x7f26880ac1a0 msgr2=0x7f26880ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 -- 172.21.15.67:0/711282900 shutdown_connections 2024-02-26T10:38:57.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.655+0000 7f268da00640 1 -- 172.21.15.67:0/711282900 wait complete. 2024-02-26T10:38:57.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.657+0000 7f268da00640 1 Processor -- start 2024-02-26T10:38:57.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.657+0000 7f268da00640 1 -- start start 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268806d290 0x7f2688154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 0x7f268814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2688152380 0x7f268814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268806d720 con 0x7f26880c7020 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268806cd70 con 0x7f268806d290 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268806d8a0 con 0x7f2688152380 2024-02-26T10:38:57.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 0x7f268814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 0x7f268814b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44556/0 (socket says 172.21.15.67:44556) 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f2687400640 1 -- 172.21.15.67:0/2299337121 learned_addr learned my addr 172.21.15.67:0/2299337121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f2687e00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268806d290 0x7f2688154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.658+0000 7f268d000640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2688152380 0x7f268814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f2687400640 1 -- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2688152380 msgr2=0x7f268814bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f2687400640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2688152380 0x7f268814bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f2687400640 1 -- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268806d290 msgr2=0x7f2688154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f2687400640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268806d290 0x7f2688154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f2687400640 1 -- 172.21.15.67:0/2299337121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880bff50 con 0x7f26880c7020 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f2687400640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 0x7f268814b610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f267800ce40 tx=0x7f267800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f267fe00640 1 -- 172.21.15.67:0/2299337121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678003b90 con 0x7f26880c7020 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f267fe00640 1 -- 172.21.15.67:0/2299337121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678003d30 con 0x7f26880c7020 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f267fe00640 1 -- 172.21.15.67:0/2299337121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678004260 con 0x7f26880c7020 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.659+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26881ee830 con 0x7f26880c7020 2024-02-26T10:38:57.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.660+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2688154690 con 0x7f26880c7020 2024-02-26T10:38:57.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.663+0000 7f267fe00640 1 -- 172.21.15.67:0/2299337121 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2678004400 con 0x7f26880c7020 2024-02-26T10:38:57.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.664+0000 7f267fe00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.664+0000 7f267fe00640 1 -- 172.21.15.67:0/2299337121 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f267808ac80 con 0x7f26880c7020 2024-02-26T10:38:57.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.664+0000 7f2691e18740 1 Processor -- start 2024-02-26T10:38:57.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.664+0000 7f2687e00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.664+0000 7f2691e18740 1 -- start start 2024-02-26T10:38:57.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9f171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2691e18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 0x563fe9f16450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f191e0 0x563fe9edb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2691e18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe9e3f5d0 con 0x563fe9edc410 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe9de8900 con 0x563fe9f16070 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe9e3b670 con 0x563fe9f191e0 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2687e00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2670036000 tx=0x7f267003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.665+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 0x563fe9f16450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 0x563fe9f16450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44572/0 (socket says 172.21.15.67:44572) 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 -- 172.21.15.67:0/3747571669 learned_addr learned my addr 172.21.15.67:0/3747571669 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687e00640 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9f171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 -- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f191e0 msgr2=0x563fe9edb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f191e0 0x563fe9edb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 -- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 msgr2=0x563fe9f171e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9f171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 -- 172.21.15.67:0/3747571669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe9b542d0 con 0x563fe9edc410 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f2687400640 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 0x563fe9f16450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2678061670 tx=0x7f2678058eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52974046459e0332 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f267cc00640 1 -- 172.21.15.67:0/3747571669 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267805d5e0 con 0x563fe9edc410 2024-02-26T10:38:57.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f267cc00640 1 -- 172.21.15.67:0/3747571669 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678061ba0 con 0x563fe9edc410 2024-02-26T10:38:57.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.666+0000 7f267cc00640 1 -- 172.21.15.67:0/3747571669 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26780046f0 con 0x563fe9edc410 2024-02-26T10:38:57.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 -- 172.21.15.67:0/3747571669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 msgr2=0x563fe9f16450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 0x563fe9f16450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2678061670 tx=0x7f2678058eb0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 -- 172.21.15.67:0/3747571669 shutdown_connections 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f191e0 0x563fe9edb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9edc410 0x563fe9f16450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 --2- 172.21.15.67:0/3747571669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9f171e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 -- 172.21.15.67:0/3747571669 >> 172.21.15.67:0/3747571669 conn(0x563fe9ed7a50 msgr2=0x563fe9efc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 -- 172.21.15.67:0/3747571669 shutdown_connections 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.667+0000 7f2691e18740 1 -- 172.21.15.67:0/3747571669 wait complete. 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.668+0000 7f2691e18740 1 -- start start 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.668+0000 7f2691e18740 1 Processor -- start 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 0x563fe9f0efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f191e0 0x563fe9f27210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9ef9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe9f18d50 con 0x563fe9f16070 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe9dfbd60 con 0x563fe9f191e0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe9f18810 con 0x563fe9f2b420 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2691e18740 10 client.0 _ll_register_callbacks cb 0x563fe9e37320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2687e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 0x563fe9f0efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2687e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 0x563fe9f0efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44586/0 (socket says 172.21.15.67:44586) 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.669+0000 7f2687e00640 1 -- 172.21.15.67:0/770690738 learned_addr learned my addr 172.21.15.67:0/770690738 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687400640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f191e0 0x563fe9f27210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f268d000640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9ef9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687e00640 1 -- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 msgr2=0x563fe9ef9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687e00640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9ef9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687e00640 1 -- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f191e0 msgr2=0x563fe9f27210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687e00640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f191e0 0x563fe9f27210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687e00640 1 -- 172.21.15.67:0/770690738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fe9c1e6b0 con 0x563fe9f16070 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2687e00640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 0x563fe9f0efb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2670034040 tx=0x7f2670004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cce6a927437fdd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2675600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:38:57.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2675600640 1 -- 172.21.15.67:0/770690738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670053060 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2675600640 1 -- 172.21.15.67:0/770690738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2670042070 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.670+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2678061ba0 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2675600640 1 -- 172.21.15.67:0/770690738 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2670045600 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 10 client.4656 fetch_fsmap learned FSMap version 8 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563fe9f2a940 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2675600640 1 -- 172.21.15.67:0/770690738 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678061ba0 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2675600640 1 -- 172.21.15.67:0/770690738 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f267004d070 con 0x563fe9f16070 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 20 client.4656 populate_metadata read hostname 'smithi067' 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f1c780 0x563fe9f35dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 4 client.4656 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:38:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.671+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 --> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fe9e4cb10 con 0x563fe9f1c780 2024-02-26T10:38:57.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.672+0000 7f2687400640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f1c780 0x563fe9f35dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.673+0000 7f2687400640 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f1c780 0x563fe9f35dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a1a76e4568fbc4 server_cookie=d399a903cbeb50b2 in_seq=0 out_seq=0 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.673+0000 7f2675600640 10 client.4656 ms_handle_connect on v2:172.21.15.82:6836/1820988808 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2675600640 1 -- 172.21.15.67:0/770690738 <== mds.0 v2:172.21.15.82:6836/1820988808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fe9e4cb10 con 0x563fe9f1c780 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2675600640 10 client.4656 handle_command_reply: tid=0 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 client.4656 shutdown 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 msgr2=0x563fe9f0efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 0x563fe9f0efb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2670034040 tx=0x7f2670004730 comp rx=0 tx=0).stop 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 shutdown_connections 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f1c780 0x563fe9f35dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9ef9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f191e0 0x563fe9f27210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 --2- 172.21.15.67:0/770690738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f16070 0x563fe9f0efb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2675600640 0 client.4656 ms_handle_reset on v2:172.21.15.82:6836/1820988808 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 >> 172.21.15.67:0/770690738 conn(0x563fe9edc410 msgr2=0x563fe9f16d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 shutdown_connections 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.675+0000 7f2691e18740 1 -- 172.21.15.67:0/770690738 wait complete. 2024-02-26T10:38:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.676+0000 7f2691e18740 20 client.4656 trim_cache size 0 max 16384 2024-02-26T10:38:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.693+0000 7f2691e18740 1 Processor -- start 2024-02-26T10:38:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.693+0000 7f2691e18740 1 -- start start 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f16070 0x563fe9f0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2691e18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9ed9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f2b420 0x563fe9eda070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2691e18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881f7270 con 0x563fe9f191e0 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881f6480 con 0x563fe9f2b420 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880bea50 con 0x563fe9f16070 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9ed9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9ed9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44592/0 (socket says 172.21.15.67:44592) 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.694+0000 7f2687400640 1 -- 172.21.15.67:0/1285848866 learned_addr learned my addr 172.21.15.67:0/1285848866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687e00640 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f16070 0x563fe9f0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687400640 1 -- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f16070 msgr2=0x563fe9f0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687400640 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f16070 0x563fe9f0d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687400640 1 -- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f2b420 msgr2=0x563fe9eda070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f268d000640 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f2b420 0x563fe9eda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687400640 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f2b420 0x563fe9eda070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687400640 1 -- 172.21.15.67:0/1285848866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fe9f283e0 con 0x563fe9f191e0 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2687400640 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9ed9b30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2678061670 tx=0x7f267808a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4f2b49de0b2718 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2677e00640 1 -- 172.21.15.67:0/1285848866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678061ba0 con 0x563fe9f191e0 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2677e00640 1 -- 172.21.15.67:0/1285848866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678057790 con 0x563fe9f191e0 2024-02-26T10:38:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.695+0000 7f2677e00640 1 -- 172.21.15.67:0/1285848866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678055b00 con 0x563fe9f191e0 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 -- 172.21.15.67:0/1285848866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 msgr2=0x563fe9ed9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9ed9b30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2678061670 tx=0x7f267808a000 comp rx=0 tx=0).stop 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 -- 172.21.15.67:0/1285848866 shutdown_connections 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f2b420 0x563fe9eda070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9ed9b30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 --2- 172.21.15.67:0/1285848866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f16070 0x563fe9f0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 -- 172.21.15.67:0/1285848866 >> 172.21.15.67:0/1285848866 conn(0x563fe9edc410 msgr2=0x563fe9ed0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 -- 172.21.15.67:0/1285848866 shutdown_connections 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.696+0000 7f2691e18740 1 -- 172.21.15.67:0/1285848866 wait complete. 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.697+0000 7f2691e18740 1 -- start start 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.697+0000 7f2691e18740 1 Processor -- start 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9ed4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9f0e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9f176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fe9ef9c00 con 0x563fe9f191e0 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881f6080 con 0x563fe9f16070 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fe9f1bbd0 con 0x563fe9f2b420 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2691e18740 10 client.0 _ll_register_callbacks cb 0x7f2678007a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:38:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9f0e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9f0e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44606/0 (socket says 172.21.15.67:44606) 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 -- 172.21.15.67:0/4149889288 learned_addr learned my addr 172.21.15.67:0/4149889288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687e00640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9ed4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f268d000640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9f176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 -- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 msgr2=0x563fe9f176f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9f176f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 -- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 msgr2=0x563fe9ed4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9ed4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687400640 1 -- 172.21.15.67:0/4149889288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fe9f0d9e0 con 0x563fe9f191e0 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.698+0000 7f2687e00640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9ed4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2687400640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9f0e230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2678059190 tx=0x7f267800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754bd6fd36f19e57 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2675600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2675600640 1 -- 172.21.15.67:0/4149889288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678061ba0 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2675600640 1 -- 172.21.15.67:0/4149889288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2678082860 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2678057790 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2675600640 1 -- 172.21.15.67:0/4149889288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2678055b00 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2691e18740 10 client.4662 fetch_fsmap learned FSMap version 8 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563fe9efcf20 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2675600640 1 -- 172.21.15.67:0/4149889288 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678057790 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2675600640 1 -- 172.21.15.67:0/4149889288 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2678076ce0 con 0x563fe9f191e0 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2691e18740 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.699+0000 7f2691e18740 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.700+0000 7f2691e18740 20 client.4662 populate_metadata read hostname 'smithi067' 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.700+0000 7f2691e18740 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f7f510 0x563fe9ed5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.700+0000 7f2691e18740 4 client.4662 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:38:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.700+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 --> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563fe9e46e60 con 0x563fe9f7f510 2024-02-26T10:38:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.700+0000 7f2687e00640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f7f510 0x563fe9ed5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.701+0000 7f2687e00640 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f7f510 0x563fe9ed5680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94cdd09c5567c18 server_cookie=7b52178b27c8f791 in_seq=0 out_seq=0 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.701+0000 7f2675600640 10 client.4662 ms_handle_connect on v2:172.21.15.82:6836/1820988808 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2675600640 1 -- 172.21.15.67:0/4149889288 <== mds.0 v2:172.21.15.82:6836/1820988808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x563fe9e46e60 con 0x563fe9f7f510 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2675600640 10 client.4662 handle_command_reply: tid=0 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 client.4662 shutdown 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 msgr2=0x563fe9f0e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9f0e230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2678059190 tx=0x7f267800cf30 comp rx=0 tx=0).stop 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 shutdown_connections 2024-02-26T10:38:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:6836/1820988808,v1:172.21.15.82:6837/1820988808] conn(0x563fe9f7f510 0x563fe9ed5680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563fe9f2b420 0x563fe9f176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2675600640 0 client.4662 ms_handle_reset on v2:172.21.15.82:6836/1820988808 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563fe9f191e0 0x563fe9f0e230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 --2- 172.21.15.67:0/4149889288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fe9f16070 0x563fe9ed4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 >> 172.21.15.67:0/4149889288 conn(0x563fe9edc410 msgr2=0x563fe9f0edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 shutdown_connections 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 1 -- 172.21.15.67:0/4149889288 wait complete. 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.702+0000 7f2691e18740 20 client.4662 trim_cache size 0 max 16384 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "f4a3a702-2ce8-46d3-b32d-e4f6872b8b58", 2024-02-26T10:38:57.708 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4264, 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 22.536562670999999, 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 8, 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 21, 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 21, 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 30.097593002 2024-02-26T10:38:57.709 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 msgr2=0x7f266406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2670036000 tx=0x7f267003d040 comp rx=0 tx=0).stop 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 msgr2=0x7f268814b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 0x7f268814b610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f267800ce40 tx=0x7f267800a660 comp rx=0 tx=0).stop 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 shutdown_connections 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2688152380 0x7f268814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26880c7020 0x7f268814b610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 --2- 172.21.15.67:0/2299337121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268806d290 0x7f2688154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 >> 172.21.15.67:0/2299337121 conn(0x7f26880ac1a0 msgr2=0x7f2688150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 shutdown_connections 2024-02-26T10:38:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:57.703+0000 7f268da00640 1 -- 172.21.15.67:0/2299337121 wait complete. 2024-02-26T10:38:57.721 INFO:teuthology.run_tasks:Running task kclient... 2024-02-26T10:38:57.745 INFO:tasks.kclient:Mounting kernel clients... 2024-02-26T10:38:57.745 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-02-26T10:38:57.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:57.745 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:38:57.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:57.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-02-26T10:38:57.907 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2024-02-26T10:38:57.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:38:57.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:57.911 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:38:57.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:57.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-02-26T10:38:58.054 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-02-26T10:38:58.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:38:58.058 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-02-26T10:38:58.058 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T10:38:58.058 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:38:58.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:38:58.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.284+0000 7f14df200640 1 Processor -- start 2024-02-26T10:38:58.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.284+0000 7f14df200640 1 -- start start 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.285+0000 7f14df200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14d80c4a50 0x7f14d80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.285+0000 7f14df200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f14d80c2770 0x7f14d8155340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.285+0000 7f14df200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d8159130 con 0x7f14d81581f0 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.285+0000 7f14df200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d8154eb0 con 0x7f14d80c4a50 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.285+0000 7f14df200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d8070010 con 0x7f14d80c2770 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14d80c4a50 0x7f14d80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14d80c4a50 0x7f14d80c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44620/0 (socket says 172.21.15.67:44620) 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 -- 172.21.15.67:0/705688121 learned_addr learned my addr 172.21.15.67:0/705688121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dd400640 1 --2- 172.21.15.67:0/705688121 >> v2:172.21.15.82:3300/0 conn(0x7f14d80c2770 0x7f14d8155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:58.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 -- 172.21.15.67:0/705688121 >> v2:172.21.15.82:3300/0 conn(0x7f14d80c2770 msgr2=0x7f14d8155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:58.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 --2- 172.21.15.67:0/705688121 >> v2:172.21.15.82:3300/0 conn(0x7f14d80c2770 0x7f14d8155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 -- 172.21.15.67:0/705688121 >> v1:172.21.15.82:6789/0 conn(0x7f14d81581f0 legacy=0x7f14d80aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:38:58.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.286+0000 7f14dde00640 1 -- 172.21.15.67:0/705688121 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80c6bf0 con 0x7f14d80c4a50 2024-02-26T10:38:58.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.287+0000 7f14dde00640 1 --2- 172.21.15.67:0/705688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d80c4a50 0x7f14d80c4e30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14d806ba40 tx=0x7f14cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4d86d4646a864b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:58.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14dca00640 1 -- 172.21.15.67:0/705688121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc02e070 con 0x7f14d80c4a50 2024-02-26T10:38:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14dca00640 1 -- 172.21.15.67:0/705688121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc037070 con 0x7f14d80c4a50 2024-02-26T10:38:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14dca00640 1 -- 172.21.15.67:0/705688121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc031a00 con 0x7f14d80c4a50 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14df200640 1 -- 172.21.15.67:0/705688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d80c4a50 msgr2=0x7f14d80c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14df200640 1 --2- 172.21.15.67:0/705688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d80c4a50 0x7f14d80c4e30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14d806ba40 tx=0x7f14cc02b880 comp rx=0 tx=0).stop 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14df200640 1 -- 172.21.15.67:0/705688121 shutdown_connections 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14df200640 1 --2- 172.21.15.67:0/705688121 >> v2:172.21.15.82:3300/0 conn(0x7f14d80c2770 0x7f14d8155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14df200640 1 --2- 172.21.15.67:0/705688121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d80c4a50 0x7f14d80c4e30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.288+0000 7f14df200640 1 -- 172.21.15.67:0/705688121 >> 172.21.15.67:0/705688121 conn(0x7f14d806f8e0 msgr2=0x7f14d806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.289+0000 7f14df200640 1 -- 172.21.15.67:0/705688121 shutdown_connections 2024-02-26T10:38:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.289+0000 7f14df200640 1 -- 172.21.15.67:0/705688121 wait complete. 2024-02-26T10:38:58.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.290+0000 7f14df200640 1 Processor -- start 2024-02-26T10:38:58.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.290+0000 7f14df200640 1 -- start start 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14df200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14d80c4a50 0x7f14d814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14df200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d81581f0 0x7f14d81ee210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14df200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 0x7f14d814d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14df200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d8152660 con 0x7f14d8152ac0 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14df200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80c71b0 con 0x7f14d81581f0 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14df200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80c1710 con 0x7f14d80c4a50 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.291+0000 7f14de800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 0x7f14d814d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:58.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 0x7f14d814d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44626/0 (socket says 172.21.15.67:44626) 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 -- 172.21.15.67:0/3181195034 learned_addr learned my addr 172.21.15.67:0/3181195034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14dde00640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14d80c4a50 0x7f14d814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 -- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14d80c4a50 msgr2=0x7f14d814f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14d80c4a50 0x7f14d814f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 -- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d81581f0 msgr2=0x7f14d81ee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d81581f0 0x7f14d81ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 -- 172.21.15.67:0/3181195034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80a89f0 con 0x7f14d8152ac0 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14dde00640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14d80c4a50 0x7f14d814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:38:58.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.292+0000 7f14de800640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 0x7f14d814d6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14d400b9e0 tx=0x7f14d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.293+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4019070 con 0x7f14d8152ac0 2024-02-26T10:38:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.293+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d400cac0 con 0x7f14d8152ac0 2024-02-26T10:38:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.293+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4014440 con 0x7f14d8152ac0 2024-02-26T10:38:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.293+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d806cdc0 con 0x7f14d8152ac0 2024-02-26T10:38:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.293+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d814f360 con 0x7f14d8152ac0 2024-02-26T10:38:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.294+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d8075020 con 0x7f14d8152ac0 2024-02-26T10:38:58.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.295+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14d401c360 con 0x7f14d8152ac0 2024-02-26T10:38:58.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.296+0000 7f14d2000640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14bc06b500 0x7f14bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:38:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.296+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14d4089c80 con 0x7f14d8152ac0 2024-02-26T10:38:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.296+0000 7f14dde00640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14bc06b500 0x7f14bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:38:58.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.298+0000 7f14dde00640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14bc06b500 0x7f14bc06d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14cc036000 tx=0x7f14cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:38:58.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.300+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14d8075020 con 0x7f14d8152ac0 2024-02-26T10:38:58.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.537+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f14d8074a30 con 0x7f14d8152ac0 2024-02-26T10:38:58.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.538+0000 7f14d2000640 1 -- 172.21.15.67:0/3181195034 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f14d4058b90 con 0x7f14d8152ac0 2024-02-26T10:38:58.540 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:38:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14bc06b500 msgr2=0x7f14bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14bc06b500 0x7f14bc06d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14cc036000 tx=0x7f14cc008030 comp rx=0 tx=0).stop 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 msgr2=0x7f14d814d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 0x7f14d814d6c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14d400b9e0 tx=0x7f14d400bea0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 shutdown_connections 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14d8152ac0 0x7f14d814d6c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14bc06b500 0x7f14bc06d9c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14d81581f0 0x7f14d81ee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 --2- 172.21.15.67:0/3181195034 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14d80c4a50 0x7f14d814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 >> 172.21.15.67:0/3181195034 conn(0x7f14d806f8e0 msgr2=0x7f14d8157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 shutdown_connections 2024-02-26T10:38:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:38:58.541+0000 7f14df200640 1 -- 172.21.15.67:0/3181195034 wait complete. 2024-02-26T10:38:58.561 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-26T10:38:58.561 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:38:58.561 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:38:58.561 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T10:38:58.562 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:38:58.562 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-26T10:38:58.562 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T10:38:58.562 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:38:58.562 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:38:58.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:58.562 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:38:58.612 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:38:58.613 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42512sec preferred_lft 42512sec 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:38:58.614 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:38:58.615 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T10:38:58.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:58.615 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:38:58.615 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:38:58.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-02-26T10:38:58.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-02-26T10:38:58.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-02-26T10:38:58.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T10:38:58.616 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:38:59.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:59.080 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:38:59.168 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:38:59.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:59.172 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:38:59.248 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:38:59.249 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:38:59.249 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:38:59.249 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:38:59.249 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:38:59.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:59.250 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:38:59.250 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:38:59.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:38:59.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:38:59.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:38:59.250 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:38:59.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:38:59.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:38:59.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:38:59.974 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:39:00.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:00.059 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-02-26T10:39:00.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:00.150 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:39:00.150 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:39:00.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:39:00.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-26T10:39:00.150 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:39:00.279 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-26T10:39:00.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-26T10:39:00.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-26T10:39:00.280 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:39:00.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:00.537 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:39:00.537 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:39:00.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:39:00.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T10:39:00.537 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:39:00.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:00.693 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:00.738 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:39:00.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:00.738 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:00.830 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-26T10:39:00.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:00.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:01.006 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:01.614 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:01.614 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:01.614 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:39:01.614 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:39:01.614 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:39:01.618 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T10:39:01.618 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:39:01.618 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:39:01.618 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:39:01.618 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:39:01.624 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:39:01.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:01.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:01.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:01.724 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:01.803 INFO:teuthology.orchestra.run.smithi067.stdout:client4561 2024-02-26T10:39:01.803 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:39:01.803 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:01.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:01.861 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:01.940 INFO:teuthology.orchestra.run.smithi067.stdout:client4561 2024-02-26T10:39:01.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:01.941 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:02.024 INFO:teuthology.orchestra.run.smithi067.stdout:client4561 2024-02-26T10:39:02.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:02.024 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.client4561/status') 2024-02-26T10:39:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.4561 (3)172.21.15.67:0/566410641 2024-02-26T10:39:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2024-02-26T10:39:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2024-02-26T10:39:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2024-02-26T10:39:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:69 bytes copied, 8.9562e-05 s, 770 kB/s 2024-02-26T10:39:02.139 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-02-26T10:39:02.139 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-26T10:39:02.139 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:02.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:39:02.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.374+0000 7f76f1e00640 1 Processor -- start 2024-02-26T10:39:02.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.374+0000 7f76f1e00640 1 -- start start 2024-02-26T10:39:02.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec0a94a0 0x7f76ec06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:02.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76f1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76ec06db00 0x7f76ec0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76f1e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ec06df90 con 0x7f76ec06ac20 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76f1e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ec0a9a80 con 0x7f76ec06db00 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76f1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0a9d60 con 0x7f76ec0a94a0 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76f1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76ec06db00 0x7f76ec0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.376+0000 7f76ebe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec0a94a0 0x7f76ec06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76ebe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec0a94a0 0x7f76ec06b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:54852/0 (socket says 172.21.15.67:54852) 2024-02-26T10:39:02.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76f1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76ec06db00 0x7f76ec0abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:53550/0 (socket says 172.21.15.67:53550) 2024-02-26T10:39:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76ebe00640 1 -- 172.21.15.67:0/3274717753 learned_addr learned my addr 172.21.15.67:0/3274717753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76f1400640 1 -- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec0a94a0 msgr2=0x7f76ec06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76f1400640 1 --2- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec0a94a0 0x7f76ec06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76f1400640 1 -- 172.21.15.67:0/3274717753 >> v1:172.21.15.82:6789/0 conn(0x7f76ec06ac20 legacy=0x7f76ec153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.377+0000 7f76f1400640 1 -- 172.21.15.67:0/3274717753 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0ab750 con 0x7f76ec06db00 2024-02-26T10:39:02.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.379+0000 7f76f1400640 1 --2- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06db00 0x7f76ec0abf50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009d90 tx=0x7f76e402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87de7b709f7c96ee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:02.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.379+0000 7f76eb400640 1 -- 172.21.15.67:0/3274717753 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4039070 con 0x7f76ec06db00 2024-02-26T10:39:02.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.379+0000 7f76eb400640 1 -- 172.21.15.67:0/3274717753 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e402cc50 con 0x7f76ec06db00 2024-02-26T10:39:02.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.379+0000 7f76eb400640 1 -- 172.21.15.67:0/3274717753 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4034440 con 0x7f76ec06db00 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 -- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06db00 msgr2=0x7f76ec0abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 --2- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06db00 0x7f76ec0abf50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009d90 tx=0x7f76e402bea0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 -- 172.21.15.67:0/3274717753 shutdown_connections 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 --2- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06db00 0x7f76ec0abf50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 --2- 172.21.15.67:0/3274717753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec0a94a0 0x7f76ec06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 -- 172.21.15.67:0/3274717753 >> 172.21.15.67:0/3274717753 conn(0x7f76ec0c63f0 msgr2=0x7f76ec0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:02.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.380+0000 7f76f1e00640 1 -- 172.21.15.67:0/3274717753 shutdown_connections 2024-02-26T10:39:02.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.381+0000 7f76f1e00640 1 -- 172.21.15.67:0/3274717753 wait complete. 2024-02-26T10:39:02.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.383+0000 7f76f1e00640 1 Processor -- start 2024-02-26T10:39:02.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.383+0000 7f76f1e00640 1 -- start start 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06ac20 0x7f76ec1ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec06db00 0x7f76ec156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 0x7f76ec14a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec150410 con 0x7f76ec0a94a0 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0ac490 con 0x7f76ec06ac20 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec150590 con 0x7f76ec06db00 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.384+0000 7f76f1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 0x7f76ec14a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 0x7f76ec14a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47658/0 (socket says 172.21.15.67:47658) 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 -- 172.21.15.67:0/1115146183 learned_addr learned my addr 172.21.15.67:0/1115146183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76ebe00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec06db00 0x7f76ec156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:02.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f0a00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06ac20 0x7f76ec1ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 -- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec06db00 msgr2=0x7f76ec156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec06db00 0x7f76ec156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 -- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06ac20 msgr2=0x7f76ec1ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06ac20 0x7f76ec1ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 -- 172.21.15.67:0/1115146183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec1ed240 con 0x7f76ec0a94a0 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f0a00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06ac20 0x7f76ec1ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:02.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.385+0000 7f76f1400640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 0x7f76ec14a1b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009d90 tx=0x7f76e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.386+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4039050 con 0x7f76ec0a94a0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.386+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4045070 con 0x7f76ec0a94a0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.386+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4009450 con 0x7f76ec0a94a0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.386+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec154d60 con 0x7f76ec0a94a0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.386+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec14b510 con 0x7f76ec0a94a0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.387+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ec0a9290 con 0x7f76ec0a94a0 2024-02-26T10:39:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.388+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76e403e610 con 0x7f76ec0a94a0 2024-02-26T10:39:02.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.389+0000 7f76e8c00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:02.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.389+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e40acc80 con 0x7f76ec0a94a0 2024-02-26T10:39:02.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.389+0000 7f76f0a00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.390+0000 7f76f0a00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f76e0001680 tx=0x7f76e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:02.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.394+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76ec0a9290 con 0x7f76ec0a94a0 2024-02-26T10:39:02.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.631+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f76ec155280 con 0x7f76ec0a94a0 2024-02-26T10:39:02.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.632+0000 7f76e8c00640 1 -- 172.21.15.67:0/1115146183 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f76ec1ed240 con 0x7f76ec0a94a0 2024-02-26T10:39:02.635 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:39:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.634+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 msgr2=0x7f76cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.634+0000 7f76f1e00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f76e0001680 tx=0x7f76e0009040 comp rx=0 tx=0).stop 2024-02-26T10:39:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.634+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 msgr2=0x7f76ec14a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.634+0000 7f76f1e00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 0x7f76ec14a1b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009d90 tx=0x7f76e4004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 shutdown_connections 2024-02-26T10:39:02.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec0a94a0 0x7f76ec14a1b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec06db00 0x7f76ec156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 --2- 172.21.15.67:0/1115146183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec06ac20 0x7f76ec1ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 >> 172.21.15.67:0/1115146183 conn(0x7f76ec0c63f0 msgr2=0x7f76ec06b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 shutdown_connections 2024-02-26T10:39:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:02.635+0000 7f76f1e00640 1 -- 172.21.15.67:0/1115146183 wait complete. 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:39:02.658 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:39:02.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:02.659 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:39:02.717 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42336sec preferred_lft 42336sec 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:02.718 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-26T10:39:02.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-26T10:39:02.719 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:39:03.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:03.298 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:39:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:39:03.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:03.377 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:39:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:39:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:39:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:39:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:39:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:39:03.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:03.454 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:39:03.454 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:39:03.454 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:39:03.454 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:39:03.454 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:39:03.454 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:39:04.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:04.209 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:39:04.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:04.268 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-02-26T10:39:04.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:04.321 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:39:04.321 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:39:04.321 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:39:04.321 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-26T10:39:04.321 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:39:04.465 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-26T10:39:04.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-26T10:39:04.465 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:39:04.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:04.746 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:39:04.747 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:39:04.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:39:04.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-02-26T10:39:04.747 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:39:04.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:04.910 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:04.956 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:39:04.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:04.956 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:05.053 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-26T10:39:05.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:05.053 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:05.197 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:05.197 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:05.826 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:05.826 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:05.826 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:39:05.826 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:39:05.826 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:39:05.829 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:39:05.829 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:39:05.829 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:39:05.829 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:39:05.829 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:39:05.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:39:05.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:05.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:05.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:05.925 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:client4496 2024-02-26T10:39:05.982 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:39:05.982 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:06.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:06.038 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:06.090 INFO:teuthology.orchestra.run.smithi082.stdout:client4496 2024-02-26T10:39:06.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:06.091 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:06.145 INFO:teuthology.orchestra.run.smithi082.stdout:client4496 2024-02-26T10:39:06.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:06.146 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.client4496/status') 2024-02-26T10:39:06.234 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4496 (3)192.168.0.1:0/359294535 2024-02-26T10:39:06.235 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-02-26T10:39:06.235 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-02-26T10:39:06.235 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-02-26T10:39:06.235 INFO:teuthology.orchestra.run.smithi082.stderr:68 bytes copied, 7.563e-05 s, 899 kB/s 2024-02-26T10:39:06.238 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-02-26T10:39:06.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:06.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.488+0000 7f3063e00640 1 Processor -- start 2024-02-26T10:39:06.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.488+0000 7f3063e00640 1 -- start start 2024-02-26T10:39:06.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.489+0000 7f3063e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f306406ac20 0x7f30640a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.489+0000 7f3063e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f306406b000 0x7f306406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.489+0000 7f3063e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306406b460 con 0x7f3064153610 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.489+0000 7f3063e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30640a9a80 con 0x7f306406b000 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.489+0000 7f3063e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30640a9d60 con 0x7f306406ac20 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f306406b000 0x7f306406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f306406b000 0x7f306406db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47670/0 (socket says 172.21.15.67:47670) 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 -- 172.21.15.67:0/542496087 learned_addr learned my addr 172.21.15.67:0/542496087 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:06.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062a00640 1 --2- 172.21.15.67:0/542496087 >> v2:172.21.15.82:3300/0 conn(0x7f306406ac20 0x7f30640a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 -- 172.21.15.67:0/542496087 >> v2:172.21.15.82:3300/0 conn(0x7f306406ac20 msgr2=0x7f30640a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:06.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 --2- 172.21.15.67:0/542496087 >> v2:172.21.15.82:3300/0 conn(0x7f306406ac20 0x7f30640a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 -- 172.21.15.67:0/542496087 >> v1:172.21.15.82:6789/0 conn(0x7f3064153610 legacy=0x7f30640abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:06.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.490+0000 7f3062000640 1 -- 172.21.15.67:0/542496087 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30640ab750 con 0x7f306406b000 2024-02-26T10:39:06.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.492+0000 7f3062000640 1 --2- 172.21.15.67:0/542496087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f306406db00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f306406ecb0 tx=0x7f304c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53d9351663317c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.492+0000 7f3061600640 1 -- 172.21.15.67:0/542496087 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c035020 con 0x7f306406b000 2024-02-26T10:39:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.492+0000 7f3061600640 1 -- 172.21.15.67:0/542496087 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c031870 con 0x7f306406b000 2024-02-26T10:39:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.492+0000 7f3061600640 1 -- 172.21.15.67:0/542496087 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c031c40 con 0x7f306406b000 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 -- 172.21.15.67:0/542496087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 msgr2=0x7f306406db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 --2- 172.21.15.67:0/542496087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f306406db00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f306406ecb0 tx=0x7f304c02b910 comp rx=0 tx=0).stop 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 -- 172.21.15.67:0/542496087 shutdown_connections 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 --2- 172.21.15.67:0/542496087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f306406db00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 --2- 172.21.15.67:0/542496087 >> v2:172.21.15.82:3300/0 conn(0x7f306406ac20 0x7f30640a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 -- 172.21.15.67:0/542496087 >> 172.21.15.67:0/542496087 conn(0x7f30640c63f0 msgr2=0x7f30640c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 -- 172.21.15.67:0/542496087 shutdown_connections 2024-02-26T10:39:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.493+0000 7f3063e00640 1 -- 172.21.15.67:0/542496087 wait complete. 2024-02-26T10:39:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.495+0000 7f3063e00640 1 Processor -- start 2024-02-26T10:39:06.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.495+0000 7f3063e00640 1 -- start start 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f306406ac20 0x7f3064149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f30640bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3064153610 0x7f30640c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064150410 con 0x7f306406b000 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064153a60 con 0x7f306406ac20 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3064150590 con 0x7f3064153610 2024-02-26T10:39:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3062000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f30640bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3062000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f30640bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47672/0 (socket says 172.21.15.67:47672) 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3062000640 1 -- 172.21.15.67:0/61644091 learned_addr learned my addr 172.21.15.67:0/61644091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3062a00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f306406ac20 0x7f3064149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3063400640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3064153610 0x7f30640c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.496+0000 7f3062000640 1 -- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3064153610 msgr2=0x7f30640c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3062000640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3064153610 0x7f30640c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3062000640 1 -- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f306406ac20 msgr2=0x7f3064149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3062000640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f306406ac20 0x7f3064149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3062000640 1 -- 172.21.15.67:0/61644091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064152e30 con 0x7f306406b000 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3063400640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3064153610 0x7f30640c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3062a00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f306406ac20 0x7f3064149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3062000640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f30640bfb90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f304c000c00 tx=0x7f304c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c040070 con 0x7f306406b000 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c030d00 con 0x7f306406b000 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c035040 con 0x7f306406b000 2024-02-26T10:39:06.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3064151500 con 0x7f306406b000 2024-02-26T10:39:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.497+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30640c0750 con 0x7f306406b000 2024-02-26T10:39:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.498+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3064151720 con 0x7f306406b000 2024-02-26T10:39:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.499+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f304c04b060 con 0x7f306406b000 2024-02-26T10:39:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.500+0000 7f305b400640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.501+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f304c0abfe0 con 0x7f306406b000 2024-02-26T10:39:06.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.501+0000 7f3062a00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.502+0000 7f3062a00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f304006b4d0 0x7f304006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3054006fd0 tx=0x7f3054008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:06.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.505+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f304c07aeb0 con 0x7f306406b000 2024-02-26T10:39:06.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.704+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3064074a30 con 0x7f306406b000 2024-02-26T10:39:06.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.707+0000 7f305b400640 1 -- 172.21.15.67:0/61644091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f304c07a7b0 con 0x7f306406b000 2024-02-26T10:39:06.710 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:06.710 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f304006b4d0 msgr2=0x7f304006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f304006b4d0 0x7f304006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3054006fd0 tx=0x7f3054008040 comp rx=0 tx=0).stop 2024-02-26T10:39:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 msgr2=0x7f30640bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f30640bfb90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f304c000c00 tx=0x7f304c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 shutdown_connections 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3064153610 0x7f30640c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f306406b000 0x7f30640bfb90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 --2- 172.21.15.67:0/61644091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f306406ac20 0x7f3064149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.710+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 >> 172.21.15.67:0/61644091 conn(0x7f30640c63f0 msgr2=0x7f30640c2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.711+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 shutdown_connections 2024-02-26T10:39:06.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.711+0000 7f3063e00640 1 -- 172.21.15.67:0/61644091 wait complete. 2024-02-26T10:39:06.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:06.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:06.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.956+0000 7fcedca00640 1 Processor -- start 2024-02-26T10:39:06.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.957+0000 7fcedca00640 1 -- start start 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fcedca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fced80c5290 0x7fced81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fcedca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fced8155880 0x7fced806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fcedca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced806ce30 0x7fced80af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fcedca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced806c110 con 0x7fced80c5290 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fcedca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced80c72c0 con 0x7fced8155880 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fcedca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced80c75a0 con 0x7fced806ce30 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fced7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fced80c5290 0x7fced81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fced7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fced80c5290 0x7fced81562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47678/0 (socket says 172.21.15.67:47678) 2024-02-26T10:39:06.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.958+0000 7fced7400640 1 -- 172.21.15.67:0/4209169620 learned_addr learned my addr 172.21.15.67:0/4209169620 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced6a00640 1 --2- 172.21.15.67:0/4209169620 >> v2:172.21.15.82:3300/0 conn(0x7fced8155880 0x7fced806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced7e00640 1 --2- 172.21.15.67:0/4209169620 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced806ce30 0x7fced80af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced7400640 1 -- 172.21.15.67:0/4209169620 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced806ce30 msgr2=0x7fced80af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:06.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced7400640 1 --2- 172.21.15.67:0/4209169620 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced806ce30 0x7fced80af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced7400640 1 -- 172.21.15.67:0/4209169620 >> v2:172.21.15.82:3300/0 conn(0x7fced8155880 msgr2=0x7fced806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced7400640 1 --2- 172.21.15.67:0/4209169620 >> v2:172.21.15.82:3300/0 conn(0x7fced8155880 0x7fced806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced7400640 1 -- 172.21.15.67:0/4209169620 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80c7880 con 0x7fced80c5290 2024-02-26T10:39:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.959+0000 7fced6a00640 1 --2- 172.21.15.67:0/4209169620 >> v2:172.21.15.82:3300/0 conn(0x7fced8155880 0x7fced806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:06.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.960+0000 7fced7400640 1 --2- 172.21.15.67:0/4209169620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced81562f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fced80cb770 tx=0x7fcec802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8470c621fe2c630 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.960+0000 7fced6000640 1 -- 172.21.15.67:0/4209169620 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec802c9f0 con 0x7fced80c5290 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.960+0000 7fced6000640 1 -- 172.21.15.67:0/4209169620 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec802cb90 con 0x7fced80c5290 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.960+0000 7fced6000640 1 -- 172.21.15.67:0/4209169620 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec80343f0 con 0x7fced80c5290 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 -- 172.21.15.67:0/4209169620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 msgr2=0x7fced81562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 --2- 172.21.15.67:0/4209169620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced81562f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fced80cb770 tx=0x7fcec802be50 comp rx=0 tx=0).stop 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 -- 172.21.15.67:0/4209169620 shutdown_connections 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 --2- 172.21.15.67:0/4209169620 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced806ce30 0x7fced80af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 --2- 172.21.15.67:0/4209169620 >> v2:172.21.15.82:3300/0 conn(0x7fced8155880 0x7fced806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 --2- 172.21.15.67:0/4209169620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced81562f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 -- 172.21.15.67:0/4209169620 >> 172.21.15.67:0/4209169620 conn(0x7fced80ac230 msgr2=0x7fced80ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:06.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 -- 172.21.15.67:0/4209169620 shutdown_connections 2024-02-26T10:39:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.961+0000 7fcedca00640 1 -- 172.21.15.67:0/4209169620 wait complete. 2024-02-26T10:39:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.963+0000 7fcedca00640 1 Processor -- start 2024-02-26T10:39:06.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.963+0000 7fcedca00640 1 -- start start 2024-02-26T10:39:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fcedca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced80c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fcedca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced8155880 0x7fced8154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fcedca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced81547c0 0x7fced814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fcedca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced80c5670 con 0x7fced80c5290 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fcedca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8155cf0 con 0x7fced81547c0 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fcedca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced80c57f0 con 0x7fced8155880 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fced7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced80c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fced7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced80c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47682/0 (socket says 172.21.15.67:47682) 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.964+0000 7fced7400640 1 -- 172.21.15.67:0/1726491860 learned_addr learned my addr 172.21.15.67:0/1726491860 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced6a00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced8155880 0x7fced8154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7e00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced81547c0 0x7fced814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7400640 1 -- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced8155880 msgr2=0x7fced8154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7400640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced8155880 0x7fced8154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7400640 1 -- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced81547c0 msgr2=0x7fced814ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7400640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced81547c0 0x7fced814ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7400640 1 -- 172.21.15.67:0/1726491860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced81ee830 con 0x7fced80c5290 2024-02-26T10:39:06.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fced7400640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced80c9530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcec8036040 tx=0x7fcec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:06.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec8043020 con 0x7fced80c5290 2024-02-26T10:39:06.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec803e450 con 0x7fced80c5290 2024-02-26T10:39:06.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec803e840 con 0x7fced80c5290 2024-02-26T10:39:06.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.965+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced81edcb0 con 0x7fced80c5290 2024-02-26T10:39:06.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.966+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced81ee270 con 0x7fced80c5290 2024-02-26T10:39:06.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.967+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced8074a30 con 0x7fced80c5290 2024-02-26T10:39:06.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.968+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcec8046020 con 0x7fced80c5290 2024-02-26T10:39:06.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.968+0000 7fcecf400640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb406b4d0 0x7fceb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:06.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.968+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcec80acbd0 con 0x7fced80c5290 2024-02-26T10:39:06.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.969+0000 7fced6a00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb406b4d0 0x7fceb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:06.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.970+0000 7fced6a00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb406b4d0 0x7fceb406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fced80af5a0 tx=0x7fcec0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:06.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:06.973+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcec804d030 con 0x7fced80c5290 2024-02-26T10:39:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.172+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fced81526f0 con 0x7fced80c5290 2024-02-26T10:39:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.173+0000 7fcecf400640 1 -- 172.21.15.67:0/1726491860 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcec807bad0 con 0x7fced80c5290 2024-02-26T10:39:07.175 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb406b4d0 msgr2=0x7fceb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb406b4d0 0x7fceb406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fced80af5a0 tx=0x7fcec0008040 comp rx=0 tx=0).stop 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 msgr2=0x7fced80c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced80c9530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcec8036040 tx=0x7fcec8004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 shutdown_connections 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fceb406b4d0 0x7fceb406d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fced81547c0 0x7fced814ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fced8155880 0x7fced8154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 --2- 172.21.15.67:0/1726491860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fced80c5290 0x7fced80c9530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 >> 172.21.15.67:0/1726491860 conn(0x7fced80ac230 msgr2=0x7fced80ac610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 shutdown_connections 2024-02-26T10:39:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.176+0000 7fcedca00640 1 -- 172.21.15.67:0/1726491860 wait complete. 2024-02-26T10:39:07.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.416+0000 7efe56400640 1 Processor -- start 2024-02-26T10:39:07.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.416+0000 7efe56400640 1 -- start start 2024-02-26T10:39:07.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.417+0000 7efe56400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe500a9430 0x7efe500ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.417+0000 7efe56400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe50150780 con 0x7efe5006e2d0 2024-02-26T10:39:07.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.417+0000 7efe56400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe500a98c0 con 0x7efe501503a0 2024-02-26T10:39:07.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.417+0000 7efe56400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe500a9ba0 con 0x7efe500a9430 2024-02-26T10:39:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.417+0000 7efe4fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe500a9430 0x7efe500ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.417+0000 7efe55a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efe5006e2d0 0x7efe500c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50670/0 (socket says 172.21.15.67:50670) 2024-02-26T10:39:07.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.418+0000 7efe55a00640 1 -- 172.21.15.67:0/77771034 learned_addr learned my addr 172.21.15.67:0/77771034 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.418+0000 7efe4fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe500a9430 0x7efe500ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47688/0 (socket says 172.21.15.67:47688) 2024-02-26T10:39:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.418+0000 7efe4fe00640 1 -- 172.21.15.67:0/77771034 >> v1:172.21.15.82:6789/0 conn(0x7efe501503a0 legacy=0x7efe501535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.418+0000 7efe4fe00640 1 -- 172.21.15.67:0/77771034 >> v1:172.21.15.67:6789/0 conn(0x7efe5006e2d0 legacy=0x7efe500c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.418+0000 7efe4fe00640 1 -- 172.21.15.67:0/77771034 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe500a9e80 con 0x7efe500a9430 2024-02-26T10:39:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.419+0000 7efe4fe00640 1 --2- 172.21.15.67:0/77771034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe500a9430 0x7efe500ab480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097a0 tx=0x7efe3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce6a4a6e0ef79c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.419+0000 7efe4f400640 1 -- 172.21.15.67:0/77771034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c035020 con 0x7efe500a9430 2024-02-26T10:39:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.419+0000 7efe4f400640 1 -- 172.21.15.67:0/77771034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c031830 con 0x7efe500a9430 2024-02-26T10:39:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.419+0000 7efe4f400640 1 -- 172.21.15.67:0/77771034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c031c00 con 0x7efe500a9430 2024-02-26T10:39:07.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 -- 172.21.15.67:0/77771034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe500a9430 msgr2=0x7efe500ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 --2- 172.21.15.67:0/77771034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe500a9430 0x7efe500ab480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0097a0 tx=0x7efe3c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 -- 172.21.15.67:0/77771034 shutdown_connections 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 --2- 172.21.15.67:0/77771034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe500a9430 0x7efe500ab480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 -- 172.21.15.67:0/77771034 >> 172.21.15.67:0/77771034 conn(0x7efe500c6380 msgr2=0x7efe500c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 -- 172.21.15.67:0/77771034 shutdown_connections 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.420+0000 7efe56400640 1 -- 172.21.15.67:0/77771034 wait complete. 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.422+0000 7efe56400640 1 Processor -- start 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.422+0000 7efe56400640 1 -- start start 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe56400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe5006e2d0 0x7efe50156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe56400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 0x7efe500c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe56400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe501ec6d0 0x7efe50149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe56400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe500c0f80 con 0x7efe501503a0 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe56400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5006ddb0 con 0x7efe5006e2d0 2024-02-26T10:39:07.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe56400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe500c3520 con 0x7efe501ec6d0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe4fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 0x7efe500c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe4fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 0x7efe500c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47698/0 (socket says 172.21.15.67:47698) 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.423+0000 7efe4fe00640 1 -- 172.21.15.67:0/3536814650 learned_addr learned my addr 172.21.15.67:0/3536814650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4fe00640 1 -- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe501ec6d0 msgr2=0x7efe50149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe55000640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe5006e2d0 0x7efe50156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe55a00640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe501ec6d0 0x7efe50149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4fe00640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe501ec6d0 0x7efe50149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4fe00640 1 -- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe5006e2d0 msgr2=0x7efe50156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4fe00640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe5006e2d0 0x7efe50156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4fe00640 1 -- 172.21.15.67:0/3536814650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe500c5310 con 0x7efe501503a0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe55000640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe5006e2d0 0x7efe50156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4fe00640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 0x7efe500c4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efe3c02bee0 tx=0x7efe3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c041070 con 0x7efe501503a0 2024-02-26T10:39:07.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c03aa30 con 0x7efe501503a0 2024-02-26T10:39:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c035040 con 0x7efe501503a0 2024-02-26T10:39:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.424+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe501ecbb0 con 0x7efe501503a0 2024-02-26T10:39:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.425+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe500c42f0 con 0x7efe501503a0 2024-02-26T10:39:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.426+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe500678e0 con 0x7efe501503a0 2024-02-26T10:39:07.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.427+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe3c038020 con 0x7efe501503a0 2024-02-26T10:39:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.428+0000 7efe4cc00640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efe3006b4d0 0x7efe3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.428+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe3c0abc90 con 0x7efe501503a0 2024-02-26T10:39:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.428+0000 7efe55000640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efe3006b4d0 0x7efe3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.430+0000 7efe55000640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efe3006b4d0 0x7efe3006d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe4400a830 tx=0x7efe44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.432+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe3c07aba0 con 0x7efe501503a0 2024-02-26T10:39:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.632+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe50074a30 con 0x7efe501503a0 2024-02-26T10:39:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.633+0000 7efe4cc00640 1 -- 172.21.15.67:0/3536814650 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe3c07a4a0 con 0x7efe501503a0 2024-02-26T10:39:07.635 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efe3006b4d0 msgr2=0x7efe3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efe3006b4d0 0x7efe3006d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe4400a830 tx=0x7efe44008040 comp rx=0 tx=0).stop 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 msgr2=0x7efe500c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 0x7efe500c4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efe3c02bee0 tx=0x7efe3c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 shutdown_connections 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe501ec6d0 0x7efe50149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe501503a0 0x7efe500c4dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efe3006b4d0 0x7efe3006d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 --2- 172.21.15.67:0/3536814650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe5006e2d0 0x7efe50156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.635+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 >> 172.21.15.67:0/3536814650 conn(0x7efe500c6380 msgr2=0x7efe5014ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:07.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.636+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 shutdown_connections 2024-02-26T10:39:07.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.636+0000 7efe56400640 1 -- 172.21.15.67:0/3536814650 wait complete. 2024-02-26T10:39:07.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:07.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.878+0000 7fd212c00640 1 Processor -- start 2024-02-26T10:39:07.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.879+0000 7fd212c00640 1 -- start start 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd212c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd20c0c2950 0x7fd20c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd212c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd20c153920 0x7fd20c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd212c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20c0a6eb0 con 0x7fd20c1503d0 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd212c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20c153d50 con 0x7fd20c153920 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd212c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20c0a7190 con 0x7fd20c0c2950 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd210c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd20c153920 0x7fd20c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd210c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd20c153920 0x7fd20c15d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47708/0 (socket says 172.21.15.67:47708) 2024-02-26T10:39:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.880+0000 7fd210c00640 1 -- 172.21.15.67:0/1499998709 learned_addr learned my addr 172.21.15.67:0/1499998709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.881+0000 7fd203e00640 1 --2- 172.21.15.67:0/1499998709 >> v2:172.21.15.82:3300/0 conn(0x7fd20c0c2950 0x7fd20c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.881+0000 7fd210c00640 1 -- 172.21.15.67:0/1499998709 >> v2:172.21.15.82:3300/0 conn(0x7fd20c0c2950 msgr2=0x7fd20c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.881+0000 7fd210c00640 1 --2- 172.21.15.67:0/1499998709 >> v2:172.21.15.82:3300/0 conn(0x7fd20c0c2950 0x7fd20c1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.881+0000 7fd210c00640 1 -- 172.21.15.67:0/1499998709 >> v1:172.21.15.82:6789/0 conn(0x7fd20c1503d0 legacy=0x7fd20c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.881+0000 7fd210c00640 1 -- 172.21.15.67:0/1499998709 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c0a7470 con 0x7fd20c153920 2024-02-26T10:39:07.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.882+0000 7fd210c00640 1 --2- 172.21.15.67:0/1499998709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c153920 0x7fd20c15d650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0734b0 tx=0x7fd20402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f883b53f3bdd656 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:07.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.883+0000 7fd20b400640 1 -- 172.21.15.67:0/1499998709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20402c960 con 0x7fd20c153920 2024-02-26T10:39:07.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.883+0000 7fd20b400640 1 -- 172.21.15.67:0/1499998709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20402cb00 con 0x7fd20c153920 2024-02-26T10:39:07.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.883+0000 7fd20b400640 1 -- 172.21.15.67:0/1499998709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040343f0 con 0x7fd20c153920 2024-02-26T10:39:07.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.883+0000 7fd212c00640 1 -- 172.21.15.67:0/1499998709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c153920 msgr2=0x7fd20c15d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.883+0000 7fd212c00640 1 --2- 172.21.15.67:0/1499998709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c153920 0x7fd20c15d650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0734b0 tx=0x7fd20402bdc0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.884+0000 7fd212c00640 1 -- 172.21.15.67:0/1499998709 shutdown_connections 2024-02-26T10:39:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.884+0000 7fd212c00640 1 --2- 172.21.15.67:0/1499998709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c153920 0x7fd20c15d650 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.884+0000 7fd212c00640 1 --2- 172.21.15.67:0/1499998709 >> v2:172.21.15.82:3300/0 conn(0x7fd20c0c2950 0x7fd20c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.884+0000 7fd212c00640 1 -- 172.21.15.67:0/1499998709 >> 172.21.15.67:0/1499998709 conn(0x7fd20c0c6da0 msgr2=0x7fd20c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.884+0000 7fd212c00640 1 -- 172.21.15.67:0/1499998709 shutdown_connections 2024-02-26T10:39:07.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.884+0000 7fd212c00640 1 -- 172.21.15.67:0/1499998709 wait complete. 2024-02-26T10:39:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.885+0000 7fd212c00640 1 Processor -- start 2024-02-26T10:39:07.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.886+0000 7fd212c00640 1 -- start start 2024-02-26T10:39:07.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.886+0000 7fd212c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd20c0c2950 0x7fd20c1ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.886+0000 7fd212c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 0x7fd20c1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd212c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd20c153920 0x7fd20c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd212c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c06ef00 con 0x7fd20c1503d0 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd212c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c152ec0 con 0x7fd20c0c2950 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd212c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1507b0 con 0x7fd20c153920 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 0x7fd20c1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 0x7fd20c1549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47712/0 (socket says 172.21.15.67:47712) 2024-02-26T10:39:07.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 -- 172.21.15.67:0/2732559036 learned_addr learned my addr 172.21.15.67:0/2732559036 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:07.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd210c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd20c153920 0x7fd20c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd20be00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd20c0c2950 0x7fd20c1ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 -- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd20c153920 msgr2=0x7fd20c154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd20c153920 0x7fd20c154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 -- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd20c0c2950 msgr2=0x7fd20c1ee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:07.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd20c0c2950 0x7fd20c1ee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.887+0000 7fd203e00640 1 -- 172.21.15.67:0/2732559036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c14cc00 con 0x7fd20c1503d0 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.888+0000 7fd203e00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 0x7fd20c1549c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400d800 tx=0x7fd1f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.888+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400b6c0 con 0x7fd20c1503d0 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.888+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4019070 con 0x7fd20c1503d0 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.888+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20c14cee0 con 0x7fd20c1503d0 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.888+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400ba80 con 0x7fd20c1503d0 2024-02-26T10:39:07.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.888+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20c14f520 con 0x7fd20c1503d0 2024-02-26T10:39:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.889+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20c074a30 con 0x7fd20c1503d0 2024-02-26T10:39:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.890+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1f400bc20 con 0x7fd20c1503d0 2024-02-26T10:39:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.891+0000 7fd208c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd1e806b4d0 0x7fd1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.891+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1f408a6f0 con 0x7fd20c1503d0 2024-02-26T10:39:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.892+0000 7fd20be00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd1e806b4d0 0x7fd1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.893+0000 7fd20be00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd1e806b4d0 0x7fd1e806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc004ea0 tx=0x7fd1fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:07.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:07.895+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1f4059560 con 0x7fd20c1503d0 2024-02-26T10:39:08.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.098+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd20c06ec50 con 0x7fd20c1503d0 2024-02-26T10:39:08.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.099+0000 7fd208c00640 1 -- 172.21.15.67:0/2732559036 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1f4058e60 con 0x7fd20c1503d0 2024-02-26T10:39:08.101 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd1e806b4d0 msgr2=0x7fd1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd1e806b4d0 0x7fd1e806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc004ea0 tx=0x7fd1fc009040 comp rx=0 tx=0).stop 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 msgr2=0x7fd20c1549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 0x7fd20c1549c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400d800 tx=0x7fd1f400dcc0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 shutdown_connections 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd20c153920 0x7fd20c154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd20c1503d0 0x7fd20c1549c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd1e806b4d0 0x7fd1e806d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 --2- 172.21.15.67:0/2732559036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd20c0c2950 0x7fd20c1ee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 >> 172.21.15.67:0/2732559036 conn(0x7fd20c0c6da0 msgr2=0x7fd20c151180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 shutdown_connections 2024-02-26T10:39:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.102+0000 7fd212c00640 1 -- 172.21.15.67:0/2732559036 wait complete. 2024-02-26T10:39:08.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.348+0000 7f80bba00640 1 Processor -- start 2024-02-26T10:39:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.349+0000 7f80bba00640 1 -- start start 2024-02-26T10:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.350+0000 7f80bba00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80b406e410 0x7f80b40aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.350+0000 7f80bba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80b4156230 0x7f80b40ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.350+0000 7f80bba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b4152e50 con 0x7f80b4150300 2024-02-26T10:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.350+0000 7f80bba00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b41506e0 con 0x7f80b406e410 2024-02-26T10:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.350+0000 7f80bba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b4153a60 con 0x7f80b4156230 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80b406e410 0x7f80b40aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b9a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80b4156230 0x7f80b40ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b9a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80b4156230 0x7f80b40ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:53652/0 (socket says 172.21.15.67:53652) 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b9a00640 1 -- 172.21.15.67:0/2178483148 learned_addr learned my addr 172.21.15.67:0/2178483148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b3e00640 1 -- 172.21.15.67:0/2178483148 >> v2:172.21.15.82:3300/0 conn(0x7f80b4156230 msgr2=0x7f80b40ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b3e00640 1 --2- 172.21.15.67:0/2178483148 >> v2:172.21.15.82:3300/0 conn(0x7f80b4156230 0x7f80b40ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b3e00640 1 -- 172.21.15.67:0/2178483148 >> v1:172.21.15.82:6789/0 conn(0x7f80b4150300 legacy=0x7f80b40c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.351+0000 7f80b3e00640 1 -- 172.21.15.67:0/2178483148 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4153130 con 0x7f80b406e410 2024-02-26T10:39:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.352+0000 7f80b3e00640 1 --2- 172.21.15.67:0/2178483148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b406e410 0x7f80b40aaba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80b406e380 tx=0x7f80a002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf27cb829ea7ae23 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.353+0000 7f80b3400640 1 -- 172.21.15.67:0/2178483148 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0038070 con 0x7f80b406e410 2024-02-26T10:39:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.353+0000 7f80b3400640 1 -- 172.21.15.67:0/2178483148 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a0031710 con 0x7f80b406e410 2024-02-26T10:39:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.353+0000 7f80b3400640 1 -- 172.21.15.67:0/2178483148 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0031ae0 con 0x7f80b406e410 2024-02-26T10:39:08.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.353+0000 7f80bba00640 1 -- 172.21.15.67:0/2178483148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b406e410 msgr2=0x7f80b40aaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.353+0000 7f80bba00640 1 --2- 172.21.15.67:0/2178483148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b406e410 0x7f80b40aaba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80b406e380 tx=0x7f80a002b7b0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.354+0000 7f80bba00640 1 -- 172.21.15.67:0/2178483148 shutdown_connections 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.354+0000 7f80bba00640 1 --2- 172.21.15.67:0/2178483148 >> v2:172.21.15.82:3300/0 conn(0x7f80b4156230 0x7f80b40ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.354+0000 7f80bba00640 1 --2- 172.21.15.67:0/2178483148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b406e410 0x7f80b40aaba0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.354+0000 7f80bba00640 1 -- 172.21.15.67:0/2178483148 >> 172.21.15.67:0/2178483148 conn(0x7f80b40a84a0 msgr2=0x7f80b40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.354+0000 7f80bba00640 1 -- 172.21.15.67:0/2178483148 shutdown_connections 2024-02-26T10:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.354+0000 7f80bba00640 1 -- 172.21.15.67:0/2178483148 wait complete. 2024-02-26T10:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.356+0000 7f80bba00640 1 Processor -- start 2024-02-26T10:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.356+0000 7f80bba00640 1 -- start start 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80bba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80b406e410 0x7f80b41deca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80bba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80b4150300 0x7f80b4155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80bba00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 0x7f80b40c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80bba00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b406dd80 con 0x7f80b4156230 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80bba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41536f0 con 0x7f80b406e410 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80bba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c3700 con 0x7f80b4150300 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80b9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 0x7f80b40c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80b9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 0x7f80b40c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47734/0 (socket says 172.21.15.67:47734) 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80b9a00640 1 -- 172.21.15.67:0/2925980286 learned_addr learned my addr 172.21.15.67:0/2925980286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80b9000640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80b406e410 0x7f80b41deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.357+0000 7f80b3e00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80b4150300 0x7f80b4155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b9a00640 1 -- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80b4150300 msgr2=0x7f80b4155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b9a00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80b4150300 0x7f80b4155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b9a00640 1 -- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80b406e410 msgr2=0x7f80b41deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b9a00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80b406e410 0x7f80b41deca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b9a00640 1 -- 172.21.15.67:0/2925980286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4154260 con 0x7f80b4156230 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b3e00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80b4150300 0x7f80b4155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b9a00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 0x7f80b40c41f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009cf0 tx=0x7f80ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00cd20 con 0x7f80b4156230 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac003990 con 0x7f80b4156230 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac003d80 con 0x7f80b4156230 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b41eb6c0 con 0x7f80b4156230 2024-02-26T10:39:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.358+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b4152a80 con 0x7f80b4156230 2024-02-26T10:39:08.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.360+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b4074a30 con 0x7f80b4156230 2024-02-26T10:39:08.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.361+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80ac005d10 con 0x7f80b4156230 2024-02-26T10:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.361+0000 7f80b0c00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f809406b4d0 0x7f809406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.362+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80ac08a680 con 0x7f80b4156230 2024-02-26T10:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.362+0000 7f80b9000640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f809406b4d0 0x7f809406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.363+0000 7f80b9000640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f809406b4d0 0x7f809406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f80a4001780 tx=0x7f80a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.366+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80ac022020 con 0x7f80b4156230 2024-02-26T10:39:08.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.572+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80b406e0d0 con 0x7f80b4156230 2024-02-26T10:39:08.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.574+0000 7f80b0c00640 1 -- 172.21.15.67:0/2925980286 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f80ac010040 con 0x7f80b4156230 2024-02-26T10:39:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:08.577 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.577+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f809406b4d0 msgr2=0x7f809406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.577+0000 7f80bba00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f809406b4d0 0x7f809406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f80a4001780 tx=0x7f80a4009040 comp rx=0 tx=0).stop 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.577+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 msgr2=0x7f80b40c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.577+0000 7f80bba00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 0x7f80b40c41f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009cf0 tx=0x7f80ac00c490 comp rx=0 tx=0).stop 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 shutdown_connections 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80b4156230 0x7f80b40c41f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80b4150300 0x7f80b4155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f809406b4d0 0x7f809406d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 --2- 172.21.15.67:0/2925980286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80b406e410 0x7f80b41deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 >> 172.21.15.67:0/2925980286 conn(0x7f80b40a84a0 msgr2=0x7f80b40c2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 shutdown_connections 2024-02-26T10:39:08.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.578+0000 7f80bba00640 1 -- 172.21.15.67:0/2925980286 wait complete. 2024-02-26T10:39:08.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:08.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:08.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.822+0000 7f370f000640 1 Processor -- start 2024-02-26T10:39:08.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.822+0000 7f370f000640 1 -- start start 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 0x7f37080c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370f000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37080c0b40 0x7f3708154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37080c3b00 con 0x7f3708071ff0 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370f000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080c38d0 con 0x7f37080c0b40 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3708155b30 con 0x7f37080c4630 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370d000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3708071ff0 0x7f3708155420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50682/0 (socket says 172.21.15.67:50682) 2024-02-26T10:39:08.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.823+0000 7f370d000640 1 -- 172.21.15.67:0/4154761344 learned_addr learned my addr 172.21.15.67:0/4154761344 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707e00640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 0x7f37080c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f36ff400640 1 --2- 172.21.15.67:0/4154761344 >> v2:172.21.15.82:3300/0 conn(0x7f37080c0b40 0x7f3708154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951379302 0 0) 0x7f37080c3b00 con 0x7f3708071ff0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ec003520 con 0x7f3708071ff0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932200701 0 0) 0x7f36ec003520 con 0x7f3708071ff0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 msgr2=0x7f37080c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 0x7f37080c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 >> v2:172.21.15.82:3300/0 conn(0x7f37080c0b40 msgr2=0x7f3708154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 --2- 172.21.15.67:0/4154761344 >> v2:172.21.15.82:3300/0 conn(0x7f37080c0b40 0x7f3708154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080c3de0 con 0x7f3708071ff0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f37000019f0 con 0x7f3708071ff0 2024-02-26T10:39:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 >> v1:172.21.15.67:6789/0 conn(0x7f3708071ff0 legacy=0x7f3708155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36ec0039b0 0x7f36ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 0x7f36ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36ec004d40 0x7f36ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708155b30 con 0x7f36ec0039b0 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec00b9c0 con 0x7f36ec004d40 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.824+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec00bb40 con 0x7f37080c4630 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f3707e00640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36ec0039b0 0x7f36ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f3707e00640 1 -- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 msgr2=0x7f36ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f36ff400640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 0x7f36ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f3707e00640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37080c4630 0x7f36ec004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f3707e00640 1 -- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36ec004d40 msgr2=0x7f36ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f3707e00640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36ec004d40 0x7f36ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.825+0000 7f3707e00640 1 -- 172.21.15.67:0/4154761344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec003520 con 0x7f36ec0039b0 2024-02-26T10:39:08.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.826+0000 7f3707e00640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36ec0039b0 0x7f36ec003dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009ac0 tx=0x7f36ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebd9c8249e8632f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8039070 con 0x7f36ec0039b0 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f802c6b0 con 0x7f36ec0039b0 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f3707400640 1 -- 172.21.15.67:0/4154761344 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8009690 con 0x7f36ec0039b0 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370f000640 1 -- 172.21.15.67:0/4154761344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36ec0039b0 msgr2=0x7f36ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370f000640 1 --2- 172.21.15.67:0/4154761344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f36ec0039b0 0x7f36ec003dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009ac0 tx=0x7f36ec005830 comp rx=0 tx=0).stop 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370d000640 1 -- 172.21.15.67:0/4154761344 reap_dead start 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370f000640 1 -- 172.21.15.67:0/4154761344 shutdown_connections 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370f000640 1 -- 172.21.15.67:0/4154761344 >> 172.21.15.67:0/4154761344 conn(0x7f37080c7b00 msgr2=0x7f37080c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370f000640 1 -- 172.21.15.67:0/4154761344 shutdown_connections 2024-02-26T10:39:08.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.827+0000 7f370f000640 1 -- 172.21.15.67:0/4154761344 wait complete. 2024-02-26T10:39:08.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.829+0000 7f370f000640 1 Processor -- start 2024-02-26T10:39:08.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.829+0000 7f370f000640 1 -- start start 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f370f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36ec0039b0 0x7f37081ede90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f370f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 0x7f37080a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f370f000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37081ee3d0 0x7f37080a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f370f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708072490 con 0x7f37081565c0 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f370f000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37080c2340 con 0x7f37081ee3d0 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f370f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3708152770 con 0x7f36ec0039b0 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f36ff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 0x7f37080a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f36ff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 0x7f37080a8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47754/0 (socket says 172.21.15.67:47754) 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.830+0000 7f36ff400640 1 -- 172.21.15.67:0/1952761460 learned_addr learned my addr 172.21.15.67:0/1952761460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f3707e00640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36ec0039b0 0x7f37081ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f370d000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37081ee3d0 0x7f37080a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f36ff400640 1 -- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36ec0039b0 msgr2=0x7f37081ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f36ff400640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36ec0039b0 0x7f37081ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f36ff400640 1 -- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37081ee3d0 msgr2=0x7f37080a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f36ff400640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37081ee3d0 0x7f37080a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f36ff400640 1 -- 172.21.15.67:0/1952761460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080a97d0 con 0x7f37081565c0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f370d000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37081ee3d0 0x7f37080a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f36ff400640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 0x7f37080a8470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36f0010940 tx=0x7f36f0010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000f040 con 0x7f37081565c0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f000ebc0 con 0x7f37081565c0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f001c400 con 0x7f37081565c0 2024-02-26T10:39:08.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.831+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37080a9ab0 con 0x7f37081565c0 2024-02-26T10:39:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.832+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37080a70a0 con 0x7f37081565c0 2024-02-26T10:39:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.833+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3708074a30 con 0x7f37081565c0 2024-02-26T10:39:08.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.834+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36f001c5a0 con 0x7f37081565c0 2024-02-26T10:39:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.835+0000 7f3704c00640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f36e406b550 0x7f36e406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.835+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36f008a750 con 0x7f37081565c0 2024-02-26T10:39:08.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.835+0000 7f3707e00640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f36e406b550 0x7f36e406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:08.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.836+0000 7f3707e00640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f36e406b550 0x7f36e406da10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f36f8036000 tx=0x7f36f8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:08.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:08.839+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36f0018030 con 0x7f37081565c0 2024-02-26T10:39:09.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.040+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37080a9c80 con 0x7f37081565c0 2024-02-26T10:39:09.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.041+0000 7f3704c00640 1 -- 172.21.15.67:0/1952761460 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36f0059640 con 0x7f37081565c0 2024-02-26T10:39:09.043 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f36e406b550 msgr2=0x7f36e406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f36e406b550 0x7f36e406da10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f36f8036000 tx=0x7f36f8030000 comp rx=0 tx=0).stop 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 msgr2=0x7f37080a8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 0x7f37080a8470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36f0010940 tx=0x7f36f0010e00 comp rx=0 tx=0).stop 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 shutdown_connections 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f36e406b550 0x7f36e406da10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37081ee3d0 0x7f37080a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37081565c0 0x7f37080a8470 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 --2- 172.21.15.67:0/1952761460 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36ec0039b0 0x7f37081ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 >> 172.21.15.67:0/1952761460 conn(0x7f37080c7b00 msgr2=0x7f370814cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 shutdown_connections 2024-02-26T10:39:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.044+0000 7f370f000640 1 -- 172.21.15.67:0/1952761460 wait complete. 2024-02-26T10:39:09.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:09.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.290+0000 7f963be00640 1 Processor -- start 2024-02-26T10:39:09.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.291+0000 7f963be00640 1 -- start start 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.292+0000 7f963be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f96340a6150 0x7f96340c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.292+0000 7f963be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f963406e860 0x7f96340a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.292+0000 7f963be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9634155170 0x7f9634155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.292+0000 7f963be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9634152b10 con 0x7f9634155170 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.292+0000 7f963be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9634152680 con 0x7f96340a6150 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.292+0000 7f963be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9634151d70 con 0x7f963406e860 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9634155170 0x7f9634155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9634155170 0x7f9634155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47758/0 (socket says 172.21.15.67:47758) 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 -- 172.21.15.67:0/3659776652 learned_addr learned my addr 172.21.15.67:0/3659776652 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639400640 1 --2- 172.21.15.67:0/3659776652 >> v2:172.21.15.82:3300/0 conn(0x7f96340a6150 0x7f96340c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 -- 172.21.15.67:0/3659776652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f963406e860 msgr2=0x7f96340a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 --2- 172.21.15.67:0/3659776652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f963406e860 0x7f96340a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 -- 172.21.15.67:0/3659776652 >> v2:172.21.15.82:3300/0 conn(0x7f96340a6150 msgr2=0x7f96340c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 --2- 172.21.15.67:0/3659776652 >> v2:172.21.15.82:3300/0 conn(0x7f96340a6150 0x7f96340c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639e00640 1 -- 172.21.15.67:0/3659776652 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634152030 con 0x7f9634155170 2024-02-26T10:39:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.293+0000 7f9639400640 1 --2- 172.21.15.67:0/3659776652 >> v2:172.21.15.82:3300/0 conn(0x7f96340a6150 0x7f96340c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.294+0000 7f9639e00640 1 --2- 172.21.15.67:0/3659776652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9634155170 0x7f9634155600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f963406ef20 tx=0x7f962c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cf9c63150a9923 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:09.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.295+0000 7f9633e00640 1 -- 172.21.15.67:0/3659776652 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c0317d0 con 0x7f9634155170 2024-02-26T10:39:09.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.295+0000 7f9633e00640 1 -- 172.21.15.67:0/3659776652 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c037070 con 0x7f9634155170 2024-02-26T10:39:09.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.295+0000 7f9633e00640 1 -- 172.21.15.67:0/3659776652 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c031b90 con 0x7f9634155170 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.295+0000 7f963be00640 1 -- 172.21.15.67:0/3659776652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9634155170 msgr2=0x7f9634155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.295+0000 7f963be00640 1 --2- 172.21.15.67:0/3659776652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9634155170 0x7f9634155600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f963406ef20 tx=0x7f962c02b870 comp rx=0 tx=0).stop 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 -- 172.21.15.67:0/3659776652 shutdown_connections 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 --2- 172.21.15.67:0/3659776652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9634155170 0x7f9634155600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 --2- 172.21.15.67:0/3659776652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f963406e860 0x7f96340a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 --2- 172.21.15.67:0/3659776652 >> v2:172.21.15.82:3300/0 conn(0x7f96340a6150 0x7f96340c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 -- 172.21.15.67:0/3659776652 >> 172.21.15.67:0/3659776652 conn(0x7f96340c6da0 msgr2=0x7f96340c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 -- 172.21.15.67:0/3659776652 shutdown_connections 2024-02-26T10:39:09.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.296+0000 7f963be00640 1 -- 172.21.15.67:0/3659776652 wait complete. 2024-02-26T10:39:09.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.297+0000 7f963be00640 1 Processor -- start 2024-02-26T10:39:09.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.297+0000 7f963be00640 1 -- start start 2024-02-26T10:39:09.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.298+0000 7f963be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 0x7f9634153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.298+0000 7f963be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96340a6150 0x7f9634153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.298+0000 7f963be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9634155170 0x7f96341eada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.298+0000 7f963be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9634155b40 con 0x7f963406e860 2024-02-26T10:39:09.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.298+0000 7f963be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96340c2220 con 0x7f9634155170 2024-02-26T10:39:09.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.298+0000 7f963be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9634155cc0 con 0x7f96340a6150 2024-02-26T10:39:09.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 0x7f9634153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 0x7f9634153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47772/0 (socket says 172.21.15.67:47772) 2024-02-26T10:39:09.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 -- 172.21.15.67:0/491894293 learned_addr learned my addr 172.21.15.67:0/491894293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 -- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96340a6150 msgr2=0x7f9634153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96340a6150 0x7f9634153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639e00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9634155170 0x7f96341eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 -- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9634155170 msgr2=0x7f96341eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9634155170 0x7f96341eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 -- 172.21.15.67:0/491894293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9634152cb0 con 0x7f963406e860 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639e00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9634155170 0x7f96341eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:09.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.299+0000 7f9639400640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 0x7f9634153270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f962800cf60 tx=0x7f962800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.300+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962801a070 con 0x7f963406e860 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.300+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962800d840 con 0x7f963406e860 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.300+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962800dc10 con 0x7f963406e860 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.300+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9634152f90 con 0x7f963406e860 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.300+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9634150860 con 0x7f963406e860 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.301+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9634074a30 con 0x7f963406e860 2024-02-26T10:39:09.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.302+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f962801d070 con 0x7f963406e860 2024-02-26T10:39:09.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.303+0000 7f9631600640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f961406b4d0 0x7f961406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.303+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f962808aa30 con 0x7f963406e860 2024-02-26T10:39:09.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.303+0000 7f9638a00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f961406b4d0 0x7f961406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.304+0000 7f9638a00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f961406b4d0 0x7f961406d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96340ac650 tx=0x7f9620006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.307+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9628059250 con 0x7f963406e860 2024-02-26T10:39:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.505+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9634073280 con 0x7f963406e860 2024-02-26T10:39:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.506+0000 7f9631600640 1 -- 172.21.15.67:0/491894293 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f962805d2f0 con 0x7f963406e860 2024-02-26T10:39:09.508 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:09.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f961406b4d0 msgr2=0x7f961406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f961406b4d0 0x7f961406d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f96340ac650 tx=0x7f9620006c90 comp rx=0 tx=0).stop 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 msgr2=0x7f9634153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 0x7f9634153270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f962800cf60 tx=0x7f962800cf90 comp rx=0 tx=0).stop 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 shutdown_connections 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f961406b4d0 0x7f961406d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9634155170 0x7f96341eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96340a6150 0x7f9634153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 --2- 172.21.15.67:0/491894293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f963406e860 0x7f9634153270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 >> 172.21.15.67:0/491894293 conn(0x7f96340c6da0 msgr2=0x7f9634072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 shutdown_connections 2024-02-26T10:39:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.509+0000 7f963be00640 1 -- 172.21.15.67:0/491894293 wait complete. 2024-02-26T10:39:09.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:09.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.754+0000 7fa4c9800640 1 Processor -- start 2024-02-26T10:39:09.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.754+0000 7fa4c9800640 1 -- start start 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c9800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa4c40c25a0 0x7fa4c40c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c9800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40c1be0 con 0x7fa4c414f260 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c9800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40c1750 con 0x7fa4c40c25a0 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c3660 con 0x7fa4c40c1eb0 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c8e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa4c414f260 0x7fa4c4151980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39612/0 (socket says 172.21.15.67:39612) 2024-02-26T10:39:09.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.756+0000 7fa4c8e00640 1 -- 172.21.15.67:0/1016935253 learned_addr learned my addr 172.21.15.67:0/1016935253 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c3400640 1 --2- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/1016935253 >> v2:172.21.15.82:3300/0 conn(0x7fa4c40c25a0 0x7fa4c40c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c2a00640 1 -- 172.21.15.67:0/1016935253 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113467818 0 0) 0x7fa4c40c1be0 con 0x7fa4c414f260 2024-02-26T10:39:09.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c2a00640 1 -- 172.21.15.67:0/1016935253 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4a8003520 con 0x7fa4c414f260 2024-02-26T10:39:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c2a00640 1 -- 172.21.15.67:0/1016935253 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954997695 0 0) 0x7fa4a8003520 con 0x7fa4c414f260 2024-02-26T10:39:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c3e00640 1 -- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 msgr2=0x7fa4c414ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c414ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c3e00640 1 -- 172.21.15.67:0/1016935253 >> v1:172.21.15.67:6789/0 conn(0x7fa4c414f260 legacy=0x7fa4c4151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.757+0000 7fa4c3e00640 1 -- 172.21.15.67:0/1016935253 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40c0e20 con 0x7fa4c40c25a0 2024-02-26T10:39:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.759+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c40c25a0 0x7fa4c40c2e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40097a0 tx=0x7fa4b402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8315a5a5f8f768 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.759+0000 7fa4c2a00640 1 -- 172.21.15.67:0/1016935253 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4037070 con 0x7fa4c40c25a0 2024-02-26T10:39:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.759+0000 7fa4c2a00640 1 -- 172.21.15.67:0/1016935253 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b4030510 con 0x7fa4c40c25a0 2024-02-26T10:39:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.759+0000 7fa4c2a00640 1 -- 172.21.15.67:0/1016935253 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0019f0 con 0x7fa4c40c25a0 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 -- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c40c25a0 msgr2=0x7fa4c40c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 --2- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c40c25a0 0x7fa4c40c2e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40097a0 tx=0x7fa4b402b710 comp rx=0 tx=0).stop 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 -- 172.21.15.67:0/1016935253 shutdown_connections 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 --2- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c414ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 --2- 172.21.15.67:0/1016935253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c40c25a0 0x7fa4c40c2e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 -- 172.21.15.67:0/1016935253 >> 172.21.15.67:0/1016935253 conn(0x7fa4c40c6380 msgr2=0x7fa4c40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 -- 172.21.15.67:0/1016935253 shutdown_connections 2024-02-26T10:39:09.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.760+0000 7fa4c9800640 1 -- 172.21.15.67:0/1016935253 wait complete. 2024-02-26T10:39:09.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.762+0000 7fa4c9800640 1 Processor -- start 2024-02-26T10:39:09.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.762+0000 7fa4c9800640 1 -- start start 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c41554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 0x7fa4c4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c414f260 0x7fa4c414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c3350 con 0x7fa4c40c25a0 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c13e0 con 0x7fa4c414f260 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c2980 con 0x7fa4c40c1eb0 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 0x7fa4c4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 0x7fa4c4155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43372/0 (socket says 172.21.15.67:43372) 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 -- 172.21.15.67:0/392762064 learned_addr learned my addr 172.21.15.67:0/392762064 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c41554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c8e00640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c414f260 0x7fa4c414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 -- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 msgr2=0x7fa4c41554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c41554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 -- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c414f260 msgr2=0x7fa4c414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c414f260 0x7fa4c414fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3400640 1 -- 172.21.15.67:0/392762064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c414a500 con 0x7fa4c40c25a0 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.763+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c41554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.764+0000 7fa4c3400640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 0x7fa4c4155a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000e9e0 tx=0x7fa4b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.764+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0017070 con 0x7fa4c40c25a0 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.764+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b000c9d0 con 0x7fa4c40c25a0 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.764+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c41eb580 con 0x7fa4c40c25a0 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.764+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000cdc0 con 0x7fa4c40c25a0 2024-02-26T10:39:09.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.764+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c414aaa0 con 0x7fa4c40c25a0 2024-02-26T10:39:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.765+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c40678e0 con 0x7fa4c40c25a0 2024-02-26T10:39:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.766+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4b001a070 con 0x7fa4c40c25a0 2024-02-26T10:39:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.767+0000 7fa4bbe00640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa4a406b4d0 0x7fa4a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.767+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4b008a660 con 0x7fa4c40c25a0 2024-02-26T10:39:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.768+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa4a406b4d0 0x7fa4a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:09.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.769+0000 7fa4c3e00640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa4a406b4d0 0x7fa4a406d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4034000 tx=0x7fa4b4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:09.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.772+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4b0059440 con 0x7fa4c40c25a0 2024-02-26T10:39:09.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.971+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4c4074a30 con 0x7fa4c40c25a0 2024-02-26T10:39:09.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.972+0000 7fa4bbe00640 1 -- 172.21.15.67:0/392762064 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4b0058d40 con 0x7fa4c40c25a0 2024-02-26T10:39:09.974 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa4a406b4d0 msgr2=0x7fa4a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa4a406b4d0 0x7fa4a406d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4034000 tx=0x7fa4b4008030 comp rx=0 tx=0).stop 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 msgr2=0x7fa4c4155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 0x7fa4c4155a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000e9e0 tx=0x7fa4b000eea0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 shutdown_connections 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa4a406b4d0 0x7fa4a406d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c414f260 0x7fa4c414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c40c25a0 0x7fa4c4155a00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 --2- 172.21.15.67:0/392762064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa4c40c1eb0 0x7fa4c41554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 >> 172.21.15.67:0/392762064 conn(0x7fa4c40c6380 msgr2=0x7fa4c414e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.975+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 shutdown_connections 2024-02-26T10:39:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:09.976+0000 7fa4c9800640 1 -- 172.21.15.67:0/392762064 wait complete. 2024-02-26T10:39:09.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:10.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.208+0000 7fb5aea00640 1 Processor -- start 2024-02-26T10:39:10.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.208+0000 7fb5aea00640 1 -- start start 2024-02-26T10:39:10.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.209+0000 7fb5aea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5a80703a0 0x7fb5a81524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.209+0000 7fb5aea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a80c6dd0 con 0x7fb5a81529e0 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.209+0000 7fb5aea00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a80c7040 con 0x7fb5a80c64c0 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.209+0000 7fb5aea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a80c7320 con 0x7fb5a80703a0 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb5a81529e0 0x7fb5a80c3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39628/0 (socket says 172.21.15.67:39628) 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a7400640 1 -- 172.21.15.67:0/883130370 learned_addr learned my addr 172.21.15.67:0/883130370 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a7e00640 1 --2- 172.21.15.67:0/883130370 >> v2:172.21.15.67:3300/0 conn(0x7fb5a80703a0 0x7fb5a81524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a7e00640 1 -- 172.21.15.67:0/883130370 >> v1:172.21.15.82:6789/0 conn(0x7fb5a80c64c0 legacy=0x7fb5a815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:10.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a7e00640 1 -- 172.21.15.67:0/883130370 >> v1:172.21.15.67:6789/0 conn(0x7fb5a81529e0 legacy=0x7fb5a80c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a7e00640 1 -- 172.21.15.67:0/883130370 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80c7600 con 0x7fb5a80703a0 2024-02-26T10:39:10.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.210+0000 7fb5a6a00640 1 -- 172.21.15.67:0/883130370 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138253412 0 0) 0x7fb5a80c6dd0 con 0x7fb5a81529e0 2024-02-26T10:39:10.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.212+0000 7fb5a7e00640 1 --2- 172.21.15.67:0/883130370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80703a0 0x7fb5a81524a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5980097a0 tx=0x7fb59802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30814bb02f1ba5ed server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:10.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.212+0000 7fb5a6a00640 1 -- 172.21.15.67:0/883130370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980315a0 con 0x7fb5a80703a0 2024-02-26T10:39:10.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.212+0000 7fb5a6a00640 1 -- 172.21.15.67:0/883130370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598037070 con 0x7fb5a80703a0 2024-02-26T10:39:10.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.212+0000 7fb5a6a00640 1 -- 172.21.15.67:0/883130370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598031960 con 0x7fb5a80703a0 2024-02-26T10:39:10.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.212+0000 7fb5aea00640 1 -- 172.21.15.67:0/883130370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80703a0 msgr2=0x7fb5a81524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.212+0000 7fb5aea00640 1 --2- 172.21.15.67:0/883130370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80703a0 0x7fb5a81524a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5980097a0 tx=0x7fb59802b840 comp rx=0 tx=0).stop 2024-02-26T10:39:10.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.213+0000 7fb5aea00640 1 -- 172.21.15.67:0/883130370 shutdown_connections 2024-02-26T10:39:10.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.213+0000 7fb5aea00640 1 --2- 172.21.15.67:0/883130370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80703a0 0x7fb5a81524a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.213+0000 7fb5aea00640 1 -- 172.21.15.67:0/883130370 >> 172.21.15.67:0/883130370 conn(0x7fb5a80aa330 msgr2=0x7fb5a80aa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:10.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.213+0000 7fb5aea00640 1 -- 172.21.15.67:0/883130370 shutdown_connections 2024-02-26T10:39:10.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.213+0000 7fb5aea00640 1 -- 172.21.15.67:0/883130370 wait complete. 2024-02-26T10:39:10.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.214+0000 7fb5aea00640 1 Processor -- start 2024-02-26T10:39:10.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.215+0000 7fb5aea00640 1 -- start start 2024-02-26T10:39:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.215+0000 7fb5aea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a80703a0 0x7fb5a81db220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.215+0000 7fb5aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 0x7fb5a80ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.215+0000 7fb5aea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5a81529e0 0x7fb5a81e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5aea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8070f00 con 0x7fb5a80c64c0 2024-02-26T10:39:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5aea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a8070780 con 0x7fb5a80703a0 2024-02-26T10:39:10.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5aea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80c2ef0 con 0x7fb5a81529e0 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 0x7fb5a80ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 0x7fb5a80ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43382/0 (socket says 172.21.15.67:43382) 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 -- 172.21.15.67:0/2165659066 learned_addr learned my addr 172.21.15.67:0/2165659066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7e00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a80703a0 0x7fb5a81db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 -- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5a81529e0 msgr2=0x7fb5a81e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5a81529e0 0x7fb5a81e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 -- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a80703a0 msgr2=0x7fb5a81db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a80703a0 0x7fb5a81db220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7400640 1 -- 172.21.15.67:0/2165659066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a81eea90 con 0x7fb5a80c64c0 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.216+0000 7fb5a7e00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a80703a0 0x7fb5a81db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.217+0000 7fb5a7400640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 0x7fb5a80ada10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb59000c7c0 tx=0x7fb59000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.217+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590018070 con 0x7fb5a80c64c0 2024-02-26T10:39:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.217+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59000a5e0 con 0x7fb5a80c64c0 2024-02-26T10:39:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.217+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a80ad210 con 0x7fb5a80c64c0 2024-02-26T10:39:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.217+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000a9d0 con 0x7fb5a80c64c0 2024-02-26T10:39:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.217+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a80ae330 con 0x7fb5a80c64c0 2024-02-26T10:39:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.218+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a814fe50 con 0x7fb5a80c64c0 2024-02-26T10:39:10.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.219+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb59001b070 con 0x7fb5a80c64c0 2024-02-26T10:39:10.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.220+0000 7fb59fe00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb58406b4d0 0x7fb58406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.221+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb590089890 con 0x7fb5a80c64c0 2024-02-26T10:39:10.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.221+0000 7fb5a7e00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb58406b4d0 0x7fb58406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.222+0000 7fb5a7e00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb58406b4d0 0x7fb58406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb598036000 tx=0x7fb598008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:10.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.225+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5a814fe50 con 0x7fb5a80c64c0 2024-02-26T10:39:10.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.454+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5a8074a30 con 0x7fb5a80c64c0 2024-02-26T10:39:10.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.457+0000 7fb59fe00640 1 -- 172.21.15.67:0/2165659066 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fb590058700 con 0x7fb5a80c64c0 2024-02-26T10:39:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:10.460 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.460+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb58406b4d0 msgr2=0x7fb58406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.460+0000 7fb5aea00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb58406b4d0 0x7fb58406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb598036000 tx=0x7fb598008030 comp rx=0 tx=0).stop 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.460+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 msgr2=0x7fb5a80ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.460+0000 7fb5aea00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 0x7fb5a80ada10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb59000c7c0 tx=0x7fb59000cc80 comp rx=0 tx=0).stop 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 shutdown_connections 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5a81529e0 0x7fb5a81e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a80c64c0 0x7fb5a80ada10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb58406b4d0 0x7fb58406d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 --2- 172.21.15.67:0/2165659066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5a80703a0 0x7fb5a81db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 >> 172.21.15.67:0/2165659066 conn(0x7fb5a80aa330 msgr2=0x7fb5a80c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 shutdown_connections 2024-02-26T10:39:10.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.461+0000 7fb5aea00640 1 -- 172.21.15.67:0/2165659066 wait complete. 2024-02-26T10:39:10.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:10.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.699+0000 7f46e9a00640 1 Processor -- start 2024-02-26T10:39:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.700+0000 7f46e9a00640 1 -- start start 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.701+0000 7f46e9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46e40c8780 0x7f46e4158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.701+0000 7f46e9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.701+0000 7f46e9a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40c5aa0 con 0x7f46e4152710 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.701+0000 7f46e9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e4155b40 con 0x7f46e40c8780 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.701+0000 7f46e9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40c52a0 con 0x7f46e4158a50 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.701+0000 7f46e2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f46e4152710 0x7f46e40c7290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39640/0 (socket says 172.21.15.67:39640) 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e2a00640 1 -- 172.21.15.67:0/2985337076 learned_addr learned my addr 172.21.15.67:0/2985337076 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e2000640 1 --2- 172.21.15.67:0/2985337076 >> v2:172.21.15.67:3300/0 conn(0x7f46e40c8780 0x7f46e4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e3400640 1 --2- 172.21.15.67:0/2985337076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e1600640 1 -- 172.21.15.67:0/2985337076 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345387322 0 0) 0x7f46e40c5aa0 con 0x7f46e4152710 2024-02-26T10:39:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e1600640 1 -- 172.21.15.67:0/2985337076 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c8003520 con 0x7f46e4152710 2024-02-26T10:39:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e2000640 1 -- 172.21.15.67:0/2985337076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 msgr2=0x7f46e415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e2000640 1 --2- 172.21.15.67:0/2985337076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e415f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e2000640 1 -- 172.21.15.67:0/2985337076 >> v1:172.21.15.67:6789/0 conn(0x7f46e4152710 legacy=0x7f46e40c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.702+0000 7f46e2000640 1 -- 172.21.15.67:0/2985337076 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4158f00 con 0x7f46e40c8780 2024-02-26T10:39:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.703+0000 7f46e2000640 1 --2- 172.21.15.67:0/2985337076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e4158510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0097a0 tx=0x7f46cc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dc2718afa56f47 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.704+0000 7f46e1600640 1 -- 172.21.15.67:0/2985337076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc031780 con 0x7f46e40c8780 2024-02-26T10:39:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.704+0000 7f46e1600640 1 -- 172.21.15.67:0/2985337076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc037070 con 0x7f46e40c8780 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.704+0000 7f46e1600640 1 -- 172.21.15.67:0/2985337076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc031b40 con 0x7f46e40c8780 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 -- 172.21.15.67:0/2985337076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 msgr2=0x7f46e4158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 --2- 172.21.15.67:0/2985337076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e4158510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0097a0 tx=0x7f46cc02b820 comp rx=0 tx=0).stop 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 -- 172.21.15.67:0/2985337076 shutdown_connections 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 --2- 172.21.15.67:0/2985337076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 --2- 172.21.15.67:0/2985337076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e4158510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 -- 172.21.15.67:0/2985337076 >> 172.21.15.67:0/2985337076 conn(0x7f46e40ac410 msgr2=0x7f46e40ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:10.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 -- 172.21.15.67:0/2985337076 shutdown_connections 2024-02-26T10:39:10.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.705+0000 7f46e9a00640 1 -- 172.21.15.67:0/2985337076 wait complete. 2024-02-26T10:39:10.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.706+0000 7f46e9a00640 1 Processor -- start 2024-02-26T10:39:10.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 -- start start 2024-02-26T10:39:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e40c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e4152710 0x7f46e40c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e40bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40bdf10 con 0x7f46e40c8780 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40ca5d0 con 0x7f46e4152710 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.707+0000 7f46e9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40c79a0 con 0x7f46e4158a50 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e40c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e40c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43388/0 (socket says 172.21.15.67:43388) 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 -- 172.21.15.67:0/3946384670 learned_addr learned my addr 172.21.15.67:0/3946384670 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2000640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e4152710 0x7f46e40c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e3400640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e40bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 -- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 msgr2=0x7f46e40bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e40bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 -- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e4152710 msgr2=0x7f46e40c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e4152710 0x7f46e40c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2a00640 1 -- 172.21.15.67:0/3946384670 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4157e00 con 0x7f46e40c8780 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.708+0000 7f46e2000640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e4152710 0x7f46e40c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.709+0000 7f46e2a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e40c0b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46d400cb40 tx=0x7f46d400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.709+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4017070 con 0x7f46e40c8780 2024-02-26T10:39:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.709+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d400aca0 con 0x7f46e40c8780 2024-02-26T10:39:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.709+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4012620 con 0x7f46e40c8780 2024-02-26T10:39:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.709+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e41ea560 con 0x7f46e40c8780 2024-02-26T10:39:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.709+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e41eab20 con 0x7f46e40c8780 2024-02-26T10:39:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.710+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e40678e0 con 0x7f46e40c8780 2024-02-26T10:39:10.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.711+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46d401a070 con 0x7f46e40c8780 2024-02-26T10:39:10.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.712+0000 7f46daa00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f46c006b500 0x7f46c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:10.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.712+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46d408a0d0 con 0x7f46e40c8780 2024-02-26T10:39:10.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.713+0000 7f46e2000640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f46c006b500 0x7f46c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:10.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.714+0000 7f46e2000640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f46c006b500 0x7f46c006d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46cc036000 tx=0x7f46cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:10.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.717+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46e4157e00 con 0x7f46e40c8780 2024-02-26T10:39:10.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.914+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46e4074a30 con 0x7f46e40c8780 2024-02-26T10:39:10.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.915+0000 7f46daa00640 1 -- 172.21.15.67:0/3946384670 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46d4058eb0 con 0x7f46e40c8780 2024-02-26T10:39:10.917 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.917+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f46c006b500 msgr2=0x7f46c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.917+0000 7f46e9a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f46c006b500 0x7f46c006d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46cc036000 tx=0x7f46cc008030 comp rx=0 tx=0).stop 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.917+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 msgr2=0x7f46e40c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.917+0000 7f46e9a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e40c0b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46d400cb40 tx=0x7f46d400a060 comp rx=0 tx=0).stop 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 shutdown_connections 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46e4158a50 0x7f46e40bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f46c006b500 0x7f46c006d9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46e4152710 0x7f46e40c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 --2- 172.21.15.67:0/3946384670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46e40c8780 0x7f46e40c0b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 >> 172.21.15.67:0/3946384670 conn(0x7f46e40ac410 msgr2=0x7f46e41550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:10.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 shutdown_connections 2024-02-26T10:39:10.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:10.918+0000 7f46e9a00640 1 -- 172.21.15.67:0/3946384670 wait complete. 2024-02-26T10:39:10.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:11.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.151+0000 7f42d5e00640 1 Processor -- start 2024-02-26T10:39:11.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.151+0000 7f42d5e00640 1 -- start start 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42d5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42d00c5290 0x7f42d01562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42d5e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f42d00af8a0 0x7f42d00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42d5e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d006c110 con 0x7f42d0155880 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42d5e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d006d000 con 0x7f42d00c5290 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42d5e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00c72c0 con 0x7f42d00af8a0 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42cfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f42d0155880 0x7f42d006c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39648/0 (socket says 172.21.15.67:39648) 2024-02-26T10:39:11.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.153+0000 7f42cfe00640 1 -- 172.21.15.67:0/3765970009 learned_addr learned my addr 172.21.15.67:0/3765970009 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42d4a00640 1 --2- 172.21.15.67:0/3765970009 >> v2:172.21.15.67:3300/0 conn(0x7f42d00c5290 0x7f42d01562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42d5400640 1 --2- 172.21.15.67:0/3765970009 >> v2:172.21.15.82:3300/0 conn(0x7f42d00af8a0 0x7f42d00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42cf400640 1 -- 172.21.15.67:0/3765970009 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594975932 0 0) 0x7f42d006c110 con 0x7f42d0155880 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42cf400640 1 -- 172.21.15.67:0/3765970009 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b4003520 con 0x7f42d0155880 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42d4a00640 1 -- 172.21.15.67:0/3765970009 >> v2:172.21.15.82:3300/0 conn(0x7f42d00af8a0 msgr2=0x7f42d00afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42d4a00640 1 --2- 172.21.15.67:0/3765970009 >> v2:172.21.15.82:3300/0 conn(0x7f42d00af8a0 0x7f42d00afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42d4a00640 1 -- 172.21.15.67:0/3765970009 >> v1:172.21.15.67:6789/0 conn(0x7f42d0155880 legacy=0x7f42d006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.154+0000 7f42d4a00640 1 -- 172.21.15.67:0/3765970009 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00c75a0 con 0x7f42d00c5290 2024-02-26T10:39:11.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.156+0000 7f42d4a00640 1 --2- 172.21.15.67:0/3765970009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01562f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42d00ca650 tx=0x7f42c002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e905aead440ac62a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:11.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.156+0000 7f42cf400640 1 -- 172.21.15.67:0/3765970009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0039070 con 0x7f42d00c5290 2024-02-26T10:39:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.156+0000 7f42cf400640 1 -- 172.21.15.67:0/3765970009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c002cbf0 con 0x7f42d00c5290 2024-02-26T10:39:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.156+0000 7f42cf400640 1 -- 172.21.15.67:0/3765970009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0034440 con 0x7f42d00c5290 2024-02-26T10:39:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.156+0000 7f42d5e00640 1 -- 172.21.15.67:0/3765970009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 msgr2=0x7f42d01562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.156+0000 7f42d5e00640 1 --2- 172.21.15.67:0/3765970009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01562f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42d00ca650 tx=0x7f42c002be40 comp rx=0 tx=0).stop 2024-02-26T10:39:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.157+0000 7f42d5e00640 1 -- 172.21.15.67:0/3765970009 shutdown_connections 2024-02-26T10:39:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.157+0000 7f42d5e00640 1 --2- 172.21.15.67:0/3765970009 >> v2:172.21.15.82:3300/0 conn(0x7f42d00af8a0 0x7f42d00afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.157+0000 7f42d5e00640 1 --2- 172.21.15.67:0/3765970009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01562f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.157+0000 7f42d5e00640 1 -- 172.21.15.67:0/3765970009 >> 172.21.15.67:0/3765970009 conn(0x7f42d00ac230 msgr2=0x7f42d00ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.157+0000 7f42d5e00640 1 -- 172.21.15.67:0/3765970009 shutdown_connections 2024-02-26T10:39:11.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.157+0000 7f42d5e00640 1 -- 172.21.15.67:0/3765970009 wait complete. 2024-02-26T10:39:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.158+0000 7f42d5e00640 1 Processor -- start 2024-02-26T10:39:11.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 -- start start 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d00af8a0 0x7f42d00c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01ec230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d0155880 0x7f42d014d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0155cf0 con 0x7f42d00c5290 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d006c3d0 con 0x7f42d0155880 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.159+0000 7f42d5e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00c5670 con 0x7f42d00af8a0 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01ec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43406/0 (socket says 172.21.15.67:43406) 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 -- 172.21.15.67:0/4209096548 learned_addr learned my addr 172.21.15.67:0/4209096548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42d4a00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d00af8a0 0x7f42d00c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42d5400640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d0155880 0x7f42d014d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 -- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d00af8a0 msgr2=0x7f42d00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d00af8a0 0x7f42d00c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 -- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d0155880 msgr2=0x7f42d014d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d0155880 0x7f42d014d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42cfe00640 1 -- 172.21.15.67:0/4209096548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d01de920 con 0x7f42d00c5290 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.160+0000 7f42d5400640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d0155880 0x7f42d014d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:11.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.161+0000 7f42cfe00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01ec230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c390 tx=0x7f42b800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.161+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8017070 con 0x7f42d00c5290 2024-02-26T10:39:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.161+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b8005ce0 con 0x7f42d00c5290 2024-02-26T10:39:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.161+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80072f0 con 0x7f42d00c5290 2024-02-26T10:39:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.161+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d0153c20 con 0x7f42d00c5290 2024-02-26T10:39:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.161+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d01541e0 con 0x7f42d00c5290 2024-02-26T10:39:11.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.162+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0074a30 con 0x7f42d00c5290 2024-02-26T10:39:11.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.163+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42b801a070 con 0x7f42d00c5290 2024-02-26T10:39:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.165+0000 7f42ccc00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42ac06b500 0x7f42ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.165+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42b80895d0 con 0x7f42d00c5290 2024-02-26T10:39:11.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.165+0000 7f42d4a00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42ac06b500 0x7f42ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.167+0000 7f42d4a00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42ac06b500 0x7f42ac06d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42c0036000 tx=0x7f42c0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:11.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.171+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42b8058450 con 0x7f42d00c5290 2024-02-26T10:39:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.371+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42d00aad60 con 0x7f42d00c5290 2024-02-26T10:39:11.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.371+0000 7f42ccc00640 1 -- 172.21.15.67:0/4209096548 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42b8057d50 con 0x7f42d00c5290 2024-02-26T10:39:11.373 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42ac06b500 msgr2=0x7f42ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42ac06b500 0x7f42ac06d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42c0036000 tx=0x7f42c0030000 comp rx=0 tx=0).stop 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 msgr2=0x7f42d01ec230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01ec230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42b800c390 tx=0x7f42b800c850 comp rx=0 tx=0).stop 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 shutdown_connections 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42ac06b500 0x7f42ac06d9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d0155880 0x7f42d014d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42d00c5290 0x7f42d01ec230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 --2- 172.21.15.67:0/4209096548 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d00af8a0 0x7f42d00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 >> 172.21.15.67:0/4209096548 conn(0x7f42d00ac230 msgr2=0x7f42d00afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 shutdown_connections 2024-02-26T10:39:11.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.374+0000 7f42d5e00640 1 -- 172.21.15.67:0/4209096548 wait complete. 2024-02-26T10:39:11.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:11.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.620+0000 7ff446400640 1 Processor -- start 2024-02-26T10:39:11.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.621+0000 7ff446400640 1 -- start start 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff446400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4401503d0 0x7ff44006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff446400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff44006e390 0x7ff4401533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff446400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4400c2950 0x7ff440153920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff446400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4400a6eb0 con 0x7ff4400c2950 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff446400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4400a7190 con 0x7ff4401503d0 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff446400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400a7470 con 0x7ff44006e390 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.622+0000 7ff43fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4400c2950 0x7ff440153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4400c2950 0x7ff440153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43412/0 (socket says 172.21.15.67:43412) 2024-02-26T10:39:11.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 -- 172.21.15.67:0/2137066190 learned_addr learned my addr 172.21.15.67:0/2137066190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:11.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43f400640 1 --2- 172.21.15.67:0/2137066190 >> v2:172.21.15.82:3300/0 conn(0x7ff4401503d0 0x7ff44006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43ea00640 1 --2- 172.21.15.67:0/2137066190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff44006e390 0x7ff4401533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 -- 172.21.15.67:0/2137066190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff44006e390 msgr2=0x7ff4401533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:11.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 --2- 172.21.15.67:0/2137066190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff44006e390 0x7ff4401533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 -- 172.21.15.67:0/2137066190 >> v2:172.21.15.82:3300/0 conn(0x7ff4401503d0 msgr2=0x7ff44006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:11.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 --2- 172.21.15.67:0/2137066190 >> v2:172.21.15.82:3300/0 conn(0x7ff4401503d0 0x7ff44006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43fe00640 1 -- 172.21.15.67:0/2137066190 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400a7750 con 0x7ff4400c2950 2024-02-26T10:39:11.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.623+0000 7ff43f400640 1 --2- 172.21.15.67:0/2137066190 >> v2:172.21.15.82:3300/0 conn(0x7ff4401503d0 0x7ff44006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.624+0000 7ff43fe00640 1 --2- 172.21.15.67:0/2137066190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4400c2950 0x7ff440153920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff44006f1d0 tx=0x7ff43802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceabc8f7a5a5c7b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff43e000640 1 -- 172.21.15.67:0/2137066190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438038070 con 0x7ff4400c2950 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff43e000640 1 -- 172.21.15.67:0/2137066190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43802f050 con 0x7ff4400c2950 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff43e000640 1 -- 172.21.15.67:0/2137066190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438031970 con 0x7ff4400c2950 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 -- 172.21.15.67:0/2137066190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4400c2950 msgr2=0x7ff440153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 --2- 172.21.15.67:0/2137066190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4400c2950 0x7ff440153920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff44006f1d0 tx=0x7ff43802b7f0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 -- 172.21.15.67:0/2137066190 shutdown_connections 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 --2- 172.21.15.67:0/2137066190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4400c2950 0x7ff440153920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 --2- 172.21.15.67:0/2137066190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff44006e390 0x7ff4401533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 --2- 172.21.15.67:0/2137066190 >> v2:172.21.15.82:3300/0 conn(0x7ff4401503d0 0x7ff44006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 -- 172.21.15.67:0/2137066190 >> 172.21.15.67:0/2137066190 conn(0x7ff4400c6da0 msgr2=0x7ff4400c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 -- 172.21.15.67:0/2137066190 shutdown_connections 2024-02-26T10:39:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.625+0000 7ff446400640 1 -- 172.21.15.67:0/2137066190 wait complete. 2024-02-26T10:39:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.627+0000 7ff446400640 1 Processor -- start 2024-02-26T10:39:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.627+0000 7ff446400640 1 -- start start 2024-02-26T10:39:11.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff446400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 0x7ff4400a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff446400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4400c2950 0x7ff44014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff446400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4401503d0 0x7ff4400c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff446400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4401507b0 con 0x7ff44006e390 2024-02-26T10:39:11.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff446400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44006ef00 con 0x7ff4401503d0 2024-02-26T10:39:11.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff446400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400c0ef0 con 0x7ff4400c2950 2024-02-26T10:39:11.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.628+0000 7ff43f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 0x7ff4400a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 0x7ff4400a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43418/0 (socket says 172.21.15.67:43418) 2024-02-26T10:39:11.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 -- 172.21.15.67:0/3774900111 learned_addr learned my addr 172.21.15.67:0/3774900111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:11.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43ea00640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4400c2950 0x7ff44014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43fe00640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4401503d0 0x7ff4400c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 -- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4400c2950 msgr2=0x7ff44014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4400c2950 0x7ff44014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 -- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4401503d0 msgr2=0x7ff4400c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4401503d0 0x7ff4400c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 -- 172.21.15.67:0/3774900111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4401e8380 con 0x7ff44006e390 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.629+0000 7ff43f400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 0x7ff4400a9e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff43000cf60 tx=0x7ff43000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.630+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43001a070 con 0x7ff44006e390 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.630+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43000d840 con 0x7ff44006e390 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.630+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43000dc10 con 0x7ff44006e390 2024-02-26T10:39:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.630+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4400aa380 con 0x7ff44006e390 2024-02-26T10:39:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.630+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4400c45d0 con 0x7ff44006e390 2024-02-26T10:39:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.631+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4400c40e0 con 0x7ff44006e390 2024-02-26T10:39:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.632+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff43001d070 con 0x7ff44006e390 2024-02-26T10:39:11.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.633+0000 7ff437400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff41c06b4d0 0x7ff41c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:11.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.633+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff43008a9f0 con 0x7ff44006e390 2024-02-26T10:39:11.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.634+0000 7ff43ea00640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff41c06b4d0 0x7ff41c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:11.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.635+0000 7ff43ea00640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff41c06b4d0 0x7ff41c06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff428006fd0 tx=0x7ff428008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.638+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff430059180 con 0x7ff44006e390 2024-02-26T10:39:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.834+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff440074a30 con 0x7ff44006e390 2024-02-26T10:39:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.835+0000 7ff437400640 1 -- 172.21.15.67:0/3774900111 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff43005d220 con 0x7ff44006e390 2024-02-26T10:39:11.837 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff41c06b4d0 msgr2=0x7ff41c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff41c06b4d0 0x7ff41c06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff428006fd0 tx=0x7ff428008040 comp rx=0 tx=0).stop 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 msgr2=0x7ff4400a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 0x7ff4400a9e40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff43000cf60 tx=0x7ff43000cf90 comp rx=0 tx=0).stop 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 shutdown_connections 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff41c06b4d0 0x7ff41c06d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4401503d0 0x7ff4400c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.837+0000 7ff446400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4400c2950 0x7ff44014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.838+0000 7ff446400640 1 --2- 172.21.15.67:0/3774900111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff44006e390 0x7ff4400a9e40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.838+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 >> 172.21.15.67:0/3774900111 conn(0x7ff4400c6da0 msgr2=0x7ff4400c2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.838+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 shutdown_connections 2024-02-26T10:39:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:11.838+0000 7ff446400640 1 -- 172.21.15.67:0/3774900111 wait complete. 2024-02-26T10:39:11.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.072+0000 7fe174800640 1 Processor -- start 2024-02-26T10:39:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.072+0000 7fe174800640 1 -- start start 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe174800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe16c14f3d0 0x7fe16c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe174800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe16c0c2870 0x7fe16c155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe174800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c06ed10 con 0x7fe16c1556a0 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe174800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c06dd80 con 0x7fe16c14f3d0 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe174800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c155c00 con 0x7fe16c0c2870 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe171e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe16c14f3d0 0x7fe16c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe171e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe16c14f3d0 0x7fe16c06e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43428/0 (socket says 172.21.15.67:43428) 2024-02-26T10:39:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.074+0000 7fe171e00640 1 -- 172.21.15.67:0/3616509002 learned_addr learned my addr 172.21.15.67:0/3616509002 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe171400640 1 --2- 172.21.15.67:0/3616509002 >> v2:172.21.15.82:3300/0 conn(0x7fe16c0c2870 0x7fe16c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe171e00640 1 -- 172.21.15.67:0/3616509002 >> v2:172.21.15.82:3300/0 conn(0x7fe16c0c2870 msgr2=0x7fe16c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe171e00640 1 --2- 172.21.15.67:0/3616509002 >> v2:172.21.15.82:3300/0 conn(0x7fe16c0c2870 0x7fe16c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe171e00640 1 -- 172.21.15.67:0/3616509002 >> v1:172.21.15.67:6789/0 conn(0x7fe16c1556a0 legacy=0x7fe16c071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe171e00640 1 -- 172.21.15.67:0/3616509002 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c155ee0 con 0x7fe16c14f3d0 2024-02-26T10:39:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe170a00640 1 -- 172.21.15.67:0/3616509002 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109810825 0 0) 0x7fe16c06ed10 con 0x7fe16c1556a0 2024-02-26T10:39:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.075+0000 7fe171400640 1 --2- 172.21.15.67:0/3616509002 >> v2:172.21.15.82:3300/0 conn(0x7fe16c0c2870 0x7fe16c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.076+0000 7fe171e00640 1 --2- 172.21.15.67:0/3616509002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c14f3d0 0x7fe16c06e7d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0c1120 tx=0x7fe16002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256d19192561b5ea server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.076+0000 7fe170a00640 1 -- 172.21.15.67:0/3616509002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16002e070 con 0x7fe16c14f3d0 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.076+0000 7fe170a00640 1 -- 172.21.15.67:0/3616509002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe160037070 con 0x7fe16c14f3d0 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe170a00640 1 -- 172.21.15.67:0/3616509002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160031a00 con 0x7fe16c14f3d0 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 -- 172.21.15.67:0/3616509002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c14f3d0 msgr2=0x7fe16c06e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 --2- 172.21.15.67:0/3616509002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c14f3d0 0x7fe16c06e7d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0c1120 tx=0x7fe16002b880 comp rx=0 tx=0).stop 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 -- 172.21.15.67:0/3616509002 shutdown_connections 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 --2- 172.21.15.67:0/3616509002 >> v2:172.21.15.82:3300/0 conn(0x7fe16c0c2870 0x7fe16c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 --2- 172.21.15.67:0/3616509002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c14f3d0 0x7fe16c06e7d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 -- 172.21.15.67:0/3616509002 >> 172.21.15.67:0/3616509002 conn(0x7fe16c0a84a0 msgr2=0x7fe16c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:12.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 -- 172.21.15.67:0/3616509002 shutdown_connections 2024-02-26T10:39:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.077+0000 7fe174800640 1 -- 172.21.15.67:0/3616509002 wait complete. 2024-02-26T10:39:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.079+0000 7fe174800640 1 Processor -- start 2024-02-26T10:39:12.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.079+0000 7fe174800640 1 -- start start 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe174800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16c0c2870 0x7fe16c142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe174800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe16c14f3d0 0x7fe16c1edf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe174800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 0x7fe16c1ee490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe174800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0c1810 con 0x7fe16c1556a0 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe174800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c06e2b0 con 0x7fe16c0c2870 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe174800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c071e10 con 0x7fe16c14f3d0 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe172800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 0x7fe16c1ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe172800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 0x7fe16c1ee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43432/0 (socket says 172.21.15.67:43432) 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.080+0000 7fe172800640 1 -- 172.21.15.67:0/2933010677 learned_addr learned my addr 172.21.15.67:0/2933010677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe171e00640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16c0c2870 0x7fe16c142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe171400640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe16c14f3d0 0x7fe16c1edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe172800640 1 -- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe16c14f3d0 msgr2=0x7fe16c1edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe172800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe16c14f3d0 0x7fe16c1edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe172800640 1 -- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16c0c2870 msgr2=0x7fe16c142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe172800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16c0c2870 0x7fe16c142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe172800640 1 -- 172.21.15.67:0/2933010677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0c4e70 con 0x7fe16c1556a0 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe171400640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe16c14f3d0 0x7fe16c1edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe172800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 0x7fe16c1ee490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c760 tx=0x7fe16800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168003da0 con 0x7fe16c1556a0 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168005070 con 0x7fe16c1556a0 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168005460 con 0x7fe16c1556a0 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c0c5540 con 0x7fe16c1556a0 2024-02-26T10:39:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.081+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c143490 con 0x7fe16c1556a0 2024-02-26T10:39:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.082+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c074a30 con 0x7fe16c1556a0 2024-02-26T10:39:12.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.084+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe168005600 con 0x7fe16c1556a0 2024-02-26T10:39:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.084+0000 7fe166000640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe15006b4d0 0x7fe15006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.084+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe16808a100 con 0x7fe16c1556a0 2024-02-26T10:39:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.085+0000 7fe171e00640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe15006b4d0 0x7fe15006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.086+0000 7fe171e00640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe15006b4d0 0x7fe15006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe160036000 tx=0x7fe160008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.089+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe168059010 con 0x7fe16c1556a0 2024-02-26T10:39:12.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.290+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe16c0c13f0 con 0x7fe16c1556a0 2024-02-26T10:39:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.293+0000 7fe166000640 1 -- 172.21.15.67:0/2933010677 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7fe168058910 con 0x7fe16c1556a0 2024-02-26T10:39:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:12.296 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:12.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe15006b4d0 msgr2=0x7fe15006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe15006b4d0 0x7fe15006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe160036000 tx=0x7fe160008030 comp rx=0 tx=0).stop 2024-02-26T10:39:12.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 msgr2=0x7fe16c1ee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 0x7fe16c1ee490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c760 tx=0x7fe16800cc20 comp rx=0 tx=0).stop 2024-02-26T10:39:12.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 shutdown_connections 2024-02-26T10:39:12.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe16c1556a0 0x7fe16c1ee490 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe16c14f3d0 0x7fe16c1edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe15006b4d0 0x7fe15006d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 --2- 172.21.15.67:0/2933010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16c0c2870 0x7fe16c142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 >> 172.21.15.67:0/2933010677 conn(0x7fe16c0a84a0 msgr2=0x7fe16c14cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:12.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.296+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 shutdown_connections 2024-02-26T10:39:12.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.297+0000 7fe174800640 1 -- 172.21.15.67:0/2933010677 wait complete. 2024-02-26T10:39:12.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:12.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.533+0000 7f866a800640 1 Processor -- start 2024-02-26T10:39:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.533+0000 7f866a800640 1 -- start start 2024-02-26T10:39:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f866a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f86641591b0 0x7f8664158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f866a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c1400 0x7f866406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f866a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8664072ce0 con 0x7f86640c0ef0 2024-02-26T10:39:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f866a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8664073010 con 0x7f86641591b0 2024-02-26T10:39:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f866a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866414f490 con 0x7f86640c1400 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f8663400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f86641591b0 0x7f8664158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f8663400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f86641591b0 0x7f8664158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43440/0 (socket says 172.21.15.67:43440) 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f8663400640 1 -- 172.21.15.67:0/2723469696 learned_addr learned my addr 172.21.15.67:0/2723469696 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.535+0000 7f8662a00640 1 --1- 172.21.15.67:0/2723469696 >> v1:172.21.15.67:6789/0 conn(0x7f86640c0ef0 0x7f8664070ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39658/0 (socket says 172.21.15.67:39658) 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.536+0000 7f8663e00640 1 --2- 172.21.15.67:0/2723469696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c1400 0x7f866406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.536+0000 7f8663400640 1 -- 172.21.15.67:0/2723469696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c1400 msgr2=0x7f866406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.536+0000 7f8663400640 1 --2- 172.21.15.67:0/2723469696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c1400 0x7f866406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.536+0000 7f8663400640 1 -- 172.21.15.67:0/2723469696 >> v1:172.21.15.67:6789/0 conn(0x7f86640c0ef0 legacy=0x7f8664070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.536+0000 7f8663400640 1 -- 172.21.15.67:0/2723469696 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664155bf0 con 0x7f86641591b0 2024-02-26T10:39:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.537+0000 7f8663400640 1 --2- 172.21.15.67:0/2723469696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86641591b0 0x7f8664158580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8664073190 tx=0x7f865402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3ab86261568030 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.537+0000 7f8662000640 1 -- 172.21.15.67:0/2723469696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654035020 con 0x7f86641591b0 2024-02-26T10:39:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.537+0000 7f8662000640 1 -- 172.21.15.67:0/2723469696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654031640 con 0x7f86641591b0 2024-02-26T10:39:12.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.537+0000 7f8662000640 1 -- 172.21.15.67:0/2723469696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865402d040 con 0x7f86641591b0 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 -- 172.21.15.67:0/2723469696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86641591b0 msgr2=0x7f8664158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 --2- 172.21.15.67:0/2723469696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86641591b0 0x7f8664158580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8664073190 tx=0x7f865402b8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 -- 172.21.15.67:0/2723469696 shutdown_connections 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 --2- 172.21.15.67:0/2723469696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c1400 0x7f866406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 --2- 172.21.15.67:0/2723469696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86641591b0 0x7f8664158580 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 -- 172.21.15.67:0/2723469696 >> 172.21.15.67:0/2723469696 conn(0x7f86640c6da0 msgr2=0x7f86640c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 -- 172.21.15.67:0/2723469696 shutdown_connections 2024-02-26T10:39:12.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.538+0000 7f866a800640 1 -- 172.21.15.67:0/2723469696 wait complete. 2024-02-26T10:39:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.540+0000 7f866a800640 1 Processor -- start 2024-02-26T10:39:12.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.541+0000 7f866a800640 1 -- start start 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f866a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c0ef0 0x7f8664148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f866a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 0x7f86641ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f866a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86641591b0 0x7f86641540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f866a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640a8ba0 con 0x7f86640c1400 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f866a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86640715e0 con 0x7f86641591b0 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f866a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640a8d20 con 0x7f86640c0ef0 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 0x7f86641ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 0x7f86641ee0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43444/0 (socket says 172.21.15.67:43444) 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 -- 172.21.15.67:0/825298745 learned_addr learned my addr 172.21.15.67:0/825298745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8663e00640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86641591b0 0x7f86641540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8663400640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c0ef0 0x7f8664148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 -- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c0ef0 msgr2=0x7f8664148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c0ef0 0x7f8664148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 -- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86641591b0 msgr2=0x7f86641540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.542+0000 7f8662a00640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86641591b0 0x7f86641540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f8662a00640 1 -- 172.21.15.67:0/825298745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640ac140 con 0x7f86640c1400 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f8662a00640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 0x7f86641ee0b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8650001eb0 tx=0x7f865000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86500074e0 con 0x7f86640c1400 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86500076a0 con 0x7f86640c1400 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650004a20 con 0x7f86640c1400 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86640ac420 con 0x7f86640c1400 2024-02-26T10:39:12.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.543+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8664151d20 con 0x7f86640c1400 2024-02-26T10:39:12.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.544+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8664074a30 con 0x7f86640c1400 2024-02-26T10:39:12.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.545+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f865001a020 con 0x7f86640c1400 2024-02-26T10:39:12.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.546+0000 7f865b400640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f864406b4d0 0x7f864406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.546+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86500899d0 con 0x7f86640c1400 2024-02-26T10:39:12.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.547+0000 7f8663400640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f864406b4d0 0x7f864406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.548+0000 7f8663400640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f864406b4d0 0x7f864406d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8654036000 tx=0x7f8654008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.551+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8650010300 con 0x7f86640c1400 2024-02-26T10:39:12.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.747+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f866406eb10 con 0x7f86640c1400 2024-02-26T10:39:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.748+0000 7f865b400640 1 -- 172.21.15.67:0/825298745 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f865005e070 con 0x7f86640c1400 2024-02-26T10:39:12.750 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f864406b4d0 msgr2=0x7f864406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f864406b4d0 0x7f864406d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8654036000 tx=0x7f8654008030 comp rx=0 tx=0).stop 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 msgr2=0x7f86641ee0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 0x7f86641ee0b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8650001eb0 tx=0x7f865000c890 comp rx=0 tx=0).stop 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 shutdown_connections 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f864406b4d0 0x7f864406d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86641591b0 0x7f86641540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86640c1400 0x7f86641ee0b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 --2- 172.21.15.67:0/825298745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86640c0ef0 0x7f8664148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 >> 172.21.15.67:0/825298745 conn(0x7f86640c6da0 msgr2=0x7f8664156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 shutdown_connections 2024-02-26T10:39:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.751+0000 7f866a800640 1 -- 172.21.15.67:0/825298745 wait complete. 2024-02-26T10:39:12.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:12.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.984+0000 7f5b41600640 1 Processor -- start 2024-02-26T10:39:12.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.984+0000 7f5b41600640 1 -- start start 2024-02-26T10:39:12.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.985+0000 7f5b41600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5b3c156320 0x7f5b3c156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.985+0000 7f5b41600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c156cd0 con 0x7f5b3c071710 2024-02-26T10:39:12.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.985+0000 7f5b41600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c15d0b0 con 0x7f5b3c0c2540 2024-02-26T10:39:12.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.985+0000 7f5b41600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c15d390 con 0x7f5b3c156320 2024-02-26T10:39:12.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.986+0000 7f5b3be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5b3c071710 0x7f5b3c06e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39660/0 (socket says 172.21.15.67:39660) 2024-02-26T10:39:12.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.986+0000 7f5b3be00640 1 -- 172.21.15.67:0/1944189403 learned_addr learned my addr 172.21.15.67:0/1944189403 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.986+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102737204 0 0) 0x7f5b3c156cd0 con 0x7f5b3c071710 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.986+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b20003520 con 0x7f5b3c071710 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.986+0000 7f5b40c00640 1 --2- 172.21.15.67:0/1944189403 >> v2:172.21.15.82:3300/0 conn(0x7f5b3c156320 0x7f5b3c156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570668050 0 0) 0x7f5b20003520 con 0x7f5b3c071710 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 >> v2:172.21.15.82:3300/0 conn(0x7f5b3c156320 msgr2=0x7f5b3c156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 --2- 172.21.15.67:0/1944189403 >> v2:172.21.15.82:3300/0 conn(0x7f5b3c156320 0x7f5b3c156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 >> v1:172.21.15.82:6789/0 conn(0x7f5b3c0c2540 legacy=0x7f5b3c15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c06dbd0 con 0x7f5b3c071710 2024-02-26T10:39:12.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f5b2c001670 con 0x7f5b3c071710 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 >> v1:172.21.15.67:6789/0 conn(0x7f5b3c071710 legacy=0x7f5b3c06e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b20004800 0x7f5b20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b200052e0 0x7f5b2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c15d390 con 0x7f5b20003a00 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2000c200 con 0x7f5b20004800 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2000c400 con 0x7f5b200052e0 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b40c00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b200052e0 0x7f5b2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b33e00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b20004800 0x7f5b20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3be00640 1 -- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b200052e0 msgr2=0x7f5b2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b200052e0 0x7f5b2000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3be00640 1 -- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b20004800 msgr2=0x7f5b20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:12.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b20004800 0x7f5b20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.987+0000 7f5b3be00640 1 -- 172.21.15.67:0/1944189403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b20003520 con 0x7f5b20003a00 2024-02-26T10:39:12.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.988+0000 7f5b40c00640 1 -- 172.21.15.67:0/1944189403 reap_dead start 2024-02-26T10:39:12.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.988+0000 7f5b33e00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b20004800 0x7f5b20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:12.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.989+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b20003e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c0e0 tx=0x7f5b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ada60aac2a066e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.989+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c007c20 con 0x7f5b20003a00 2024-02-26T10:39:12.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.989+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c030070 con 0x7f5b20003a00 2024-02-26T10:39:12.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.989+0000 7f5b3b400640 1 -- 172.21.15.67:0/1944189403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c005d70 con 0x7f5b20003a00 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.989+0000 7f5b41600640 1 -- 172.21.15.67:0/1944189403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 msgr2=0x7f5b20003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.989+0000 7f5b41600640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b20003e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c0e0 tx=0x7f5b20005830 comp rx=0 tx=0).stop 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.990+0000 7f5b41600640 1 -- 172.21.15.67:0/1944189403 shutdown_connections 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.990+0000 7f5b41600640 1 --2- 172.21.15.67:0/1944189403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b20003e20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.990+0000 7f5b41600640 1 -- 172.21.15.67:0/1944189403 >> 172.21.15.67:0/1944189403 conn(0x7f5b3c0c6380 msgr2=0x7f5b3c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.990+0000 7f5b41600640 1 -- 172.21.15.67:0/1944189403 shutdown_connections 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.990+0000 7f5b41600640 1 -- 172.21.15.67:0/1944189403 wait complete. 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.991+0000 7f5b41600640 1 Processor -- start 2024-02-26T10:39:12.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.992+0000 7f5b41600640 1 -- start start 2024-02-26T10:39:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.992+0000 7f5b41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b3c1e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b41600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b3c14f590 0x7f5b3c1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b41600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c0c07b0 0x7f5b3c0a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b41600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c159290 con 0x7f5b20003a00 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b41600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c0c29d0 con 0x7f5b3c0c07b0 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b41600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c159410 con 0x7f5b3c14f590 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b3c1e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b3c1e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43456/0 (socket says 172.21.15.67:43456) 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 -- 172.21.15.67:0/1515975448 learned_addr learned my addr 172.21.15.67:0/1515975448 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b40c00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c0c07b0 0x7f5b3c0a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 -- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b3c14f590 msgr2=0x7f5b3c1f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b33e00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b3c14f590 0x7f5b3c1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b3c14f590 0x7f5b3c1f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 -- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c0c07b0 msgr2=0x7f5b3c0a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c0c07b0 0x7f5b3c0a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b3be00640 1 -- 172.21.15.67:0/1515975448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c1579e0 con 0x7f5b20003a00 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.993+0000 7f5b40c00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c0c07b0 0x7f5b3c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.994+0000 7f5b3be00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b3c1e5ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c0e0 tx=0x7f5b3c159090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.994+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c001b60 con 0x7f5b20003a00 2024-02-26T10:39:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.994+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c000ed0 con 0x7f5b20003a00 2024-02-26T10:39:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.994+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c14fa20 con 0x7f5b20003a00 2024-02-26T10:39:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.994+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c02cdd0 con 0x7f5b20003a00 2024-02-26T10:39:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.994+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c1f29c0 con 0x7f5b20003a00 2024-02-26T10:39:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.995+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c0678e0 con 0x7f5b20003a00 2024-02-26T10:39:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.996+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b2c048060 con 0x7f5b20003a00 2024-02-26T10:39:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.997+0000 7f5b38c00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5b1806b4d0 0x7f5b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.997+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b2c0ab2c0 con 0x7f5b20003a00 2024-02-26T10:39:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.998+0000 7f5b33e00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5b1806b4d0 0x7f5b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:12.999+0000 7f5b33e00640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5b1806b4d0 0x7f5b1806d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0c59d0 tx=0x7f5b24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.002+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b2c07a1d0 con 0x7f5b20003a00 2024-02-26T10:39:13.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.197+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b3c06e4b0 con 0x7f5b20003a00 2024-02-26T10:39:13.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.198+0000 7f5b38c00640 1 -- 172.21.15.67:0/1515975448 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b2c079ad0 con 0x7f5b20003a00 2024-02-26T10:39:13.199 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5b1806b4d0 msgr2=0x7f5b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5b1806b4d0 0x7f5b1806d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0c59d0 tx=0x7f5b24009040 comp rx=0 tx=0).stop 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 msgr2=0x7f5b3c1e5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b3c1e5ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c0e0 tx=0x7f5b3c159090 comp rx=0 tx=0).stop 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 shutdown_connections 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5b1806b4d0 0x7f5b1806d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c0c07b0 0x7f5b3c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b3c14f590 0x7f5b3c1f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 --2- 172.21.15.67:0/1515975448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b20003a00 0x7f5b3c1e5ac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 >> 172.21.15.67:0/1515975448 conn(0x7f5b3c0c6380 msgr2=0x7f5b3c0c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:13.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.200+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 shutdown_connections 2024-02-26T10:39:13.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.201+0000 7f5b41600640 1 -- 172.21.15.67:0/1515975448 wait complete. 2024-02-26T10:39:13.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:13.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.443+0000 7f1d58a00640 1 Processor -- start 2024-02-26T10:39:13.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.443+0000 7f1d58a00640 1 -- start start 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.444+0000 7f1d58a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d541503d0 0x7f1d5406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.444+0000 7f1d58a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d54153af0 0x7f1d5415d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.444+0000 7f1d58a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d540a6eb0 con 0x7f1d5406e390 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.444+0000 7f1d58a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540a7190 con 0x7f1d541503d0 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.444+0000 7f1d58a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540a7470 con 0x7f1d54153af0 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d541503d0 0x7f1d5406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d541503d0 0x7f1d5406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43462/0 (socket says 172.21.15.67:43462) 2024-02-26T10:39:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 -- 172.21.15.67:0/225448817 learned_addr learned my addr 172.21.15.67:0/225448817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d52800640 1 --2- 172.21.15.67:0/225448817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d54153af0 0x7f1d5415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 -- 172.21.15.67:0/225448817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d54153af0 msgr2=0x7f1d5415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 --2- 172.21.15.67:0/225448817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d54153af0 0x7f1d5415d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 -- 172.21.15.67:0/225448817 >> v1:172.21.15.82:6789/0 conn(0x7f1d5406e390 legacy=0x7f1d541533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.445+0000 7f1d51e00640 1 -- 172.21.15.67:0/225448817 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540a7750 con 0x7f1d541503d0 2024-02-26T10:39:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.446+0000 7f1d51e00640 1 --2- 172.21.15.67:0/225448817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d541503d0 0x7f1d5406de50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009870 tx=0x7f1d4402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667777dac0df2602 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.446+0000 7f1d50a00640 1 -- 172.21.15.67:0/225448817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44038070 con 0x7f1d541503d0 2024-02-26T10:39:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d50a00640 1 -- 172.21.15.67:0/225448817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44031740 con 0x7f1d541503d0 2024-02-26T10:39:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d50a00640 1 -- 172.21.15.67:0/225448817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4402f050 con 0x7f1d541503d0 2024-02-26T10:39:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d58a00640 1 -- 172.21.15.67:0/225448817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d541503d0 msgr2=0x7f1d5406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d58a00640 1 --2- 172.21.15.67:0/225448817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d541503d0 0x7f1d5406de50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009870 tx=0x7f1d4402b7e0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d58a00640 1 -- 172.21.15.67:0/225448817 shutdown_connections 2024-02-26T10:39:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d58a00640 1 --2- 172.21.15.67:0/225448817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d54153af0 0x7f1d5415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d58a00640 1 --2- 172.21.15.67:0/225448817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d541503d0 0x7f1d5406de50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.447+0000 7f1d58a00640 1 -- 172.21.15.67:0/225448817 >> 172.21.15.67:0/225448817 conn(0x7f1d540c6da0 msgr2=0x7f1d540c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.448+0000 7f1d58a00640 1 -- 172.21.15.67:0/225448817 shutdown_connections 2024-02-26T10:39:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.448+0000 7f1d58a00640 1 -- 172.21.15.67:0/225448817 wait complete. 2024-02-26T10:39:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.449+0000 7f1d58a00640 1 Processor -- start 2024-02-26T10:39:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.449+0000 7f1d58a00640 1 -- start start 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.450+0000 7f1d58a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d5406e390 0x7f1d541e9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.450+0000 7f1d58a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d541503d0 0x7f1d540ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.450+0000 7f1d58a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 0x7f1d54151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.450+0000 7f1d58a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d541507b0 con 0x7f1d54153af0 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.450+0000 7f1d58a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5406ef00 con 0x7f1d541503d0 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.450+0000 7f1d58a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540c0ef0 con 0x7f1d5406e390 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 0x7f1d54151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 0x7f1d54151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43472/0 (socket says 172.21.15.67:43472) 2024-02-26T10:39:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 -- 172.21.15.67:0/3690529749 learned_addr learned my addr 172.21.15.67:0/3690529749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d51e00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d5406e390 0x7f1d541e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d51400640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d541503d0 0x7f1d540ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 -- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d5406e390 msgr2=0x7f1d541e9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d5406e390 0x7f1d541e9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 -- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d541503d0 msgr2=0x7f1d540ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d541503d0 0x7f1d540ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.451+0000 7f1d52800640 1 -- 172.21.15.67:0/3690529749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54154870 con 0x7f1d54153af0 2024-02-26T10:39:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.452+0000 7f1d52800640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 0x7f1d54151e30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00cc80 tx=0x7f1d4c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.452+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ce30 con 0x7f1d54153af0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.452+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c003990 con 0x7f1d54153af0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.452+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d540c2200 con 0x7f1d54153af0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.452+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c003de0 con 0x7f1d54153af0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.452+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d54154df0 con 0x7f1d54153af0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.453+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54074a30 con 0x7f1d54153af0 2024-02-26T10:39:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.454+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d4c003f80 con 0x7f1d54153af0 2024-02-26T10:39:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.455+0000 7f1d4a000640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d3406b4d0 0x7f1d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.455+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d4c08a5f0 con 0x7f1d54153af0 2024-02-26T10:39:13.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.455+0000 7f1d51e00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d3406b4d0 0x7f1d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.457+0000 7f1d51e00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d3406b4d0 0x7f1d3406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d44035000 tx=0x7f1d44008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.460+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d4c014030 con 0x7f1d54153af0 2024-02-26T10:39:13.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.658+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d5406ec50 con 0x7f1d54153af0 2024-02-26T10:39:13.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.659+0000 7f1d4a000640 1 -- 172.21.15.67:0/3690529749 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d4c059500 con 0x7f1d54153af0 2024-02-26T10:39:13.661 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:13.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d3406b4d0 msgr2=0x7f1d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d3406b4d0 0x7f1d3406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d44035000 tx=0x7f1d44008030 comp rx=0 tx=0).stop 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 msgr2=0x7f1d54151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 0x7f1d54151e30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00cc80 tx=0x7f1d4c00a5c0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 shutdown_connections 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d54153af0 0x7f1d54151e30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d3406b4d0 0x7f1d3406d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d541503d0 0x7f1d540ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 --2- 172.21.15.67:0/3690529749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d5406e390 0x7f1d541e9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 >> 172.21.15.67:0/3690529749 conn(0x7f1d540c6da0 msgr2=0x7f1d5414e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 shutdown_connections 2024-02-26T10:39:13.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.662+0000 7f1d58a00640 1 -- 172.21.15.67:0/3690529749 wait complete. 2024-02-26T10:39:13.692 INFO:tasks.cephfs_test_runner:Starting test: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2024-02-26T10:39:13.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:13.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.913+0000 7f7939800640 1 Processor -- start 2024-02-26T10:39:13.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.914+0000 7f7939800640 1 -- start start 2024-02-26T10:39:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.915+0000 7f7939800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 0x7f79340c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.915+0000 7f7939800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7934152760 con 0x7f79340c7020 2024-02-26T10:39:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.915+0000 7f7939800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793406d9a0 con 0x7f7934152380 2024-02-26T10:39:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.915+0000 7f7939800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340c7550 con 0x7f79340c8510 2024-02-26T10:39:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.915+0000 7f7933400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f79340c7020 0x7f793406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39668/0 (socket says 172.21.15.67:39668) 2024-02-26T10:39:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.915+0000 7f7933400640 1 -- 172.21.15.67:0/712106436 learned_addr learned my addr 172.21.15.67:0/712106436 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:13.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7938e00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 0x7f79340c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548751918 0 0) 0x7f7934152760 con 0x7f79340c7020 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7914003520 con 0x7f79340c7020 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321501563 0 0) 0x7f7914003520 con 0x7f79340c7020 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 msgr2=0x7f79340c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 0x7f79340c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 >> v1:172.21.15.82:6789/0 conn(0x7f7934152380 legacy=0x7f7934155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340c7830 con 0x7f79340c7020 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f791c0019f0 con 0x7f79340c7020 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 >> v1:172.21.15.67:6789/0 conn(0x7f79340c7020 legacy=0x7f793406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7914003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 0x7f7914004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914004d40 0x7f79140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340c7550 con 0x7f79140039e0 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791400baf0 con 0x7f7914004d40 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.916+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791400bcf0 con 0x7f79340c8510 2024-02-26T10:39:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7933e00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7914003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7938e00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914004d40 0x7f79140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7933e00640 1 -- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 msgr2=0x7f7914004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7933e00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c8510 0x7f7914004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7933e00640 1 -- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914004d40 msgr2=0x7f79140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7933e00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7914004d40 0x7f79140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.917+0000 7f7933e00640 1 -- 172.21.15.67:0/712106436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7914003520 con 0x7f79140039e0 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.918+0000 7f7933e00640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7914003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7924001750 tx=0x7f7914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e49cebb061fd3c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.918+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924001370 con 0x7f79140039e0 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.918+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240052d0 con 0x7f79140039e0 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.918+0000 7f7932a00640 1 -- 172.21.15.67:0/712106436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924003830 con 0x7f79140039e0 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7939800640 1 -- 172.21.15.67:0/712106436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 msgr2=0x7f7914003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7939800640 1 --2- 172.21.15.67:0/712106436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7914003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7924001750 tx=0x7f7914005830 comp rx=0 tx=0).stop 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7938e00640 1 -- 172.21.15.67:0/712106436 reap_dead start 2024-02-26T10:39:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7939800640 1 -- 172.21.15.67:0/712106436 shutdown_connections 2024-02-26T10:39:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7939800640 1 -- 172.21.15.67:0/712106436 >> 172.21.15.67:0/712106436 conn(0x7f79340ac1a0 msgr2=0x7f79340ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7939800640 1 -- 172.21.15.67:0/712106436 shutdown_connections 2024-02-26T10:39:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.919+0000 7f7939800640 1 -- 172.21.15.67:0/712106436 wait complete. 2024-02-26T10:39:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.921+0000 7f7939800640 1 Processor -- start 2024-02-26T10:39:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.921+0000 7f7939800640 1 -- start start 2024-02-26T10:39:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7939800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7934147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7939800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c42b0 0x7f79340ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7939800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79340ab5a0 0x7f79340be470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7939800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340c4f90 con 0x7f79140039e0 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7939800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793406cd70 con 0x7f79340ab5a0 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7939800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79340c5110 con 0x7f79340c42b0 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7934147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7934147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43478/0 (socket says 172.21.15.67:43478) 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 -- 172.21.15.67:0/3220637415 learned_addr learned my addr 172.21.15.67:0/3220637415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 -- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c42b0 msgr2=0x7f79340ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7938e00640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79340ab5a0 0x7f79340be470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933400640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c42b0 0x7f79340ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c42b0 0x7f79340ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 -- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79340ab5a0 msgr2=0x7f79340be470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79340ab5a0 0x7f79340be470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.922+0000 7f7933e00640 1 -- 172.21.15.67:0/3220637415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340ae080 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.923+0000 7f7933e00640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7934147db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7924008fd0 tx=0x7f7924001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.923+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924041070 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.923+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924005480 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.923+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924001370 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.923+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79340c0cd0 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.923+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79340adca0 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.924+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7934074a30 con 0x7f79140039e0 2024-02-26T10:39:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.925+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7924044070 con 0x7f79140039e0 2024-02-26T10:39:13.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.926+0000 7f792be00640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f791006b500 0x7f791006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:13.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.926+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79240abe20 con 0x7f79140039e0 2024-02-26T10:39:13.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.927+0000 7f7933400640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f791006b500 0x7f791006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:13.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.928+0000 7f7933400640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f791006b500 0x7f791006d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f79340c3ca0 tx=0x7f791c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:13.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:13.930+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f792407adb0 con 0x7f79140039e0 2024-02-26T10:39:14.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.127+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79340c7c50 con 0x7f79140039e0 2024-02-26T10:39:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.129+0000 7f792be00640 1 -- 172.21.15.67:0/3220637415 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f792407a6b0 con 0x7f79140039e0 2024-02-26T10:39:14.132 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:14.132 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f791006b500 msgr2=0x7f791006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f791006b500 0x7f791006d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f79340c3ca0 tx=0x7f791c009040 comp rx=0 tx=0).stop 2024-02-26T10:39:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 msgr2=0x7f7934147db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7934147db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7924008fd0 tx=0x7f7924001120 comp rx=0 tx=0).stop 2024-02-26T10:39:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 shutdown_connections 2024-02-26T10:39:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f791006b500 0x7f791006d9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79340ab5a0 0x7f79340be470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79340c42b0 0x7f79340ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 --2- 172.21.15.67:0/3220637415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79140039e0 0x7f7934147db0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.132+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 >> 172.21.15.67:0/3220637415 conn(0x7f79340ac1a0 msgr2=0x7f793414d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.133+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 shutdown_connections 2024-02-26T10:39:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.133+0000 7f7939800640 1 -- 172.21.15.67:0/3220637415 wait complete. 2024-02-26T10:39:14.156 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:14.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:14.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.379+0000 7f5f4c600640 1 Processor -- start 2024-02-26T10:39:14.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.379+0000 7f5f4c600640 1 -- start start 2024-02-26T10:39:14.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f4c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f441533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f4c600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440a6eb0 con 0x7f5f441503d0 2024-02-26T10:39:14.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f4c600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440a7190 con 0x7f5f44153920 2024-02-26T10:39:14.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f4c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440a7470 con 0x7f5f440c2950 2024-02-26T10:39:14.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f49c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5f441503d0 0x7f5f4406de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39672/0 (socket says 172.21.15.67:39672) 2024-02-26T10:39:14.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f49c00640 1 -- 172.21.15.67:0/2228225255 learned_addr learned my addr 172.21.15.67:0/2228225255 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:14.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.381+0000 7f5f49200640 1 --2- 172.21.15.67:0/2228225255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f441533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384867810 0 0) 0x7f5f440a6eb0 con 0x7f5f441503d0 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f2c003520 con 0x7f5f441503d0 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257316247 0 0) 0x7f5f440a7190 con 0x7f5f44153920 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f440a6eb0 con 0x7f5f44153920 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3059491872 0 0) 0x7f5f2c003520 con 0x7f5f441503d0 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f49200640 1 -- 172.21.15.67:0/2228225255 >> v1:172.21.15.82:6789/0 conn(0x7f5f44153920 legacy=0x7f5f4415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f49200640 1 -- 172.21.15.67:0/2228225255 >> v1:172.21.15.67:6789/0 conn(0x7f5f441503d0 legacy=0x7f5f4406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.382+0000 7f5f49200640 1 -- 172.21.15.67:0/2228225255 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440a7750 con 0x7f5f440c2950 2024-02-26T10:39:14.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.384+0000 7f5f49200640 1 --2- 172.21.15.67:0/2228225255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f441533e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f300097d0 tx=0x7f5f3002b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f65c6a3d8117fd1e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:14.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.384+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30037070 con 0x7f5f440c2950 2024-02-26T10:39:14.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.384+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30030510 con 0x7f5f440c2950 2024-02-26T10:39:14.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.384+0000 7f5f43e00640 1 -- 172.21.15.67:0/2228225255 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38001670 con 0x7f5f440c2950 2024-02-26T10:39:14.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 -- 172.21.15.67:0/2228225255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 msgr2=0x7f5f441533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2228225255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f441533e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f300097d0 tx=0x7f5f3002b6e0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 -- 172.21.15.67:0/2228225255 shutdown_connections 2024-02-26T10:39:14.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2228225255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f441533e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 -- 172.21.15.67:0/2228225255 >> 172.21.15.67:0/2228225255 conn(0x7f5f440c6da0 msgr2=0x7f5f440c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:14.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 -- 172.21.15.67:0/2228225255 shutdown_connections 2024-02-26T10:39:14.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.385+0000 7f5f4c600640 1 -- 172.21.15.67:0/2228225255 wait complete. 2024-02-26T10:39:14.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.387+0000 7f5f4c600640 1 Processor -- start 2024-02-26T10:39:14.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.387+0000 7f5f4c600640 1 -- start start 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f44149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f441503d0 0x7f5f44151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 0x7f5f44072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4c600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441507b0 con 0x7f5f44153920 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4406ef00 con 0x7f5f441503d0 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4406e560 con 0x7f5f440c2950 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 0x7f5f44072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 0x7f5f44072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43494/0 (socket says 172.21.15.67:43494) 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 -- 172.21.15.67:0/2611714711 learned_addr learned my addr 172.21.15.67:0/2611714711 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f49c00640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f44149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f49200640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f441503d0 0x7f5f44151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 -- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 msgr2=0x7f5f44149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f44149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 -- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f441503d0 msgr2=0x7f5f44151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f441503d0 0x7f5f44151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.388+0000 7f5f4a600640 1 -- 172.21.15.67:0/2611714711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f441516d0 con 0x7f5f44153920 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f49200640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f441503d0 0x7f5f44151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f4a600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 0x7f5f44072190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00c710 tx=0x7f5f3c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c003b70 con 0x7f5f44153920 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c003d10 con 0x7f5f44153920 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f440ac540 con 0x7f5f44153920 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00aa50 con 0x7f5f44153920 2024-02-26T10:39:14.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.389+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f440726d0 con 0x7f5f44153920 2024-02-26T10:39:14.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.391+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f44074a30 con 0x7f5f44153920 2024-02-26T10:39:14.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.392+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f3c00abf0 con 0x7f5f44153920 2024-02-26T10:39:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.393+0000 7f5f41600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f2406b500 0x7f5f2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.393+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f3c089ff0 con 0x7f5f44153920 2024-02-26T10:39:14.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.394+0000 7f5f49c00640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f2406b500 0x7f5f2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.395+0000 7f5f49c00640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f2406b500 0x7f5f2406d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5f380011e0 tx=0x7f5f38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:14.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.398+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f3c00ce20 con 0x7f5f44153920 2024-02-26T10:39:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.594+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f4406ec50 con 0x7f5f44153920 2024-02-26T10:39:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.594+0000 7f5f41600640 1 -- 172.21.15.67:0/2611714711 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f3c058f00 con 0x7f5f44153920 2024-02-26T10:39:14.596 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f2406b500 msgr2=0x7f5f2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f2406b500 0x7f5f2406d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5f380011e0 tx=0x7f5f38009040 comp rx=0 tx=0).stop 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 msgr2=0x7f5f44072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 0x7f5f44072190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00c710 tx=0x7f5f3c00cbd0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 shutdown_connections 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f44153920 0x7f5f44072190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f2406b500 0x7f5f2406d9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f441503d0 0x7f5f44151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 --2- 172.21.15.67:0/2611714711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f440c2950 0x7f5f44149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 >> 172.21.15.67:0/2611714711 conn(0x7f5f440c6da0 msgr2=0x7f5f4414e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 shutdown_connections 2024-02-26T10:39:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.597+0000 7f5f4c600640 1 -- 172.21.15.67:0/2611714711 wait complete. 2024-02-26T10:39:14.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.831+0000 7fbc48000640 1 Processor -- start 2024-02-26T10:39:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.831+0000 7fbc48000640 1 -- start start 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc48000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc400a6150 0x7fbc400c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc48000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc40155880 0x7fbc400a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc48000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc40151d70 con 0x7fbc4006e860 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc48000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc40152b10 con 0x7fbc40155880 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc48000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc40152680 con 0x7fbc400a6150 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc46000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc40155880 0x7fbc400a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc46000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc40155880 0x7fbc400a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43498/0 (socket says 172.21.15.67:43498) 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.832+0000 7fbc46000640 1 -- 172.21.15.67:0/3951899141 learned_addr learned my addr 172.21.15.67:0/3951899141 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.833+0000 7fbc45600640 1 --2- 172.21.15.67:0/3951899141 >> v2:172.21.15.82:3300/0 conn(0x7fbc400a6150 0x7fbc400c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.833+0000 7fbc46000640 1 -- 172.21.15.67:0/3951899141 >> v2:172.21.15.82:3300/0 conn(0x7fbc400a6150 msgr2=0x7fbc400c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.833+0000 7fbc46000640 1 --2- 172.21.15.67:0/3951899141 >> v2:172.21.15.82:3300/0 conn(0x7fbc400a6150 0x7fbc400c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.833+0000 7fbc46000640 1 -- 172.21.15.67:0/3951899141 >> v1:172.21.15.67:6789/0 conn(0x7fbc4006e860 legacy=0x7fbc40155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.833+0000 7fbc46000640 1 -- 172.21.15.67:0/3951899141 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40152030 con 0x7fbc40155880 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.834+0000 7fbc46000640 1 --2- 172.21.15.67:0/3951899141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc40155880 0x7fbc400a6530 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbc4006ef20 tx=0x7fbc3802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a15918073d6cf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.834+0000 7fbc3fe00640 1 -- 172.21.15.67:0/3951899141 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3803b070 con 0x7fbc40155880 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.834+0000 7fbc3fe00640 1 -- 172.21.15.67:0/3951899141 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc38036440 con 0x7fbc40155880 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.834+0000 7fbc3fe00640 1 -- 172.21.15.67:0/3951899141 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38036810 con 0x7fbc40155880 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 -- 172.21.15.67:0/3951899141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc40155880 msgr2=0x7fbc400a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 --2- 172.21.15.67:0/3951899141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc40155880 0x7fbc400a6530 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbc4006ef20 tx=0x7fbc3802e0f0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 -- 172.21.15.67:0/3951899141 shutdown_connections 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 --2- 172.21.15.67:0/3951899141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc40155880 0x7fbc400a6530 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 --2- 172.21.15.67:0/3951899141 >> v2:172.21.15.82:3300/0 conn(0x7fbc400a6150 0x7fbc400c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 -- 172.21.15.67:0/3951899141 >> 172.21.15.67:0/3951899141 conn(0x7fbc400c6da0 msgr2=0x7fbc400c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 -- 172.21.15.67:0/3951899141 shutdown_connections 2024-02-26T10:39:14.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.835+0000 7fbc48000640 1 -- 172.21.15.67:0/3951899141 wait complete. 2024-02-26T10:39:14.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.837+0000 7fbc48000640 1 Processor -- start 2024-02-26T10:39:14.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.837+0000 7fbc48000640 1 -- start start 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc48000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc4006e860 0x7fbc401eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc48000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 0x7fbc4014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc48000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc40155880 0x7fbc400bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc48000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400c2220 con 0x7fbc400a6150 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc48000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40155cf0 con 0x7fbc4006e860 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc48000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4014f3a0 con 0x7fbc40155880 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc44c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 0x7fbc4014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc44c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 0x7fbc4014b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43502/0 (socket says 172.21.15.67:43502) 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.838+0000 7fbc44c00640 1 -- 172.21.15.67:0/3115492137 learned_addr learned my addr 172.21.15.67:0/3115492137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc45600640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc4006e860 0x7fbc401eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc46000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc40155880 0x7fbc400bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc44c00640 1 -- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc40155880 msgr2=0x7fbc400bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc44c00640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc40155880 0x7fbc400bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc44c00640 1 -- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc4006e860 msgr2=0x7fbc401eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc44c00640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc4006e860 0x7fbc401eb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc44c00640 1 -- 172.21.15.67:0/3115492137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc401ee980 con 0x7fbc400a6150 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc46000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc40155880 0x7fbc400bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc45600640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc4006e860 0x7fbc401eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc44c00640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 0x7fbc4014b010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c430 tx=0x7fbc2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0052d0 con 0x7fbc400a6150 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c005490 con 0x7fbc400a6150 2024-02-26T10:39:14.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c007260 con 0x7fbc400a6150 2024-02-26T10:39:14.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.839+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc401ea9e0 con 0x7fbc400a6150 2024-02-26T10:39:14.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.840+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc400c00d0 con 0x7fbc400a6150 2024-02-26T10:39:14.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.841+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40074a30 con 0x7fbc400a6150 2024-02-26T10:39:14.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.842+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc2c007400 con 0x7fbc400a6150 2024-02-26T10:39:14.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.842+0000 7fbc3d600640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc2006b4d0 0x7fbc2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:14.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.843+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc2c089700 con 0x7fbc400a6150 2024-02-26T10:39:14.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.843+0000 7fbc45600640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc2006b4d0 0x7fbc2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:14.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.844+0000 7fbc45600640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc2006b4d0 0x7fbc2006d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbc34007330 tx=0x7fbc3400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:14.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:14.847+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc2c058570 con 0x7fbc400a6150 2024-02-26T10:39:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.042+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc40073280 con 0x7fbc400a6150 2024-02-26T10:39:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.043+0000 7fbc3d600640 1 -- 172.21.15.67:0/3115492137 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc2c057e70 con 0x7fbc400a6150 2024-02-26T10:39:15.044 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.045+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc2006b4d0 msgr2=0x7fbc2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.045+0000 7fbc48000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc2006b4d0 0x7fbc2006d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbc34007330 tx=0x7fbc3400b040 comp rx=0 tx=0).stop 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.045+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 msgr2=0x7fbc4014b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.045+0000 7fbc48000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 0x7fbc4014b010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c430 tx=0x7fbc2c00c8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 shutdown_connections 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc40155880 0x7fbc400bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc400a6150 0x7fbc4014b010 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc2006b4d0 0x7fbc2006d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 --2- 172.21.15.67:0/3115492137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc4006e860 0x7fbc401eb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 >> 172.21.15.67:0/3115492137 conn(0x7fbc400c6da0 msgr2=0x7fbc4006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 shutdown_connections 2024-02-26T10:39:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.046+0000 7fbc48000640 1 -- 172.21.15.67:0/3115492137 wait complete. 2024-02-26T10:39:15.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:15.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.282+0000 7f8fad000640 1 Processor -- start 2024-02-26T10:39:15.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.282+0000 7f8fad000640 1 -- start start 2024-02-26T10:39:15.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.283+0000 7f8fad000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8fa8152710 0x7f8fa80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.283+0000 7f8fad000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c8780 0x7f8fa8158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.283+0000 7f8fad000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa80c5aa0 con 0x7f8fa80c77d0 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.283+0000 7f8fad000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa8155b40 con 0x7f8fa8152710 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.283+0000 7f8fad000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80c52a0 con 0x7f8fa80c8780 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8fa8152710 0x7f8fa80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8fa8152710 0x7f8fa80c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43512/0 (socket says 172.21.15.67:43512) 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 -- 172.21.15.67:0/315405873 learned_addr learned my addr 172.21.15.67:0/315405873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 -- 172.21.15.67:0/315405873 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c8780 msgr2=0x7f8fa8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 --2- 172.21.15.67:0/315405873 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c8780 0x7f8fa8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 -- 172.21.15.67:0/315405873 >> v1:172.21.15.82:6789/0 conn(0x7f8fa80c77d0 legacy=0x7f8fa8158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:15.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.284+0000 7f8fa5e00640 1 -- 172.21.15.67:0/315405873 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8155e20 con 0x7f8fa8152710 2024-02-26T10:39:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.285+0000 7f8fa5e00640 1 --2- 172.21.15.67:0/315405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa8152710 0x7f8fa80c7290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e80 tx=0x7f8f9802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f11d8e0dd2f1ba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fa5400640 1 -- 172.21.15.67:0/315405873 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802c9c0 con 0x7f8fa8152710 2024-02-26T10:39:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fa5400640 1 -- 172.21.15.67:0/315405873 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9802cb60 con 0x7f8fa8152710 2024-02-26T10:39:15.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fa5400640 1 -- 172.21.15.67:0/315405873 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f980343f0 con 0x7f8fa8152710 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 -- 172.21.15.67:0/315405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa8152710 msgr2=0x7f8fa80c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 --2- 172.21.15.67:0/315405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa8152710 0x7f8fa80c7290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e80 tx=0x7f8f9802be20 comp rx=0 tx=0).stop 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 -- 172.21.15.67:0/315405873 shutdown_connections 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 --2- 172.21.15.67:0/315405873 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c8780 0x7f8fa8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 --2- 172.21.15.67:0/315405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa8152710 0x7f8fa80c7290 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 -- 172.21.15.67:0/315405873 >> 172.21.15.67:0/315405873 conn(0x7f8fa80ac410 msgr2=0x7f8fa80ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 -- 172.21.15.67:0/315405873 shutdown_connections 2024-02-26T10:39:15.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.286+0000 7f8fad000640 1 -- 172.21.15.67:0/315405873 wait complete. 2024-02-26T10:39:15.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.288+0000 7f8fad000640 1 Processor -- start 2024-02-26T10:39:15.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.288+0000 7f8fad000640 1 -- start start 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8fad000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c77d0 0x7f8fa80c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8fad000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 0x7f8fa8147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8fad000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa8152710 0x7f8fa8148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8fad000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa806dba0 con 0x7f8fa80c8780 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8fad000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80bdf10 con 0x7f8fa8152710 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8fad000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa806dd20 con 0x7f8fa80c77d0 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8f9d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 0x7f8fa8147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8f9d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 0x7f8fa8147af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43524/0 (socket says 172.21.15.67:43524) 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.289+0000 7f8f9d400640 1 -- 172.21.15.67:0/1673233551 learned_addr learned my addr 172.21.15.67:0/1673233551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8fa6800640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa8152710 0x7f8fa8148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8fa5e00640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c77d0 0x7f8fa80c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9d400640 1 -- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c77d0 msgr2=0x7f8fa80c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9d400640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c77d0 0x7f8fa80c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9d400640 1 -- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa8152710 msgr2=0x7f8fa8148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9d400640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa8152710 0x7f8fa8148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9d400640 1 -- 172.21.15.67:0/1673233551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa81eabd0 con 0x7f8fa80c8780 2024-02-26T10:39:15.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8fa6800640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa8152710 0x7f8fa8148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9d400640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 0x7f8fa8147af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000b480 tx=0x7f8f9000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90014070 con 0x7f8fa80c8780 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90010870 con 0x7f8fa80c8780 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa81eae90 con 0x7f8fa80c8780 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.290+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90010c60 con 0x7f8fa80c8780 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.291+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa81eb450 con 0x7f8fa80c8780 2024-02-26T10:39:15.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.292+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa80678e0 con 0x7f8fa80c8780 2024-02-26T10:39:15.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.293+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f90017070 con 0x7f8fa80c8780 2024-02-26T10:39:15.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.293+0000 7f8f9ea00640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.294+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f90089fe0 con 0x7f8fa80c8780 2024-02-26T10:39:15.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.294+0000 7f8fa5e00640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.295+0000 7f8fa5e00640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f8406b4d0 0x7f8f8406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f98035000 tx=0x7f8f98030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:15.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.298+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f90058e50 con 0x7f8fa80c8780 2024-02-26T10:39:15.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.504+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fa8074a30 con 0x7f8fa80c8780 2024-02-26T10:39:15.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.504+0000 7f8f9ea00640 1 -- 172.21.15.67:0/1673233551 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f90058750 con 0x7f8fa80c8780 2024-02-26T10:39:15.505 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f8406b4d0 msgr2=0x7f8f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f8406b4d0 0x7f8f8406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f98035000 tx=0x7f8f98030000 comp rx=0 tx=0).stop 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 msgr2=0x7f8fa8147af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 0x7f8fa8147af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000b480 tx=0x7f8f9000b940 comp rx=0 tx=0).stop 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 shutdown_connections 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa8152710 0x7f8fa8148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fa80c8780 0x7f8fa8147af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 --2- 172.21.15.67:0/1673233551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fa80c77d0 0x7f8fa80c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.506+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 >> 172.21.15.67:0/1673233551 conn(0x7f8fa80ac410 msgr2=0x7f8fa80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:15.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.508+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 shutdown_connections 2024-02-26T10:39:15.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.508+0000 7f8fad000640 1 -- 172.21.15.67:0/1673233551 wait complete. 2024-02-26T10:39:15.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent' 2024-02-26T10:39:15.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.743+0000 7f05f4a00640 1 Processor -- start 2024-02-26T10:39:15.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.744+0000 7f05f4a00640 1 -- start start 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05f4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f05f00c8510 0x7f05f00c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05f4a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f0152760 con 0x7f05f0152380 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05f4a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f006d9a0 con 0x7f05f00c7020 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05f4a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f00c7550 con 0x7f05f00c8510 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05ef400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f05f0152380 0x7f05f0155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39688/0 (socket says 172.21.15.67:39688) 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05ef400640 1 -- 172.21.15.67:0/2349041548 learned_addr learned my addr 172.21.15.67:0/2349041548 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:15.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.745+0000 7f05efe00640 1 --2- 172.21.15.67:0/2349041548 >> v2:172.21.15.67:3300/0 conn(0x7f05f00c8510 0x7f05f00c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.746+0000 7f05eea00640 1 -- 172.21.15.67:0/2349041548 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381720449 0 0) 0x7f05f0152760 con 0x7f05f0152380 2024-02-26T10:39:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.746+0000 7f05eea00640 1 -- 172.21.15.67:0/2349041548 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d4003520 con 0x7f05f0152380 2024-02-26T10:39:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.746+0000 7f05efe00640 1 -- 172.21.15.67:0/2349041548 >> v1:172.21.15.82:6789/0 conn(0x7f05f00c7020 legacy=0x7f05f006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.746+0000 7f05efe00640 1 -- 172.21.15.67:0/2349041548 >> v1:172.21.15.67:6789/0 conn(0x7f05f0152380 legacy=0x7f05f0155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.746+0000 7f05efe00640 1 -- 172.21.15.67:0/2349041548 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00c7830 con 0x7f05f00c8510 2024-02-26T10:39:15.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.747+0000 7f05efe00640 1 --2- 172.21.15.67:0/2349041548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f00c4f90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05e80097a0 tx=0x7f05e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0b481fab2fb32 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:15.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.747+0000 7f05eea00640 1 -- 172.21.15.67:0/2349041548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e80315a0 con 0x7f05f00c8510 2024-02-26T10:39:15.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05eea00640 1 -- 172.21.15.67:0/2349041548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8037070 con 0x7f05f00c8510 2024-02-26T10:39:15.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05eea00640 1 -- 172.21.15.67:0/2349041548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8031960 con 0x7f05f00c8510 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 -- 172.21.15.67:0/2349041548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 msgr2=0x7f05f00c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2349041548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f00c4f90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05e80097a0 tx=0x7f05e802b840 comp rx=0 tx=0).stop 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 -- 172.21.15.67:0/2349041548 shutdown_connections 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2349041548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f00c4f90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 -- 172.21.15.67:0/2349041548 >> 172.21.15.67:0/2349041548 conn(0x7f05f00ac1a0 msgr2=0x7f05f00ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 -- 172.21.15.67:0/2349041548 shutdown_connections 2024-02-26T10:39:15.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.748+0000 7f05f4a00640 1 -- 172.21.15.67:0/2349041548 wait complete. 2024-02-26T10:39:15.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.750+0000 7f05f4a00640 1 Processor -- start 2024-02-26T10:39:15.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.750+0000 7f05f4a00640 1 -- start start 2024-02-26T10:39:15.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05f4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f05f00c7020 0x7f05f00c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05f4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f014b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05f4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05f0152380 0x7f05f01edac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05f4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c54d0 con 0x7f05f00c8510 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05f4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f006cd70 con 0x7f05f0152380 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05f4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c5650 con 0x7f05f00c7020 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f014b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f014b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43554/0 (socket says 172.21.15.67:43554) 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.751+0000 7f05e6a00640 1 -- 172.21.15.67:0/2989253819 learned_addr learned my addr 172.21.15.67:0/2989253819 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05ef400640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f05f00c7020 0x7f05f00c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05efe00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05f0152380 0x7f05f01edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e6a00640 1 -- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f05f00c7020 msgr2=0x7f05f00c0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e6a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f05f00c7020 0x7f05f00c0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e6a00640 1 -- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05f0152380 msgr2=0x7f05f01edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e6a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05f0152380 0x7f05f01edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e6a00640 1 -- 172.21.15.67:0/2989253819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f01582b0 con 0x7f05f00c8510 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05efe00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05f0152380 0x7f05f01edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:15.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e6a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f014b600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c430 tx=0x7f05d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8011790 con 0x7f05f00c8510 2024-02-26T10:39:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8017070 con 0x7f05f00c8510 2024-02-26T10:39:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f01ee3e0 con 0x7f05f00c8510 2024-02-26T10:39:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.752+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8011bb0 con 0x7f05f00c8510 2024-02-26T10:39:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.753+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f01ee6c0 con 0x7f05f00c8510 2024-02-26T10:39:15.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.754+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f0074a30 con 0x7f05f00c8510 2024-02-26T10:39:15.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.754+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05d801c460 con 0x7f05f00c8510 2024-02-26T10:39:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.756+0000 7f05e7e00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05cc06b500 0x7f05cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:15.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.756+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d808a0a0 con 0x7f05f00c8510 2024-02-26T10:39:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.756+0000 7f05ef400640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05cc06b500 0x7f05cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:15.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.758+0000 7f05ef400640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05cc06b500 0x7f05cc06d9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005c00 tx=0x7f05e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:15.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:15.761+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05d8058f10 con 0x7f05f00c8510 2024-02-26T10:39:16.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.031+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]} v 0) v1 -- 0x7f05f00c7c50 con 0x7f05f00c8510 2024-02-26T10:39:16.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.421+0000 7f05e7e00640 1 -- 172.21.15.67:0/2989253819 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]}]=0 v57) v1 ==== 162+0+0 (secure 0 0 0) 0x7f05d8058810 con 0x7f05f00c8510 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05cc06b500 msgr2=0x7f05cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05cc06b500 0x7f05cc06d9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005c00 tx=0x7f05e0009040 comp rx=0 tx=0).stop 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 msgr2=0x7f05f014b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f014b600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c430 tx=0x7f05d800c8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 shutdown_connections 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05cc06b500 0x7f05cc06d9c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05f0152380 0x7f05f01edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f00c8510 0x7f05f014b600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 --2- 172.21.15.67:0/2989253819 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f05f00c7020 0x7f05f00c0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 >> 172.21.15.67:0/2989253819 conn(0x7f05f00ac1a0 msgr2=0x7f05f00cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 shutdown_connections 2024-02-26T10:39:16.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.424+0000 7f05f4a00640 1 -- 172.21.15.67:0/2989253819 wait complete. 2024-02-26T10:39:16.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-26T10:39:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.670+0000 7ff4c9800640 1 Processor -- start 2024-02-26T10:39:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.670+0000 7ff4c9800640 1 -- start start 2024-02-26T10:39:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.672+0000 7ff4c9800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4c41523b0 0x7ff4c40c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.672+0000 7ff4c9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4c4071fd0 0x7ff4c40723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.672+0000 7ff4c9800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c40c3840 con 0x7ff4c4154cf0 2024-02-26T10:39:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.672+0000 7ff4c9800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c4152790 con 0x7ff4c4071fd0 2024-02-26T10:39:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.672+0000 7ff4c9800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c406da30 con 0x7ff4c41523b0 2024-02-26T10:39:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4c4071fd0 0x7ff4c40723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff4c4071fd0 0x7ff4c40723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43560/0 (socket says 172.21.15.67:43560) 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 -- 172.21.15.67:0/340840846 learned_addr learned my addr 172.21.15.67:0/340840846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c3e00640 1 --2- 172.21.15.67:0/340840846 >> v2:172.21.15.82:3300/0 conn(0x7ff4c41523b0 0x7ff4c40c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 -- 172.21.15.67:0/340840846 >> v2:172.21.15.82:3300/0 conn(0x7ff4c41523b0 msgr2=0x7ff4c40c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 --2- 172.21.15.67:0/340840846 >> v2:172.21.15.82:3300/0 conn(0x7ff4c41523b0 0x7ff4c40c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 -- 172.21.15.67:0/340840846 >> v1:172.21.15.82:6789/0 conn(0x7ff4c4154cf0 legacy=0x7ff4c40c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.673+0000 7ff4c8e00640 1 -- 172.21.15.67:0/340840846 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c406dc80 con 0x7ff4c4071fd0 2024-02-26T10:39:16.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.675+0000 7ff4c8e00640 1 --2- 172.21.15.67:0/340840846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40723e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4c406f310 tx=0x7ff4bc02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730d95ceb597f919 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:16.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.675+0000 7ff4c2a00640 1 -- 172.21.15.67:0/340840846 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc02c960 con 0x7ff4c4071fd0 2024-02-26T10:39:16.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.675+0000 7ff4c2a00640 1 -- 172.21.15.67:0/340840846 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc02cb00 con 0x7ff4c4071fd0 2024-02-26T10:39:16.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.675+0000 7ff4c2a00640 1 -- 172.21.15.67:0/340840846 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0343f0 con 0x7ff4c4071fd0 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 -- 172.21.15.67:0/340840846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 msgr2=0x7ff4c40723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 --2- 172.21.15.67:0/340840846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40723e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff4c406f310 tx=0x7ff4bc02bdc0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 -- 172.21.15.67:0/340840846 shutdown_connections 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 --2- 172.21.15.67:0/340840846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40723e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 --2- 172.21.15.67:0/340840846 >> v2:172.21.15.82:3300/0 conn(0x7ff4c41523b0 0x7ff4c40c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 -- 172.21.15.67:0/340840846 >> 172.21.15.67:0/340840846 conn(0x7ff4c40c6380 msgr2=0x7ff4c40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 -- 172.21.15.67:0/340840846 shutdown_connections 2024-02-26T10:39:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.676+0000 7ff4c9800640 1 -- 172.21.15.67:0/340840846 wait complete. 2024-02-26T10:39:16.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.678+0000 7ff4c9800640 1 Processor -- start 2024-02-26T10:39:16.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.679+0000 7ff4c9800640 1 -- start start 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4c41523b0 0x7ff4c41ea620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4c4154cf0 0x7ff4c41eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c406e520 con 0x7ff4c4071fd0 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c406e240 con 0x7ff4c41523b0 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c406e800 con 0x7ff4c4154cf0 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43572/0 (socket says 172.21.15.67:43572) 2024-02-26T10:39:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c3e00640 1 -- 172.21.15.67:0/1557839978 learned_addr learned my addr 172.21.15.67:0/1557839978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.680+0000 7ff4c3400640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4c41523b0 0x7ff4c41ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c8e00640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4c4154cf0 0x7ff4c41eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c3e00640 1 -- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4c4154cf0 msgr2=0x7ff4c41eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c3e00640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4c4154cf0 0x7ff4c41eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c3e00640 1 -- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4c41523b0 msgr2=0x7ff4c41ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c3e00640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4c41523b0 0x7ff4c41ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c3e00640 1 -- 172.21.15.67:0/1557839978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4156780 con 0x7ff4c4071fd0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c3e00640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40c16d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400d800 tx=0x7ff4b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400b6c0 con 0x7ff4c4071fd0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b4019070 con 0x7ff4c4071fd0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400ba80 con 0x7ff4c4071fd0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c41eb0a0 con 0x7ff4c4071fd0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.681+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c41ee5a0 con 0x7ff4c4071fd0 2024-02-26T10:39:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.683+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c40678e0 con 0x7ff4c4071fd0 2024-02-26T10:39:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.684+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4b400bc20 con 0x7ff4c4071fd0 2024-02-26T10:39:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.684+0000 7ff4bbe00640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff4a406b4d0 0x7ff4a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.685+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4b408a6c0 con 0x7ff4c4071fd0 2024-02-26T10:39:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.685+0000 7ff4c3400640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff4a406b4d0 0x7ff4a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.686+0000 7ff4c3400640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff4a406b4d0 0x7ff4a406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0003d90 tx=0x7ff4b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:16.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.689+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4b4059530 con 0x7ff4c4071fd0 2024-02-26T10:39:16.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.888+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff4c4074a30 con 0x7ff4c4071fd0 2024-02-26T10:39:16.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.894+0000 7ff4bbe00640 1 -- 172.21.15.67:0/1557839978 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff4b4058e30 con 0x7ff4c4071fd0 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.896+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff4a406b4d0 msgr2=0x7ff4a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff4a406b4d0 0x7ff4a406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0003d90 tx=0x7ff4b0009040 comp rx=0 tx=0).stop 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 msgr2=0x7ff4c40c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40c16d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400d800 tx=0x7ff4b400dcc0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 shutdown_connections 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4c4154cf0 0x7ff4c41eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff4a406b4d0 0x7ff4a406d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4c41523b0 0x7ff4c41ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 --2- 172.21.15.67:0/1557839978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4c4071fd0 0x7ff4c40c16d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 >> 172.21.15.67:0/1557839978 conn(0x7ff4c40c6380 msgr2=0x7ff4c4152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:16.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 shutdown_connections 2024-02-26T10:39:16.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:16.897+0000 7ff4c9800640 1 -- 172.21.15.67:0/1557839978 wait complete. 2024-02-26T10:39:16.917 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-26T10:39:16.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:39:17.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:39:17.157 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-26T10:39:17.157 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:39:17.157 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:17.218 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:39:17.218 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:17.272 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:39:17.273 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:17.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:17.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.547+0000 7f8eea400640 1 Processor -- start 2024-02-26T10:39:17.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.548+0000 7f8eea400640 1 -- start start 2024-02-26T10:39:17.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.549+0000 7f8eea400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ee406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.549+0000 7f8eea400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ee4159210 0x7f8ee4159680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.549+0000 7f8eea400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee415c8a0 con 0x7f8ee4072fe0 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.549+0000 7f8eea400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee415cb80 con 0x7f8ee4159210 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.549+0000 7f8eea400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee415ce60 con 0x7f8ee4156810 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.550+0000 7f8ee2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8ee4072fe0 0x7f8ee40c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39704/0 (socket says 172.21.15.67:39704) 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.550+0000 7f8ee2a00640 1 -- 172.21.15.67:0/534274918 learned_addr learned my addr 172.21.15.67:0/534274918 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.550+0000 7f8ee3e00640 1 --2- 172.21.15.67:0/534274918 >> v2:172.21.15.82:3300/0 conn(0x7f8ee4159210 0x7f8ee4159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.550+0000 7f8ee3400640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ee406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.550+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938838281 0 0) 0x7f8ee415c8a0 con 0x7f8ee4072fe0 2024-02-26T10:39:17.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.550+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec8003520 con 0x7f8ee4072fe0 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153913684 0 0) 0x7f8ec8003520 con 0x7f8ee4072fe0 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 msgr2=0x7f8ee406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ee406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 >> v2:172.21.15.82:3300/0 conn(0x7f8ee4159210 msgr2=0x7f8ee4159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 --2- 172.21.15.67:0/534274918 >> v2:172.21.15.82:3300/0 conn(0x7f8ee4159210 0x7f8ee4159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee415d140 con 0x7f8ee4072fe0 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8ed0001670 con 0x7f8ee4072fe0 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 >> v1:172.21.15.67:6789/0 conn(0x7f8ee4072fe0 legacy=0x7f8ee40c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee415cb80 con 0x7f8ec80039b0 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee415ce60 con 0x7f8ec8004d40 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec800b9c0 con 0x7f8ee4156810 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.551+0000 7f8ee3400640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee3e00640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee2a00640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee3400640 1 -- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 msgr2=0x7f8ec8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee3400640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ec8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee3400640 1 -- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ec8004d40 msgr2=0x7f8ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee3400640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ec8004d40 0x7f8ec80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee3400640 1 -- 172.21.15.67:0/534274918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee415d140 con 0x7f8ec80039b0 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.552+0000 7f8ee2a00640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee4156810 0x7f8ec8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:17.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.553+0000 7f8ee3400640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009ac0 tx=0x7f8ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f1c48c3d7926bf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:17.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.553+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4039070 con 0x7f8ec80039b0 2024-02-26T10:39:17.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.553+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed402c6b0 con 0x7f8ec80039b0 2024-02-26T10:39:17.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.553+0000 7f8ee2000640 1 -- 172.21.15.67:0/534274918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4009690 con 0x7f8ec80039b0 2024-02-26T10:39:17.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8eea400640 1 -- 172.21.15.67:0/534274918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 msgr2=0x7f8ec8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8eea400640 1 --2- 172.21.15.67:0/534274918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ec8003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009ac0 tx=0x7f8ec8005830 comp rx=0 tx=0).stop 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8ee3e00640 1 -- 172.21.15.67:0/534274918 reap_dead start 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8eea400640 1 -- 172.21.15.67:0/534274918 shutdown_connections 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8eea400640 1 -- 172.21.15.67:0/534274918 >> 172.21.15.67:0/534274918 conn(0x7f8ee40a84a0 msgr2=0x7f8ee40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8eea400640 1 -- 172.21.15.67:0/534274918 shutdown_connections 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.554+0000 7f8eea400640 1 -- 172.21.15.67:0/534274918 wait complete. 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.556+0000 7f8eea400640 1 Processor -- start 2024-02-26T10:39:17.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.556+0000 7f8eea400640 1 -- start start 2024-02-26T10:39:17.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8eea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ee40ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8eea400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee4151490 0x7f8ee4152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8eea400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee41f23c0 0x7f8ee41f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8eea400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee406ec40 con 0x7f8ec80039b0 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8eea400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee415d660 con 0x7f8ee4151490 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8eea400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee4159bc0 con 0x7f8ee41f23c0 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ee40ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ee40ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43588/0 (socket says 172.21.15.67:43588) 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 -- 172.21.15.67:0/3722116960 learned_addr learned my addr 172.21.15.67:0/3722116960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 -- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee41f23c0 msgr2=0x7f8ee41f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee41f23c0 0x7f8ee41f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 -- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee4151490 msgr2=0x7f8ee4152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee2a00640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee4151490 0x7f8ee4152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee4151490 0x7f8ee4152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.557+0000 7f8ee3400640 1 -- 172.21.15.67:0/3722116960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40c2c60 con 0x7f8ec80039b0 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.558+0000 7f8ee3400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ee40ab2b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4000c00 tx=0x7f8ed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:17.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.558+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4042020 con 0x7f8ec80039b0 2024-02-26T10:39:17.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.558+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed403a040 con 0x7f8ec80039b0 2024-02-26T10:39:17.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.558+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee40a96a0 con 0x7f8ec80039b0 2024-02-26T10:39:17.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.558+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4039070 con 0x7f8ec80039b0 2024-02-26T10:39:17.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.558+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee40a9940 con 0x7f8ec80039b0 2024-02-26T10:39:17.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.559+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee4074a30 con 0x7f8ec80039b0 2024-02-26T10:39:17.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.560+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ed4045020 con 0x7f8ec80039b0 2024-02-26T10:39:17.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.561+0000 7f8edb400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b500 0x7f8ec406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:17.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.561+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ed40aafb0 con 0x7f8ec80039b0 2024-02-26T10:39:17.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.562+0000 7f8ee2a00640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b500 0x7f8ec406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:17.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.563+0000 7f8ee2a00640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b500 0x7f8ec406d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0001820 tx=0x7f8ed0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:17.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.566+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ee40c2c60 con 0x7f8ec80039b0 2024-02-26T10:39:17.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.766+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ee406e440 con 0x7f8ec80039b0 2024-02-26T10:39:17.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.768+0000 7f8edb400640 1 -- 172.21.15.67:0/3722116960 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16533 (secure 0 0 0) 0x7f8ed4079ec0 con 0x7f8ec80039b0 2024-02-26T10:39:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:17.769 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":21,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:38:36.160053+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b500 msgr2=0x7f8ec406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b500 0x7f8ec406d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0001820 tx=0x7f8ed0009040 comp rx=0 tx=0).stop 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 msgr2=0x7f8ee40ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ee40ab2b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4000c00 tx=0x7f8ed4004730 comp rx=0 tx=0).stop 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 shutdown_connections 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee41f23c0 0x7f8ee41f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b500 0x7f8ec406d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee4151490 0x7f8ee4152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 --2- 172.21.15.67:0/3722116960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ec80039b0 0x7f8ee40ab2b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 >> 172.21.15.67:0/3722116960 conn(0x7f8ee40a84a0 msgr2=0x7f8ee40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:17.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 shutdown_connections 2024-02-26T10:39:17.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:17.771+0000 7f8eea400640 1 -- 172.21.15.67:0/3722116960 wait complete. 2024-02-26T10:39:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:17.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:18.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.019+0000 7f81c2200640 1 Processor -- start 2024-02-26T10:39:18.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.019+0000 7f81c2200640 1 -- start start 2024-02-26T10:39:18.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.020+0000 7f81c2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.020+0000 7f81c2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81bc1503a0 0x7f81bc152df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81c2200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc0c4260 con 0x7f81bc0c7a30 2024-02-26T10:39:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81c2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0c4030 con 0x7f81bc1503a0 2024-02-26T10:39:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81c2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c4540 con 0x7f81bc070640 2024-02-26T10:39:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81bbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81bc1503a0 0x7f81bc152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81bb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f81bc0c7a30 0x7f81bc071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39710/0 (socket says 172.21.15.67:39710) 2024-02-26T10:39:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81bb400640 1 -- 172.21.15.67:0/4211981710 learned_addr learned my addr 172.21.15.67:0/4211981710 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81baa00640 1 --2- 172.21.15.67:0/4211981710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81bbe00640 1 -- 172.21.15.67:0/4211981710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 msgr2=0x7f81bc14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.021+0000 7f81bbe00640 1 --2- 172.21.15.67:0/4211981710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc14fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.022+0000 7f81bbe00640 1 -- 172.21.15.67:0/4211981710 >> v1:172.21.15.67:6789/0 conn(0x7f81bc0c7a30 legacy=0x7f81bc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.022+0000 7f81bbe00640 1 -- 172.21.15.67:0/4211981710 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0c4820 con 0x7f81bc1503a0 2024-02-26T10:39:18.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.022+0000 7f81ba000640 1 -- 172.21.15.67:0/4211981710 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82848295 0 0) 0x7f81bc0c4260 con 0x7f81bc0c7a30 2024-02-26T10:39:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.023+0000 7f81bbe00640 1 --2- 172.21.15.67:0/4211981710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc152df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009d90 tx=0x7f81b402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcbd03d3c7a6f04 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.023+0000 7f81ba000640 1 -- 172.21.15.67:0/4211981710 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4039070 con 0x7f81bc1503a0 2024-02-26T10:39:18.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.023+0000 7f81ba000640 1 -- 172.21.15.67:0/4211981710 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b402cc50 con 0x7f81bc1503a0 2024-02-26T10:39:18.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.023+0000 7f81ba000640 1 -- 172.21.15.67:0/4211981710 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4034440 con 0x7f81bc1503a0 2024-02-26T10:39:18.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 -- 172.21.15.67:0/4211981710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 msgr2=0x7f81bc152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 --2- 172.21.15.67:0/4211981710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc152df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009d90 tx=0x7f81b402bea0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 -- 172.21.15.67:0/4211981710 shutdown_connections 2024-02-26T10:39:18.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 --2- 172.21.15.67:0/4211981710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc152df0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 --2- 172.21.15.67:0/4211981710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 -- 172.21.15.67:0/4211981710 >> 172.21.15.67:0/4211981710 conn(0x7f81bc06fa10 msgr2=0x7f81bc06ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 -- 172.21.15.67:0/4211981710 shutdown_connections 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.024+0000 7f81c2200640 1 -- 172.21.15.67:0/4211981710 wait complete. 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.026+0000 7f81c2200640 1 Processor -- start 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.026+0000 7f81c2200640 1 -- start start 2024-02-26T10:39:18.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81c2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc06d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81c2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81bc0c7a30 0x7f81bc151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81c2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81c2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c7f90 con 0x7f81bc1503a0 2024-02-26T10:39:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81c2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c7e10 con 0x7f81bc0c7a30 2024-02-26T10:39:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81c2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc153330 con 0x7f81bc070640 2024-02-26T10:39:18.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc145a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43614/0 (socket says 172.21.15.67:43614) 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81bbe00640 1 -- 172.21.15.67:0/1664084551 learned_addr learned my addr 172.21.15.67:0/1664084551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81bb400640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.027+0000 7f81baa00640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81bc0c7a30 0x7f81bc151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81bbe00640 1 -- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 msgr2=0x7f81bc06d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81bbe00640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc06d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81bbe00640 1 -- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81bc0c7a30 msgr2=0x7f81bc151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81bbe00640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81bc0c7a30 0x7f81bc151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81bbe00640 1 -- 172.21.15.67:0/1664084551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc06d770 con 0x7f81bc1503a0 2024-02-26T10:39:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81baa00640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81bc0c7a30 0x7f81bc151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81bbe00640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc145a50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009d90 tx=0x7f81b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4044020 con 0x7f81bc1503a0 2024-02-26T10:39:18.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b403a040 con 0x7f81bc1503a0 2024-02-26T10:39:18.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4039050 con 0x7f81bc1503a0 2024-02-26T10:39:18.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc14e740 con 0x7f81bc1503a0 2024-02-26T10:39:18.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.028+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc14ed00 con 0x7f81bc1503a0 2024-02-26T10:39:18.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.030+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc0701c0 con 0x7f81bc1503a0 2024-02-26T10:39:18.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.031+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81b4003990 con 0x7f81bc1503a0 2024-02-26T10:39:18.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.031+0000 7f81b3400640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f819c06b4d0 0x7f819c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.032+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81b40ac710 con 0x7f81bc1503a0 2024-02-26T10:39:18.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.032+0000 7f81bb400640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f819c06b4d0 0x7f819c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.033+0000 7f81bb400640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f819c06b4d0 0x7f819c06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000f60 tx=0x7f81ac007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.036+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81b404e060 con 0x7f81bc1503a0 2024-02-26T10:39:18.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.240+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81bc14a170 con 0x7f81bc1503a0 2024-02-26T10:39:18.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.240+0000 7f81b3400640 1 -- 172.21.15.67:0/1664084551 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81b407b580 con 0x7f81bc1503a0 2024-02-26T10:39:18.241 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.243+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f819c06b4d0 msgr2=0x7f819c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.243+0000 7f81c2200640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f819c06b4d0 0x7f819c06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000f60 tx=0x7f81ac007b50 comp rx=0 tx=0).stop 2024-02-26T10:39:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.243+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 msgr2=0x7f81bc145a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.243+0000 7f81c2200640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc145a50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009d90 tx=0x7f81b4004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 shutdown_connections 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc1503a0 0x7f81bc145a50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f819c06b4d0 0x7f819c06d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81bc0c7a30 0x7f81bc151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 --2- 172.21.15.67:0/1664084551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81bc070640 0x7f81bc06d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 >> 172.21.15.67:0/1664084551 conn(0x7f81bc06fa10 msgr2=0x7f81bc0c2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 shutdown_connections 2024-02-26T10:39:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.244+0000 7f81c2200640 1 -- 172.21.15.67:0/1664084551 wait complete. 2024-02-26T10:39:18.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:18.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.496+0000 7f2810200640 1 Processor -- start 2024-02-26T10:39:18.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.496+0000 7f2810200640 1 -- start start 2024-02-26T10:39:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.497+0000 7f2810200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f2808158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.497+0000 7f2810200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080c5aa0 con 0x7f2808152710 2024-02-26T10:39:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.497+0000 7f2810200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2808155b40 con 0x7f2808158a50 2024-02-26T10:39:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.497+0000 7f2810200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c52a0 con 0x7f28080c8780 2024-02-26T10:39:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.498+0000 7f280d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2808152710 0x7f28080c7290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39724/0 (socket says 172.21.15.67:39724) 2024-02-26T10:39:18.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.498+0000 7f280d800640 1 -- 172.21.15.67:0/1182411754 learned_addr learned my addr 172.21.15.67:0/1182411754 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:18.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.498+0000 7f280ce00640 1 --2- 172.21.15.67:0/1182411754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f2808158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.498+0000 7f2807e00640 1 -- 172.21.15.67:0/1182411754 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761583642 0 0) 0x7f28080c5aa0 con 0x7f2808152710 2024-02-26T10:39:18.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.498+0000 7f2807e00640 1 -- 172.21.15.67:0/1182411754 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f0003520 con 0x7f2808152710 2024-02-26T10:39:18.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.499+0000 7f280ce00640 1 -- 172.21.15.67:0/1182411754 >> v1:172.21.15.82:6789/0 conn(0x7f2808158a50 legacy=0x7f280815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.499+0000 7f280ce00640 1 -- 172.21.15.67:0/1182411754 >> v1:172.21.15.67:6789/0 conn(0x7f2808152710 legacy=0x7f28080c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.499+0000 7f2807e00640 1 -- 172.21.15.67:0/1182411754 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607969806 0 0) 0x7f27f0003520 con 0x7f2808152710 2024-02-26T10:39:18.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.499+0000 7f280ce00640 1 -- 172.21.15.67:0/1182411754 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808158f00 con 0x7f28080c8780 2024-02-26T10:39:18.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.500+0000 7f280ce00640 1 --2- 172.21.15.67:0/1182411754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f2808158510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27f40097d0 tx=0x7f27f402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8502b5ccd5fea22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.500+0000 7f2807e00640 1 -- 172.21.15.67:0/1182411754 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4037070 con 0x7f28080c8780 2024-02-26T10:39:18.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2807e00640 1 -- 172.21.15.67:0/1182411754 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4030510 con 0x7f28080c8780 2024-02-26T10:39:18.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2807e00640 1 -- 172.21.15.67:0/1182411754 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0019f0 con 0x7f28080c8780 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 -- 172.21.15.67:0/1182411754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 msgr2=0x7f2808158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 --2- 172.21.15.67:0/1182411754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f2808158510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27f40097d0 tx=0x7f27f402b770 comp rx=0 tx=0).stop 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 -- 172.21.15.67:0/1182411754 shutdown_connections 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 --2- 172.21.15.67:0/1182411754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f2808158510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 -- 172.21.15.67:0/1182411754 >> 172.21.15.67:0/1182411754 conn(0x7f28080ac410 msgr2=0x7f28080ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 -- 172.21.15.67:0/1182411754 shutdown_connections 2024-02-26T10:39:18.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.501+0000 7f2810200640 1 -- 172.21.15.67:0/1182411754 wait complete. 2024-02-26T10:39:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.503+0000 7f2810200640 1 Processor -- start 2024-02-26T10:39:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.504+0000 7f2810200640 1 -- start start 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f2810200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f28080c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f2810200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 0x7f280814b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f2810200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2808158a50 0x7f2808157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f2810200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080bdf10 con 0x7f2808152710 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f2810200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080ca5d0 con 0x7f2808158a50 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f2810200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c79a0 con 0x7f28080c8780 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f280ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 0x7f280814b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f280ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 0x7f280814b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43626/0 (socket says 172.21.15.67:43626) 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f280ce00640 1 -- 172.21.15.67:0/3897108243 learned_addr learned my addr 172.21.15.67:0/3897108243 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f280d800640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f28080c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.505+0000 7f280e200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2808158a50 0x7f2808157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f280ce00640 1 -- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 msgr2=0x7f28080c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f280ce00640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f28080c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f280ce00640 1 -- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2808158a50 msgr2=0x7f2808157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f280ce00640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2808158a50 0x7f2808157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f280ce00640 1 -- 172.21.15.67:0/3897108243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080c3550 con 0x7f2808152710 2024-02-26T10:39:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f280ce00640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 0x7f280814b750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098f0 tx=0x7f27f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4041070 con 0x7f2808152710 2024-02-26T10:39:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.506+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f403c870 con 0x7f2808152710 2024-02-26T10:39:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.507+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28080c3830 con 0x7f2808152710 2024-02-26T10:39:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.507+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4037040 con 0x7f2808152710 2024-02-26T10:39:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.507+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2808158400 con 0x7f2808152710 2024-02-26T10:39:18.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.508+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28080678e0 con 0x7f2808152710 2024-02-26T10:39:18.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.509+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27f4044070 con 0x7f2808152710 2024-02-26T10:39:18.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.510+0000 7f2805600640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f27e806b520 0x7f27e806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.510+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27f40abe10 con 0x7f2808152710 2024-02-26T10:39:18.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.510+0000 7f280d800640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f27e806b520 0x7f27e806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.511+0000 7f280d800640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f27e806b520 0x7f27e806d9e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27fc005c00 tx=0x7f27fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.514+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27f40af030 con 0x7f2808152710 2024-02-26T10:39:18.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.711+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2808074a30 con 0x7f2808152710 2024-02-26T10:39:18.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.712+0000 7f2805600640 1 -- 172.21.15.67:0/3897108243 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27f407ae20 con 0x7f2808152710 2024-02-26T10:39:18.714 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f27e806b520 msgr2=0x7f27e806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f27e806b520 0x7f27e806d9e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27fc005c00 tx=0x7f27fc009040 comp rx=0 tx=0).stop 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 msgr2=0x7f280814b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 0x7f280814b750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098f0 tx=0x7f27f40046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 shutdown_connections 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f27e806b520 0x7f27e806d9e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2808158a50 0x7f2808157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2808152710 0x7f280814b750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 --2- 172.21.15.67:0/3897108243 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28080c8780 0x7f28080c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 >> 172.21.15.67:0/3897108243 conn(0x7f28080ac410 msgr2=0x7f28080aafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 shutdown_connections 2024-02-26T10:39:18.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.715+0000 7f2810200640 1 -- 172.21.15.67:0/3897108243 wait complete. 2024-02-26T10:39:18.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:18.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.958+0000 7f2023e00640 1 Processor -- start 2024-02-26T10:39:18.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.959+0000 7f2023e00640 1 -- start start 2024-02-26T10:39:18.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.960+0000 7f2023e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20240a9430 0x7f20240ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.960+0000 7f2023e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2024150780 con 0x7f20241503a0 2024-02-26T10:39:18.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.960+0000 7f2023e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20240a98c0 con 0x7f202406e2d0 2024-02-26T10:39:18.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.960+0000 7f2023e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240a9ba0 con 0x7f20240a9430 2024-02-26T10:39:18.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.961+0000 7f2022a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f20241503a0 0x7f20241535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39726/0 (socket says 172.21.15.67:39726) 2024-02-26T10:39:18.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.961+0000 7f2022a00640 1 -- 172.21.15.67:0/2638926160 learned_addr learned my addr 172.21.15.67:0/2638926160 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:18.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.961+0000 7f2022000640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20240a9430 0x7f20240ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.961+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661142230 0 0) 0x7f2024150780 con 0x7f20241503a0 2024-02-26T10:39:18.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.961+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2008003520 con 0x7f20241503a0 2024-02-26T10:39:18.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1204740797 0 0) 0x7f2008003520 con 0x7f20241503a0 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20240a9430 msgr2=0x7f20240ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20240a9430 0x7f20240ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 >> v1:172.21.15.82:6789/0 conn(0x7f202406e2d0 legacy=0x7f20240c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240a9e80 con 0x7f20241503a0 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f2014001670 con 0x7f20241503a0 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 >> v1:172.21.15.67:6789/0 conn(0x7f20241503a0 legacy=0x7f20241535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20080039e0 0x7f2008003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2008004800 0x7f2008004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2008005180 0x7f200800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20240a9ba0 con 0x7f20080039e0 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200800bf00 con 0x7f2008005180 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200800c080 con 0x7f2008004800 2024-02-26T10:39:18.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.962+0000 7f2022a00640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20080039e0 0x7f2008003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2023400640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2008005180 0x7f200800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2022000640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2008004800 0x7f2008004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2022a00640 1 -- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2008004800 msgr2=0x7f2008004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2022a00640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2008004800 0x7f2008004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2022a00640 1 -- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2008005180 msgr2=0x7f200800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2022a00640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2008005180 0x7f200800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.963+0000 7f2022a00640 1 -- 172.21.15.67:0/2638926160 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008003520 con 0x7f20080039e0 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.964+0000 7f2022a00640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20080039e0 0x7f2008003e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f201400c110 tx=0x7f2008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9efa922c2ed9d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.964+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014007c40 con 0x7f20080039e0 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.964+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2014030070 con 0x7f20080039e0 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.964+0000 7f2021600640 1 -- 172.21.15.67:0/2638926160 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014005d30 con 0x7f20080039e0 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023e00640 1 -- 172.21.15.67:0/2638926160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20080039e0 msgr2=0x7f2008003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023e00640 1 --2- 172.21.15.67:0/2638926160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20080039e0 0x7f2008003e00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f201400c110 tx=0x7f2008005830 comp rx=0 tx=0).stop 2024-02-26T10:39:18.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023400640 1 -- 172.21.15.67:0/2638926160 reap_dead start 2024-02-26T10:39:18.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023e00640 1 -- 172.21.15.67:0/2638926160 shutdown_connections 2024-02-26T10:39:18.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023e00640 1 -- 172.21.15.67:0/2638926160 >> 172.21.15.67:0/2638926160 conn(0x7f20240c6380 msgr2=0x7f20240c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:18.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023e00640 1 -- 172.21.15.67:0/2638926160 shutdown_connections 2024-02-26T10:39:18.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.965+0000 7f2023e00640 1 -- 172.21.15.67:0/2638926160 wait complete. 2024-02-26T10:39:18.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.967+0000 7f2023e00640 1 Processor -- start 2024-02-26T10:39:18.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.967+0000 7f2023e00640 1 -- start start 2024-02-26T10:39:18.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2023e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20080039e0 0x7f20241e1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2023e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 0x7f2024149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2023e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2024156890 0x7f202414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2023e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20240c0f80 con 0x7f202414c240 2024-02-26T10:39:18.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2023e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202406ddb0 con 0x7f20080039e0 2024-02-26T10:39:18.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2023e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20240c3520 con 0x7f2024156890 2024-02-26T10:39:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2022000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 0x7f2024149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2022000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 0x7f2024149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43642/0 (socket says 172.21.15.67:43642) 2024-02-26T10:39:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.968+0000 7f2022000640 1 -- 172.21.15.67:0/233830560 learned_addr learned my addr 172.21.15.67:0/233830560 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022a00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20080039e0 0x7f20241e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2023400640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2024156890 0x7f202414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022000640 1 -- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2024156890 msgr2=0x7f202414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022000640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2024156890 0x7f202414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022000640 1 -- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20080039e0 msgr2=0x7f20241e1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022000640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20080039e0 0x7f20241e1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022000640 1 -- 172.21.15.67:0/233830560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024153fd0 con 0x7f202414c240 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f2022000640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 0x7f2024149cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2010011630 tx=0x7f2010011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000f920 con 0x7f202414c240 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.969+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201000fac0 con 0x7f202414c240 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.970+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20241eed90 con 0x7f202414c240 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.970+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000fe70 con 0x7f202414c240 2024-02-26T10:39:18.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.970+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20241e1ea0 con 0x7f202414c240 2024-02-26T10:39:18.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.971+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20240678e0 con 0x7f202414c240 2024-02-26T10:39:18.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.972+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2010015460 con 0x7f202414c240 2024-02-26T10:39:18.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.973+0000 7f201aa00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f200406b500 0x7f200406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:18.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.973+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f201008aa70 con 0x7f202414c240 2024-02-26T10:39:18.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.973+0000 7f2022a00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f200406b500 0x7f200406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:18.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.974+0000 7f2022a00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f200406b500 0x7f200406d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2014034000 tx=0x7f2014009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:18.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:18.977+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2010019030 con 0x7f202414c240 2024-02-26T10:39:19.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.177+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2024074a30 con 0x7f202414c240 2024-02-26T10:39:19.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.178+0000 7f201aa00640 1 -- 172.21.15.67:0/233830560 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2010059290 con 0x7f202414c240 2024-02-26T10:39:19.180 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:19.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f200406b500 msgr2=0x7f200406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f200406b500 0x7f200406d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2014034000 tx=0x7f2014009030 comp rx=0 tx=0).stop 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 msgr2=0x7f2024149cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 0x7f2024149cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2010011630 tx=0x7f2010011af0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 shutdown_connections 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2024156890 0x7f202414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f202414c240 0x7f2024149cf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f200406b500 0x7f200406d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 --2- 172.21.15.67:0/233830560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20080039e0 0x7f20241e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 >> 172.21.15.67:0/233830560 conn(0x7f20240c6380 msgr2=0x7f20240bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 shutdown_connections 2024-02-26T10:39:19.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.181+0000 7f2023e00640 1 -- 172.21.15.67:0/233830560 wait complete. 2024-02-26T10:39:19.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:39:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.427+0000 7f7e0f200640 1 Processor -- start 2024-02-26T10:39:19.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.428+0000 7f7e0f200640 1 -- start start 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.429+0000 7f7e0f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e080c1c10 0x7f7e081557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.429+0000 7f7e0f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e08155cf0 con 0x7f7e080c0850 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.429+0000 7f7e0f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0806dca0 con 0x7f7e0806e180 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.429+0000 7f7e0f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e080c2a70 con 0x7f7e080c1c10 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.429+0000 7f7e0d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7e080c0850 0x7f7e0806ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36142/0 (socket says 172.21.15.67:36142) 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0d400640 1 -- 172.21.15.67:0/773080541 learned_addr learned my addr 172.21.15.67:0/773080541 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:19.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0de00640 1 --2- 172.21.15.67:0/773080541 >> v2:172.21.15.67:3300/0 conn(0x7f7e080c1c10 0x7f7e081557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0ca00640 1 -- 172.21.15.67:0/773080541 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238126449 0 0) 0x7f7df8001110 con 0x7f7e080c0850 2024-02-26T10:39:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0ca00640 1 -- 172.21.15.67:0/773080541 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df0003520 con 0x7f7e080c0850 2024-02-26T10:39:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0de00640 1 -- 172.21.15.67:0/773080541 >> v1:172.21.15.82:6789/0 conn(0x7f7e0806e180 legacy=0x7f7e0815f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0de00640 1 -- 172.21.15.67:0/773080541 >> v1:172.21.15.67:6789/0 conn(0x7f7e080c0850 legacy=0x7f7e0806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.430+0000 7f7e0de00640 1 -- 172.21.15.67:0/773080541 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080c21d0 con 0x7f7e080c1c10 2024-02-26T10:39:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.431+0000 7f7e0de00640 1 --2- 172.21.15.67:0/773080541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c1c10 0x7f7e081557b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0097a0 tx=0x7f7dfc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834120edc4af5f54 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0ca00640 1 -- 172.21.15.67:0/773080541 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0315a0 con 0x7f7e080c1c10 2024-02-26T10:39:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0ca00640 1 -- 172.21.15.67:0/773080541 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc037070 con 0x7f7e080c1c10 2024-02-26T10:39:19.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0ca00640 1 -- 172.21.15.67:0/773080541 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc031960 con 0x7f7e080c1c10 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0f200640 1 -- 172.21.15.67:0/773080541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c1c10 msgr2=0x7f7e081557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0f200640 1 --2- 172.21.15.67:0/773080541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c1c10 0x7f7e081557b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0097a0 tx=0x7f7dfc02b840 comp rx=0 tx=0).stop 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0f200640 1 -- 172.21.15.67:0/773080541 shutdown_connections 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0f200640 1 --2- 172.21.15.67:0/773080541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c1c10 0x7f7e081557b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0f200640 1 -- 172.21.15.67:0/773080541 >> 172.21.15.67:0/773080541 conn(0x7f7e080c63f0 msgr2=0x7f7e080c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.432+0000 7f7e0f200640 1 -- 172.21.15.67:0/773080541 shutdown_connections 2024-02-26T10:39:19.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.433+0000 7f7e0f200640 1 -- 172.21.15.67:0/773080541 wait complete. 2024-02-26T10:39:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.434+0000 7f7e0f200640 1 Processor -- start 2024-02-26T10:39:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.434+0000 7f7e0f200640 1 -- start start 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.435+0000 7f7e0f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e0806e180 0x7f7e08158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.435+0000 7f7e0f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 0x7f7e08158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.435+0000 7f7e0f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e080c1c10 0x7f7e081e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.435+0000 7f7e0f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0806b330 con 0x7f7e080c0850 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.435+0000 7f7e0f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080c2790 con 0x7f7e0806e180 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.435+0000 7f7e0f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0806b4b0 con 0x7f7e080c1c10 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 0x7f7e08158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 0x7f7e08158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60464/0 (socket says 172.21.15.67:60464) 2024-02-26T10:39:19.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 -- 172.21.15.67:0/3241433892 learned_addr learned my addr 172.21.15.67:0/3241433892 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:19.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0de00640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e0806e180 0x7f7e08158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0e800640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e080c1c10 0x7f7e081e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 -- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e080c1c10 msgr2=0x7f7e081e1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e080c1c10 0x7f7e081e1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 -- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e0806e180 msgr2=0x7f7e08158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e0806e180 0x7f7e08158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0d400640 1 -- 172.21.15.67:0/3241433892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e081591b0 con 0x7f7e080c0850 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0e800640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e080c1c10 0x7f7e081e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.436+0000 7f7e0de00640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e0806e180 0x7f7e08158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:19.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.437+0000 7f7e0d400640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 0x7f7e08158c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c730 tx=0x7f7df800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.437+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8017070 con 0x7f7e080c0850 2024-02-26T10:39:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.437+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df8005ab0 con 0x7f7e080c0850 2024-02-26T10:39:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.437+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8007070 con 0x7f7e080c0850 2024-02-26T10:39:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.437+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e081510f0 con 0x7f7e080c0850 2024-02-26T10:39:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.437+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08151690 con 0x7f7e080c0850 2024-02-26T10:39:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.438+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0806bea0 con 0x7f7e080c0850 2024-02-26T10:39:19.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.442+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7df801a070 con 0x7f7e080c0850 2024-02-26T10:39:19.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.442+0000 7f7e02000640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7dec06b4d0 0x7f7dec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:19.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.443+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7df8089a00 con 0x7f7e080c0850 2024-02-26T10:39:19.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.443+0000 7f7e0de00640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7dec06b4d0 0x7f7dec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:19.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.444+0000 7f7e0de00640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7dec06b4d0 0x7f7dec06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc036000 tx=0x7f7dfc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:19.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.446+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e0806bea0 con 0x7f7e080c0850 2024-02-26T10:39:19.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.657+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e080c61e0 con 0x7f7e080c0850 2024-02-26T10:39:19.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.658+0000 7f7e02000640 1 -- 172.21.15.67:0/3241433892 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e080c61e0 con 0x7f7e080c0850 2024-02-26T10:39:19.660 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:39:19.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7dec06b4d0 msgr2=0x7f7dec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7dec06b4d0 0x7f7dec06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc036000 tx=0x7f7dfc03d040 comp rx=0 tx=0).stop 2024-02-26T10:39:19.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 msgr2=0x7f7e08158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 0x7f7e08158c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c730 tx=0x7f7df800cbf0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 shutdown_connections 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e080c1c10 0x7f7e081e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e080c0850 0x7f7e08158c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7dec06b4d0 0x7f7dec06d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 --2- 172.21.15.67:0/3241433892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e0806e180 0x7f7e08158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 >> 172.21.15.67:0/3241433892 conn(0x7f7e080c63f0 msgr2=0x7f7e080c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 shutdown_connections 2024-02-26T10:39:19.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:19.661+0000 7f7e0f200640 1 -- 172.21.15.67:0/3241433892 wait complete. 2024-02-26T10:39:19.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T10:39:19.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:19.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-02-26T10:39:19.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:39:19.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:19.786 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T10:39:19.853 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:39:19.853 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:19.913 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:39:19.913 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:19.964 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:39:19.964 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:20.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.242+0000 7f77fea00640 1 Processor -- start 2024-02-26T10:39:20.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.243+0000 7f77fea00640 1 -- start start 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.244+0000 7f77fea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77f806ac20 0x7f77f80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.244+0000 7f77fea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f77f806b000 0x7f77f806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.244+0000 7f77fea00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f806b460 con 0x7f77f8153610 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.244+0000 7f77fea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80a9a80 con 0x7f77f806ac20 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.244+0000 7f77fea00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80a9d60 con 0x7f77f806b000 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77f806ac20 0x7f77f80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77f806ac20 0x7f77f80a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60472/0 (socket says 172.21.15.67:60472) 2024-02-26T10:39:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 -- 172.21.15.67:0/3883818481 learned_addr learned my addr 172.21.15.67:0/3883818481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 -- 172.21.15.67:0/3883818481 >> v2:172.21.15.82:3300/0 conn(0x7f77f806b000 msgr2=0x7f77f806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 --2- 172.21.15.67:0/3883818481 >> v2:172.21.15.82:3300/0 conn(0x7f77f806b000 0x7f77f806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 -- 172.21.15.67:0/3883818481 >> v1:172.21.15.82:6789/0 conn(0x7f77f8153610 legacy=0x7f77f80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.245+0000 7f77fd600640 1 -- 172.21.15.67:0/3883818481 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80ab750 con 0x7f77f806ac20 2024-02-26T10:39:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.246+0000 7f77fd600640 1 --2- 172.21.15.67:0/3883818481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806ac20 0x7f77f80a94a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77f806ed00 tx=0x7f77ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65df8b9c5710dfd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.246+0000 7f77f7e00640 1 -- 172.21.15.67:0/3883818481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec035020 con 0x7f77f806ac20 2024-02-26T10:39:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77f7e00640 1 -- 172.21.15.67:0/3883818481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec031870 con 0x7f77f806ac20 2024-02-26T10:39:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77f7e00640 1 -- 172.21.15.67:0/3883818481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec031c40 con 0x7f77f806ac20 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 -- 172.21.15.67:0/3883818481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806ac20 msgr2=0x7f77f80a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 --2- 172.21.15.67:0/3883818481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806ac20 0x7f77f80a94a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77f806ed00 tx=0x7f77ec02b910 comp rx=0 tx=0).stop 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 -- 172.21.15.67:0/3883818481 shutdown_connections 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 --2- 172.21.15.67:0/3883818481 >> v2:172.21.15.82:3300/0 conn(0x7f77f806b000 0x7f77f806db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 --2- 172.21.15.67:0/3883818481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806ac20 0x7f77f80a94a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 -- 172.21.15.67:0/3883818481 >> 172.21.15.67:0/3883818481 conn(0x7f77f80c63f0 msgr2=0x7f77f80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 -- 172.21.15.67:0/3883818481 shutdown_connections 2024-02-26T10:39:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.247+0000 7f77fea00640 1 -- 172.21.15.67:0/3883818481 wait complete. 2024-02-26T10:39:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.249+0000 7f77fea00640 1 Processor -- start 2024-02-26T10:39:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.250+0000 7f77fea00640 1 -- start start 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77f806ac20 0x7f77f81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 0x7f77f8149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f8153610 0x7f77f814a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8150410 con 0x7f77f806b000 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8153a60 con 0x7f77f8153610 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8150590 con 0x7f77f806ac20 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fcc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 0x7f77f8149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fcc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 0x7f77f8149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60484/0 (socket says 172.21.15.67:60484) 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.251+0000 7f77fcc00640 1 -- 172.21.15.67:0/2595065702 learned_addr learned my addr 172.21.15.67:0/2595065702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fd600640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77f806ac20 0x7f77f81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fe000640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f8153610 0x7f77f814a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fcc00640 1 -- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77f806ac20 msgr2=0x7f77f81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fcc00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77f806ac20 0x7f77f81565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fcc00640 1 -- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f8153610 msgr2=0x7f77f814a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fcc00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f8153610 0x7f77f814a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fcc00640 1 -- 172.21.15.67:0/2595065702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f81543d0 con 0x7f77f806b000 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fe000640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f8153610 0x7f77f814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fcc00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 0x7f77f8149b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77e800b4a0 tx=0x7f77e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8014070 con 0x7f77f806b000 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8010610 con 0x7f77f806b000 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8010a00 con 0x7f77f806b000 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.252+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f81ecdd0 con 0x7f77f806b000 2024-02-26T10:39:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.253+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f81dd1b0 con 0x7f77f806b000 2024-02-26T10:39:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.254+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f8074a30 con 0x7f77f806b000 2024-02-26T10:39:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.255+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77e8017070 con 0x7f77f806b000 2024-02-26T10:39:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.256+0000 7f77f5600640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77dc06b4d0 0x7f77dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.256+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e8089b90 con 0x7f77f806b000 2024-02-26T10:39:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.257+0000 7f77fd600640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77dc06b4d0 0x7f77dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.258+0000 7f77fd600640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77dc06b4d0 0x7f77dc06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77ec036000 tx=0x7f77ec03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:20.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.261+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77e8058aa0 con 0x7f77f806b000 2024-02-26T10:39:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.465+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77f80c6b70 con 0x7f77f806b000 2024-02-26T10:39:20.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.470+0000 7f77f5600640 1 -- 172.21.15.67:0/2595065702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f77e80583a0 con 0x7f77f806b000 2024-02-26T10:39:20.472 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:20.473 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:20.439824+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:20.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77dc06b4d0 msgr2=0x7f77dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77dc06b4d0 0x7f77dc06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77ec036000 tx=0x7f77ec03c040 comp rx=0 tx=0).stop 2024-02-26T10:39:20.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 msgr2=0x7f77f8149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 0x7f77f8149b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f77e800b4a0 tx=0x7f77e800b960 comp rx=0 tx=0).stop 2024-02-26T10:39:20.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 shutdown_connections 2024-02-26T10:39:20.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77dc06b4d0 0x7f77dc06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f8153610 0x7f77f814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77f806b000 0x7f77f8149b60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 --2- 172.21.15.67:0/2595065702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77f806ac20 0x7f77f81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.473+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 >> 172.21.15.67:0/2595065702 conn(0x7f77f80c63f0 msgr2=0x7f77f814d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:20.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.474+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 shutdown_connections 2024-02-26T10:39:20.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.474+0000 7f77fea00640 1 -- 172.21.15.67:0/2595065702 wait complete. 2024-02-26T10:39:20.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:20.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:20.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.709+0000 7f44c1000640 1 Processor -- start 2024-02-26T10:39:20.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.709+0000 7f44c1000640 1 -- start start 2024-02-26T10:39:20.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.710+0000 7f44c1000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44bc158c20 0x7f44bc1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.710+0000 7f44c1000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc0c5aa0 con 0x7f44bc152710 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.710+0000 7f44c1000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc155b40 con 0x7f44bc0c8780 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.710+0000 7f44c1000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc0c52a0 con 0x7f44bc158c20 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.711+0000 7f44b9e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f44bc152710 0x7f44bc0c7290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36152/0 (socket says 172.21.15.67:36152) 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.711+0000 7f44b9e00640 1 -- 172.21.15.67:0/2760686680 learned_addr learned my addr 172.21.15.67:0/2760686680 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.711+0000 7f44ba800640 1 --2- 172.21.15.67:0/2760686680 >> v2:172.21.15.82:3300/0 conn(0x7f44bc158c20 0x7f44bc1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.711+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302875949 0 0) 0x7f44bc0c5aa0 con 0x7f44bc152710 2024-02-26T10:39:20.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.711+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003520 con 0x7f44bc152710 2024-02-26T10:39:20.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4136239824 0 0) 0x7f44a0003520 con 0x7f44bc152710 2024-02-26T10:39:20.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 >> v2:172.21.15.82:3300/0 conn(0x7f44bc158c20 msgr2=0x7f44bc1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:20.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 --2- 172.21.15.67:0/2760686680 >> v2:172.21.15.82:3300/0 conn(0x7f44bc158c20 0x7f44bc1608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 >> v1:172.21.15.82:6789/0 conn(0x7f44bc0c8780 legacy=0x7f44bc158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc155e20 con 0x7f44bc152710 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f44ac0019f0 con 0x7f44bc152710 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 >> v1:172.21.15.67:6789/0 conn(0x7f44bc152710 legacy=0x7f44bc0c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a00039b0 0x7f44a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0c52a0 con 0x7f44a0004800 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c160 con 0x7f44a00051c0 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c360 con 0x7f44a00039b0 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.712+0000 7f44b9400640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44b9400640 1 -- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a00039b0 msgr2=0x7f44a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44b9e00640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a00039b0 0x7f44a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44ba800640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44b9400640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a00039b0 0x7f44a0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44b9400640 1 -- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a00051c0 msgr2=0x7f44a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:20.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44b9400640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44b9400640 1 -- 172.21.15.67:0/2760686680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0003520 con 0x7f44a0004800 2024-02-26T10:39:20.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.713+0000 7f44ba800640 1 -- 172.21.15.67:0/2760686680 reap_dead start 2024-02-26T10:39:20.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.714+0000 7f44b9400640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44a4001720 tx=0x7f44a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bf8e1e9b19fc68 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:20.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.714+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4001370 con 0x7f44a0004800 2024-02-26T10:39:20.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.714+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a40052d0 con 0x7f44a0004800 2024-02-26T10:39:20.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.714+0000 7f44b8a00640 1 -- 172.21.15.67:0/2760686680 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4003830 con 0x7f44a0004800 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 -- 172.21.15.67:0/2760686680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 msgr2=0x7f44a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44a4001720 tx=0x7f44a0005830 comp rx=0 tx=0).stop 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 -- 172.21.15.67:0/2760686680 shutdown_connections 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 --2- 172.21.15.67:0/2760686680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 -- 172.21.15.67:0/2760686680 >> 172.21.15.67:0/2760686680 conn(0x7f44bc0ac410 msgr2=0x7f44bc0ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 -- 172.21.15.67:0/2760686680 shutdown_connections 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.715+0000 7f44c1000640 1 -- 172.21.15.67:0/2760686680 wait complete. 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.716+0000 7f44c1000640 1 Processor -- start 2024-02-26T10:39:20.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.717+0000 7f44c1000640 1 -- start start 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.717+0000 7f44c1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44bc1eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.717+0000 7f44c1000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44bc0c0db0 0x7f44bc0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44c1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc1edf30 0x7f44bc1ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44c1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc06dba0 con 0x7f44a0004800 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44c1000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0bdf10 con 0x7f44bc1edf30 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44c1000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc06dd20 con 0x7f44bc0c0db0 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44bc1eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44bc1eae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60512/0 (socket says 172.21.15.67:60512) 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 -- 172.21.15.67:0/289486327 learned_addr learned my addr 172.21.15.67:0/289486327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9400640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44bc0c0db0 0x7f44bc0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44ba800640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc1edf30 0x7f44bc1ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 -- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44bc0c0db0 msgr2=0x7f44bc0c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44bc0c0db0 0x7f44bc0c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 -- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc1edf30 msgr2=0x7f44bc1ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc1edf30 0x7f44bc1ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.718+0000 7f44b9e00640 1 -- 172.21.15.67:0/289486327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0c0980 con 0x7f44a0004800 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.719+0000 7f44b9e00640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44bc1eae40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44ac002090 tx=0x7f44ac00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.719+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac019070 con 0x7f44a0004800 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.719+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac004d60 con 0x7f44a0004800 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.719+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0072d0 con 0x7f44a0004800 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.719+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc0b07b0 con 0x7f44a0004800 2024-02-26T10:39:20.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.719+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc0c06a0 con 0x7f44a0004800 2024-02-26T10:39:20.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.720+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc0678e0 con 0x7f44a0004800 2024-02-26T10:39:20.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.721+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44ac007470 con 0x7f44a0004800 2024-02-26T10:39:20.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.722+0000 7f44b2000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b4d0 0x7f449806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:20.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.722+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f44ac08a2a0 con 0x7f44a0004800 2024-02-26T10:39:20.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.723+0000 7f44b9400640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b4d0 0x7f449806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:20.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.724+0000 7f44b9400640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b4d0 0x7f449806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44a4033000 tx=0x7f44a4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:20.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.727+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44ac0590d0 con 0x7f44a0004800 2024-02-26T10:39:20.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.925+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44bc074a30 con 0x7f44a0004800 2024-02-26T10:39:20.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.926+0000 7f44b2000640 1 -- 172.21.15.67:0/289486327 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44ac0589d0 con 0x7f44a0004800 2024-02-26T10:39:20.927 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b4d0 msgr2=0x7f449806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b4d0 0x7f449806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f44a4033000 tx=0x7f44a4009030 comp rx=0 tx=0).stop 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 msgr2=0x7f44bc1eae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44bc1eae40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44ac002090 tx=0x7f44ac00b4a0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 shutdown_connections 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b4d0 0x7f449806d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc1edf30 0x7f44bc1ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44bc0c0db0 0x7f44bc0c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 --2- 172.21.15.67:0/289486327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a0004800 0x7f44bc1eae40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 >> 172.21.15.67:0/289486327 conn(0x7f44bc0ac410 msgr2=0x7f44bc155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 shutdown_connections 2024-02-26T10:39:20.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:20.929+0000 7f44c1000640 1 -- 172.21.15.67:0/289486327 wait complete. 2024-02-26T10:39:20.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:21.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.170+0000 7f1436400640 1 Processor -- start 2024-02-26T10:39:21.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.170+0000 7f1436400640 1 -- start start 2024-02-26T10:39:21.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f1436400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f14301503d0 0x7f143006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f1436400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f143006e390 0x7f14301533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f1436400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14300a6eb0 con 0x7f14300c2950 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f1436400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14300a7190 con 0x7f143006e390 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f1436400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14300a7470 con 0x7f14301503d0 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f142ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f143006e390 0x7f14301533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f142ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f143006e390 0x7f14301533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60524/0 (socket says 172.21.15.67:60524) 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.172+0000 7f142ea00640 1 -- 172.21.15.67:0/1809077642 learned_addr learned my addr 172.21.15.67:0/1809077642 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.173+0000 7f142f400640 1 --2- 172.21.15.67:0/1809077642 >> v2:172.21.15.82:3300/0 conn(0x7f14301503d0 0x7f143006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.173+0000 7f142ea00640 1 -- 172.21.15.67:0/1809077642 >> v2:172.21.15.82:3300/0 conn(0x7f14301503d0 msgr2=0x7f143006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.173+0000 7f142ea00640 1 --2- 172.21.15.67:0/1809077642 >> v2:172.21.15.82:3300/0 conn(0x7f14301503d0 0x7f143006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.173+0000 7f142ea00640 1 -- 172.21.15.67:0/1809077642 >> v1:172.21.15.67:6789/0 conn(0x7f14300c2950 legacy=0x7f1430156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.173+0000 7f142ea00640 1 -- 172.21.15.67:0/1809077642 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300a7750 con 0x7f143006e390 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.174+0000 7f142ea00640 1 --2- 172.21.15.67:0/1809077642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143006e390 0x7f14301533e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f143006f1d0 tx=0x7f141802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bb00fc178ffa55 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.174+0000 7f142e000640 1 -- 172.21.15.67:0/1809077642 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141802c9f0 con 0x7f143006e390 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.174+0000 7f142e000640 1 -- 172.21.15.67:0/1809077642 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141802cb90 con 0x7f143006e390 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.174+0000 7f142e000640 1 -- 172.21.15.67:0/1809077642 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14180343f0 con 0x7f143006e390 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 -- 172.21.15.67:0/1809077642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143006e390 msgr2=0x7f14301533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 --2- 172.21.15.67:0/1809077642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143006e390 0x7f14301533e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f143006f1d0 tx=0x7f141802be50 comp rx=0 tx=0).stop 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 -- 172.21.15.67:0/1809077642 shutdown_connections 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 --2- 172.21.15.67:0/1809077642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f143006e390 0x7f14301533e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 --2- 172.21.15.67:0/1809077642 >> v2:172.21.15.82:3300/0 conn(0x7f14301503d0 0x7f143006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 -- 172.21.15.67:0/1809077642 >> 172.21.15.67:0/1809077642 conn(0x7f14300c6da0 msgr2=0x7f14300c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:21.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 -- 172.21.15.67:0/1809077642 shutdown_connections 2024-02-26T10:39:21.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.175+0000 7f1436400640 1 -- 172.21.15.67:0/1809077642 wait complete. 2024-02-26T10:39:21.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.177+0000 7f1436400640 1 Processor -- start 2024-02-26T10:39:21.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.177+0000 7f1436400640 1 -- start start 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f1436400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f143006e390 0x7f14301eb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f1436400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14300c2950 0x7f14301eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f1436400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 0x7f14300c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f1436400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14301507b0 con 0x7f14301503d0 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f1436400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143006ef00 con 0x7f14300c2950 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f1436400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14300c0ef0 con 0x7f143006e390 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f142fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 0x7f14300c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f142fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 0x7f14300c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60528/0 (socket says 172.21.15.67:60528) 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.178+0000 7f142fe00640 1 -- 172.21.15.67:0/2631013524 learned_addr learned my addr 172.21.15.67:0/2631013524 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142f400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f143006e390 0x7f14301eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142ea00640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14300c2950 0x7f14301eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142fe00640 1 -- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f143006e390 msgr2=0x7f14301eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142fe00640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f143006e390 0x7f14301eb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142fe00640 1 -- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14300c2950 msgr2=0x7f14301eea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142fe00640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14300c2950 0x7f14301eea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142fe00640 1 -- 172.21.15.67:0/2631013524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300a95e0 con 0x7f14301503d0 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f142fe00640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 0x7f14300c5c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1428001eb0 tx=0x7f142800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428017070 con 0x7f14301503d0 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14280052d0 con 0x7f14301503d0 2024-02-26T10:39:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428003810 con 0x7f14301503d0 2024-02-26T10:39:21.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.179+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14300a98c0 con 0x7f14301503d0 2024-02-26T10:39:21.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.180+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14301eae70 con 0x7f14301503d0 2024-02-26T10:39:21.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.181+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1430074a30 con 0x7f14301503d0 2024-02-26T10:39:21.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.182+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f142801a070 con 0x7f14301503d0 2024-02-26T10:39:21.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.183+0000 7f1427400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f140c06b520 0x7f140c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.183+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1428089650 con 0x7f14301503d0 2024-02-26T10:39:21.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.183+0000 7f142f400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f140c06b520 0x7f140c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.185+0000 7f142f400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f140c06b520 0x7f140c06d9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1420006fd0 tx=0x7f1420008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:21.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.187+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14280584c0 con 0x7f14301503d0 2024-02-26T10:39:21.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.385+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14300a7a20 con 0x7f14301503d0 2024-02-26T10:39:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.386+0000 7f1427400640 1 -- 172.21.15.67:0/2631013524 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1428057dc0 con 0x7f14301503d0 2024-02-26T10:39:21.388 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.388+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f140c06b520 msgr2=0x7f140c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.388+0000 7f1436400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f140c06b520 0x7f140c06d9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1420006fd0 tx=0x7f1420008040 comp rx=0 tx=0).stop 2024-02-26T10:39:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.388+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 msgr2=0x7f14300c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.388+0000 7f1436400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 0x7f14300c5c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1428001eb0 tx=0x7f142800c890 comp rx=0 tx=0).stop 2024-02-26T10:39:21.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 shutdown_connections 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14301503d0 0x7f14300c5c90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f140c06b520 0x7f140c06d9e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14300c2950 0x7f14301eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 --2- 172.21.15.67:0/2631013524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f143006e390 0x7f14301eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 >> 172.21.15.67:0/2631013524 conn(0x7f14300c6da0 msgr2=0x7f14300bf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 shutdown_connections 2024-02-26T10:39:21.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.389+0000 7f1436400640 1 -- 172.21.15.67:0/2631013524 wait complete. 2024-02-26T10:39:21.411 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.630+0000 7fa333200640 1 Processor -- start 2024-02-26T10:39:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.630+0000 7fa333200640 1 -- start start 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa333200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3340a9430 0x7fa3340ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa333200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa33406e2d0 0x7fa3340c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa333200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa334150780 con 0x7fa3341503a0 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa333200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3340a98c0 con 0x7fa3340a9430 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa333200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3340a9ba0 con 0x7fa33406e2d0 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa331400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3340a9430 0x7fa3340ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa331400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3340a9430 0x7fa3340ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60530/0 (socket says 172.21.15.67:60530) 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.632+0000 7fa331400640 1 -- 172.21.15.67:0/1443820137 learned_addr learned my addr 172.21.15.67:0/1443820137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:21.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.633+0000 7fa332800640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa33406e2d0 0x7fa3340c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.633+0000 7fa331400640 1 -- 172.21.15.67:0/1443820137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa33406e2d0 msgr2=0x7fa3340c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:21.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.633+0000 7fa331400640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa33406e2d0 0x7fa3340c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.633+0000 7fa331400640 1 -- 172.21.15.67:0/1443820137 >> v1:172.21.15.82:6789/0 conn(0x7fa3341503a0 legacy=0x7fa3341535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:21.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.633+0000 7fa331400640 1 -- 172.21.15.67:0/1443820137 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3340a9e80 con 0x7fa3340a9430 2024-02-26T10:39:21.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.633+0000 7fa332800640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa33406e2d0 0x7fa3340c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:21.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.634+0000 7fa331400640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3340a9430 0x7fa3340ab480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa320009870 tx=0x7fa32002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd45fb1a05e916a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.634+0000 7fa330a00640 1 -- 172.21.15.67:0/1443820137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320035020 con 0x7fa3340a9430 2024-02-26T10:39:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.634+0000 7fa330a00640 1 -- 172.21.15.67:0/1443820137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320031640 con 0x7fa3340a9430 2024-02-26T10:39:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.634+0000 7fa330a00640 1 -- 172.21.15.67:0/1443820137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32002d040 con 0x7fa3340a9430 2024-02-26T10:39:21.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 -- 172.21.15.67:0/1443820137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3340a9430 msgr2=0x7fa3340ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3340a9430 0x7fa3340ab480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa320009870 tx=0x7fa32002b8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 -- 172.21.15.67:0/1443820137 shutdown_connections 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa33406e2d0 0x7fa3340c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 --2- 172.21.15.67:0/1443820137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3340a9430 0x7fa3340ab480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 -- 172.21.15.67:0/1443820137 >> 172.21.15.67:0/1443820137 conn(0x7fa3340c6380 msgr2=0x7fa3340c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 -- 172.21.15.67:0/1443820137 shutdown_connections 2024-02-26T10:39:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.635+0000 7fa333200640 1 -- 172.21.15.67:0/1443820137 wait complete. 2024-02-26T10:39:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.637+0000 7fa333200640 1 Processor -- start 2024-02-26T10:39:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.637+0000 7fa333200640 1 -- start start 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa333200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 0x7fa334153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa333200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3341503a0 0x7fa3341eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa333200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3340720b0 0x7fa3341eb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa333200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3340c0f80 con 0x7fa33406e2d0 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa333200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33406ddb0 con 0x7fa3341503a0 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa333200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33406e720 con 0x7fa3340720b0 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa331e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 0x7fa334153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa331e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 0x7fa334153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60538/0 (socket says 172.21.15.67:60538) 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa331e00640 1 -- 172.21.15.67:0/592555345 learned_addr learned my addr 172.21.15.67:0/592555345 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:21.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa332800640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3340720b0 0x7fa3341eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.638+0000 7fa331400640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3341503a0 0x7fa3341eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa331e00640 1 -- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3340720b0 msgr2=0x7fa3341eb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa331e00640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3340720b0 0x7fa3341eb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa331e00640 1 -- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3341503a0 msgr2=0x7fa3341eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa331e00640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3341503a0 0x7fa3341eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa331e00640 1 -- 172.21.15.67:0/592555345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3341eaab0 con 0x7fa33406e2d0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa331e00640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 0x7fa334153f10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa32400b3f0 tx=0x7fa32400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324015020 con 0x7fa33406e2d0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa324011830 con 0x7fa33406e2d0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3340bf8e0 con 0x7fa33406e2d0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.639+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324011c20 con 0x7fa33406e2d0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.640+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3340bfea0 con 0x7fa33406e2d0 2024-02-26T10:39:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.641+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3340678e0 con 0x7fa33406e2d0 2024-02-26T10:39:21.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.642+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa324018020 con 0x7fa33406e2d0 2024-02-26T10:39:21.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.642+0000 7fa32a000640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa31406b4d0 0x7fa31406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:21.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.643+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa324089ed0 con 0x7fa33406e2d0 2024-02-26T10:39:21.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.643+0000 7fa331400640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa31406b4d0 0x7fa31406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:21.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.644+0000 7fa331400640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa31406b4d0 0x7fa31406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa320036000 tx=0x7fa32003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:21.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.647+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa32401f030 con 0x7fa33406e2d0 2024-02-26T10:39:21.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.847+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa33414d6d0 con 0x7fa33406e2d0 2024-02-26T10:39:21.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.848+0000 7fa32a000640 1 -- 172.21.15.67:0/592555345 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa33414d6d0 con 0x7fa33406e2d0 2024-02-26T10:39:21.849 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa31406b4d0 msgr2=0x7fa31406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa31406b4d0 0x7fa31406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa320036000 tx=0x7fa32003c040 comp rx=0 tx=0).stop 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 msgr2=0x7fa334153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 0x7fa334153f10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa32400b3f0 tx=0x7fa32400b8b0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 shutdown_connections 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3340720b0 0x7fa3341eb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa31406b4d0 0x7fa31406d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3341503a0 0x7fa3341eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 --2- 172.21.15.67:0/592555345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa33406e2d0 0x7fa334153f10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.850+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 >> 172.21.15.67:0/592555345 conn(0x7fa3340c6380 msgr2=0x7fa334153390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.851+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 shutdown_connections 2024-02-26T10:39:21.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:21.851+0000 7fa333200640 1 -- 172.21.15.67:0/592555345 wait complete. 2024-02-26T10:39:21.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:39:22.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.101+0000 7f2e12400640 1 Processor -- start 2024-02-26T10:39:22.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.101+0000 7f2e12400640 1 -- start start 2024-02-26T10:39:22.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e12400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2e0c0c7020 0x7f2e0c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e12400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e0c06d290 0x7f2e0c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e12400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c152760 con 0x7f2e0c152380 2024-02-26T10:39:22.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e12400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c0c7550 con 0x7f2e0c0c7020 2024-02-26T10:39:22.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e12400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c0c7830 con 0x7f2e0c06d290 2024-02-26T10:39:22.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e11000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2e0c152380 0x7f2e0c155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36166/0 (socket says 172.21.15.67:36166) 2024-02-26T10:39:22.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e11000640 1 -- 172.21.15.67:0/215092370 learned_addr learned my addr 172.21.15.67:0/215092370 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:22.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e0be00640 1 --2- 172.21.15.67:0/215092370 >> v2:172.21.15.67:3300/0 conn(0x7f2e0c0c7020 0x7f2e0c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.103+0000 7f2e11a00640 1 --2- 172.21.15.67:0/215092370 >> v2:172.21.15.82:3300/0 conn(0x7f2e0c06d290 0x7f2e0c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.104+0000 7f2e0b400640 1 -- 172.21.15.67:0/215092370 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533501527 0 0) 0x7f2e0c152760 con 0x7f2e0c152380 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.104+0000 7f2e0b400640 1 -- 172.21.15.67:0/215092370 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df4003520 con 0x7f2e0c152380 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.104+0000 7f2e0be00640 1 -- 172.21.15.67:0/215092370 >> v2:172.21.15.82:3300/0 conn(0x7f2e0c06d290 msgr2=0x7f2e0c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.104+0000 7f2e0be00640 1 --2- 172.21.15.67:0/215092370 >> v2:172.21.15.82:3300/0 conn(0x7f2e0c06d290 0x7f2e0c0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.104+0000 7f2e0be00640 1 -- 172.21.15.67:0/215092370 >> v1:172.21.15.67:6789/0 conn(0x7f2e0c152380 legacy=0x7f2e0c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.104+0000 7f2e0be00640 1 -- 172.21.15.67:0/215092370 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c06ca90 con 0x7f2e0c0c7020 2024-02-26T10:39:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.105+0000 7f2e0be00640 1 --2- 172.21.15.67:0/215092370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c0c8510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c06b530 tx=0x7f2df802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9e888531b1df2d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.105+0000 7f2e0b400640 1 -- 172.21.15.67:0/215092370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80315a0 con 0x7f2e0c0c7020 2024-02-26T10:39:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.105+0000 7f2e0b400640 1 -- 172.21.15.67:0/215092370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8037070 con 0x7f2e0c0c7020 2024-02-26T10:39:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e0b400640 1 -- 172.21.15.67:0/215092370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8031940 con 0x7f2e0c0c7020 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 -- 172.21.15.67:0/215092370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 msgr2=0x7f2e0c0c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 --2- 172.21.15.67:0/215092370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c0c8510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c06b530 tx=0x7f2df802b840 comp rx=0 tx=0).stop 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 -- 172.21.15.67:0/215092370 shutdown_connections 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 --2- 172.21.15.67:0/215092370 >> v2:172.21.15.82:3300/0 conn(0x7f2e0c06d290 0x7f2e0c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 --2- 172.21.15.67:0/215092370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c0c8510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 -- 172.21.15.67:0/215092370 >> 172.21.15.67:0/215092370 conn(0x7f2e0c0ac1a0 msgr2=0x7f2e0c0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 -- 172.21.15.67:0/215092370 shutdown_connections 2024-02-26T10:39:22.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.106+0000 7f2e12400640 1 -- 172.21.15.67:0/215092370 wait complete. 2024-02-26T10:39:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.108+0000 7f2e12400640 1 Processor -- start 2024-02-26T10:39:22.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.108+0000 7f2e12400640 1 -- start start 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e12400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e0c06d290 0x7f2e0c147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e12400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e12400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c152380 0x7f2e0c0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e12400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c06d880 con 0x7f2e0c0c7020 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e12400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c06d700 con 0x7f2e0c152380 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e12400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c06da00 con 0x7f2e0c06d290 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.109+0000 7f2e0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c14d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60558/0 (socket says 172.21.15.67:60558) 2024-02-26T10:39:22.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 -- 172.21.15.67:0/3822387704 learned_addr learned my addr 172.21.15.67:0/3822387704 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e11000640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e0c06d290 0x7f2e0c147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e11a00640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c152380 0x7f2e0c0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 -- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e0c06d290 msgr2=0x7f2e0c147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e0c06d290 0x7f2e0c147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 -- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c152380 msgr2=0x7f2e0c0aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c152380 0x7f2e0c0aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e0be00640 1 -- 172.21.15.67:0/3822387704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c1eb160 con 0x7f2e0c0c7020 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.110+0000 7f2e11a00640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c152380 0x7f2e0c0aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.111+0000 7f2e0be00640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c14d9e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009890 tx=0x7f2df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.111+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8032630 con 0x7f2e0c0c7020 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.111+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80327d0 con 0x7f2e0c0c7020 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.111+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8032b80 con 0x7f2e0c0c7020 2024-02-26T10:39:22.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.111+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c0aba20 con 0x7f2e0c0c7020 2024-02-26T10:39:22.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.111+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1eb410 con 0x7f2e0c0c7020 2024-02-26T10:39:22.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.112+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c06ac20 con 0x7f2e0c0c7020 2024-02-26T10:39:22.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.113+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2df804c020 con 0x7f2e0c0c7020 2024-02-26T10:39:22.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.114+0000 7f2e08c00640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2dec06b4d0 0x7f2dec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.114+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2df802f020 con 0x7f2e0c0c7020 2024-02-26T10:39:22.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.115+0000 7f2e11000640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2dec06b4d0 0x7f2dec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.116+0000 7f2e11000640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2dec06b4d0 0x7f2dec06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0c5be0 tx=0x7f2e00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:22.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.118+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e0c06ac20 con 0x7f2e0c0c7020 2024-02-26T10:39:22.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.335+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e0c074a30 con 0x7f2e0c0c7020 2024-02-26T10:39:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.337+0000 7f2e08c00640 1 -- 172.21.15.67:0/3822387704 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2df8040070 con 0x7f2e0c0c7020 2024-02-26T10:39:22.338 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:39:22.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2dec06b4d0 msgr2=0x7f2dec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2dec06b4d0 0x7f2dec06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0c5be0 tx=0x7f2e00009040 comp rx=0 tx=0).stop 2024-02-26T10:39:22.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 msgr2=0x7f2e0c14d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c14d9e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009890 tx=0x7f2df80046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 shutdown_connections 2024-02-26T10:39:22.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2dec06b4d0 0x7f2dec06d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c152380 0x7f2e0c0aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e0c0c7020 0x7f2e0c14d9e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 --2- 172.21.15.67:0/3822387704 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e0c06d290 0x7f2e0c147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.339+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 >> 172.21.15.67:0/3822387704 conn(0x7f2e0c0ac1a0 msgr2=0x7f2e0c0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:22.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.340+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 shutdown_connections 2024-02-26T10:39:22.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.340+0000 7f2e12400640 1 -- 172.21.15.67:0/3822387704 wait complete. 2024-02-26T10:39:22.360 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:39:22.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:22.360 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-02-26T10:39:22.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:39:22.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:22.468 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:39:22.526 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-02-26T10:39:22.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:22.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.755+0000 7f700be00640 1 Processor -- start 2024-02-26T10:39:22.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.755+0000 7f700be00640 1 -- start start 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f700c06ac20 0x7f700c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f700c06db00 0x7f700c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c06b460 con 0x7f700c06b000 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0a9a80 con 0x7f700c06ac20 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0a9d60 con 0x7f700c06db00 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f700c06ac20 0x7f700c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f700c06ac20 0x7f700c0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60562/0 (socket says 172.21.15.67:60562) 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.757+0000 7f700aa00640 1 -- 172.21.15.67:0/181361297 learned_addr learned my addr 172.21.15.67:0/181361297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.758+0000 7f700b400640 1 --2- 172.21.15.67:0/181361297 >> v2:172.21.15.82:3300/0 conn(0x7f700c06db00 0x7f700c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.758+0000 7f700aa00640 1 -- 172.21.15.67:0/181361297 >> v2:172.21.15.82:3300/0 conn(0x7f700c06db00 msgr2=0x7f700c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.758+0000 7f700aa00640 1 --2- 172.21.15.67:0/181361297 >> v2:172.21.15.82:3300/0 conn(0x7f700c06db00 0x7f700c0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.758+0000 7f700aa00640 1 -- 172.21.15.67:0/181361297 >> v1:172.21.15.82:6789/0 conn(0x7f700c06b000 legacy=0x7f700c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.758+0000 7f700aa00640 1 -- 172.21.15.67:0/181361297 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c06df90 con 0x7f700c06ac20 2024-02-26T10:39:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.759+0000 7f700aa00640 1 --2- 172.21.15.67:0/181361297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06ac20 0x7f700c0a94a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f700c157220 tx=0x7f6ffc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fdd2b80635ccbf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:22.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.760+0000 7f7009600640 1 -- 172.21.15.67:0/181361297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc035020 con 0x7f700c06ac20 2024-02-26T10:39:22.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.760+0000 7f7009600640 1 -- 172.21.15.67:0/181361297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc031870 con 0x7f700c06ac20 2024-02-26T10:39:22.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.760+0000 7f7009600640 1 -- 172.21.15.67:0/181361297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc031c40 con 0x7f700c06ac20 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 -- 172.21.15.67:0/181361297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06ac20 msgr2=0x7f700c0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 --2- 172.21.15.67:0/181361297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06ac20 0x7f700c0a94a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f700c157220 tx=0x7f6ffc02b910 comp rx=0 tx=0).stop 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 -- 172.21.15.67:0/181361297 shutdown_connections 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 --2- 172.21.15.67:0/181361297 >> v2:172.21.15.82:3300/0 conn(0x7f700c06db00 0x7f700c0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 --2- 172.21.15.67:0/181361297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06ac20 0x7f700c0a94a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 -- 172.21.15.67:0/181361297 >> 172.21.15.67:0/181361297 conn(0x7f700c0c63f0 msgr2=0x7f700c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:22.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 -- 172.21.15.67:0/181361297 shutdown_connections 2024-02-26T10:39:22.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.761+0000 7f700be00640 1 -- 172.21.15.67:0/181361297 wait complete. 2024-02-26T10:39:22.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.762+0000 7f700be00640 1 Processor -- start 2024-02-26T10:39:22.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.763+0000 7f700be00640 1 -- start start 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.763+0000 7f700be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c06ac20 0x7f700c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c06b000 0x7f700c147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 0x7f700c147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0ac490 con 0x7f700c06db00 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0aba30 con 0x7f700c06b000 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c150410 con 0x7f700c06ac20 2024-02-26T10:39:22.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 0x7f700c147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 0x7f700c147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60574/0 (socket says 172.21.15.67:60574) 2024-02-26T10:39:22.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 -- 172.21.15.67:0/818777111 learned_addr learned my addr 172.21.15.67:0/818777111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700aa00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c06ac20 0x7f700c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700a000640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c06b000 0x7f700c147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 -- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c06ac20 msgr2=0x7f700c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c06ac20 0x7f700c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 -- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c06b000 msgr2=0x7f700c147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c06b000 0x7f700c147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.764+0000 7f700b400640 1 -- 172.21.15.67:0/818777111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c14a880 con 0x7f700c06db00 2024-02-26T10:39:22.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.765+0000 7f700b400640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 0x7f700c147180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7004007e60 tx=0x7f700400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.765+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004014420 con 0x7f700c06db00 2024-02-26T10:39:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.765+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7004019070 con 0x7f700c06db00 2024-02-26T10:39:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.765+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c1eac60 con 0x7f700c06db00 2024-02-26T10:39:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.765+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004014820 con 0x7f700c06db00 2024-02-26T10:39:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.765+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1eb1e0 con 0x7f700c06db00 2024-02-26T10:39:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.766+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c074a30 con 0x7f700c06db00 2024-02-26T10:39:22.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.768+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70040149c0 con 0x7f700c06db00 2024-02-26T10:39:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.768+0000 7f7002a00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fec06b4d0 0x7f6fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.769+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f700408aab0 con 0x7f700c06db00 2024-02-26T10:39:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.769+0000 7f700aa00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fec06b4d0 0x7f6fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.771+0000 7f700aa00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fec06b4d0 0x7f6fec06d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc036000 tx=0x7f6ffc03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:22.773+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7004059250 con 0x7f700c06db00 2024-02-26T10:39:23.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.014+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f700c0c6a20 con 0x7f700c06db00 2024-02-26T10:39:23.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.015+0000 7f7002a00640 1 -- 172.21.15.67:0/818777111 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f700405d2f0 con 0x7f700c06db00 2024-02-26T10:39:23.017 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:23.017 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:39:23.017 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:39:23.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fec06b4d0 msgr2=0x7f6fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fec06b4d0 0x7f6fec06d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc036000 tx=0x7f6ffc03c040 comp rx=0 tx=0).stop 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 msgr2=0x7f700c147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 0x7f700c147180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7004007e60 tx=0x7f700400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 shutdown_connections 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c06db00 0x7f700c147180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fec06b4d0 0x7f6fec06d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c06b000 0x7f700c147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 --2- 172.21.15.67:0/818777111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c06ac20 0x7f700c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 >> 172.21.15.67:0/818777111 conn(0x7f700c0c63f0 msgr2=0x7f700c14d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 shutdown_connections 2024-02-26T10:39:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.018+0000 7f700be00640 1 -- 172.21.15.67:0/818777111 wait complete. 2024-02-26T10:39:23.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:23.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.251+0000 7fc4f6c00640 1 Processor -- start 2024-02-26T10:39:23.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.251+0000 7fc4f6c00640 1 -- start start 2024-02-26T10:39:23.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f6c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f00bfa30 0x7fc4f0158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f6c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4f00c1090 0x7fc4f0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f6c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f00c15e0 con 0x7fc4f0152420 2024-02-26T10:39:23.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f6c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f01593a0 con 0x7fc4f00c1090 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f6c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f006e660 con 0x7fc4f00bfa30 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4f00c1090 0x7fc4f0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4f00c1090 0x7fc4f0158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60586/0 (socket says 172.21.15.67:60586) 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f4c00640 1 -- 172.21.15.67:0/1334418436 learned_addr learned my addr 172.21.15.67:0/1334418436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4ef400640 1 --2- 172.21.15.67:0/1334418436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f00bfa30 0x7fc4f0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f4c00640 1 -- 172.21.15.67:0/1334418436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f00bfa30 msgr2=0x7fc4f0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.253+0000 7fc4f4c00640 1 --2- 172.21.15.67:0/1334418436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f00bfa30 0x7fc4f0158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.254+0000 7fc4f4c00640 1 -- 172.21.15.67:0/1334418436 >> v1:172.21.15.82:6789/0 conn(0x7fc4f0152420 legacy=0x7fc4f00c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.254+0000 7fc4f4c00640 1 -- 172.21.15.67:0/1334418436 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f006e920 con 0x7fc4f00c1090 2024-02-26T10:39:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.255+0000 7fc4f4c00640 1 --2- 172.21.15.67:0/1334418436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00c1090 0x7fc4f0158e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009e50 tx=0x7fc4e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7615bf64433b81ec server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.255+0000 7fc4eea00640 1 -- 172.21.15.67:0/1334418436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e802ca70 con 0x7fc4f00c1090 2024-02-26T10:39:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.255+0000 7fc4eea00640 1 -- 172.21.15.67:0/1334418436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e802cc10 con 0x7fc4f00c1090 2024-02-26T10:39:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.255+0000 7fc4eea00640 1 -- 172.21.15.67:0/1334418436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80343f0 con 0x7fc4f00c1090 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 -- 172.21.15.67:0/1334418436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00c1090 msgr2=0x7fc4f0158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/1334418436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00c1090 0x7fc4f0158e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009e50 tx=0x7fc4e802bed0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 -- 172.21.15.67:0/1334418436 shutdown_connections 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/1334418436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00c1090 0x7fc4f0158e60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/1334418436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f00bfa30 0x7fc4f0158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 -- 172.21.15.67:0/1334418436 >> 172.21.15.67:0/1334418436 conn(0x7fc4f00a84a0 msgr2=0x7fc4f00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 -- 172.21.15.67:0/1334418436 shutdown_connections 2024-02-26T10:39:23.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.256+0000 7fc4f6c00640 1 -- 172.21.15.67:0/1334418436 wait complete. 2024-02-26T10:39:23.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.257+0000 7fc4f6c00640 1 Processor -- start 2024-02-26T10:39:23.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 -- start start 2024-02-26T10:39:23.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 0x7fc4f01539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f00c1090 0x7fc4f01ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f0152420 0x7fc4f00c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0158400 con 0x7fc4f00bfa30 2024-02-26T10:39:23.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00c1b60 con 0x7fc4f00c1090 2024-02-26T10:39:23.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.258+0000 7fc4f6c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0152800 con 0x7fc4f0152420 2024-02-26T10:39:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 0x7fc4f01539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 0x7fc4f01539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60590/0 (socket says 172.21.15.67:60590) 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4f4c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f0152420 0x7fc4f00c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4efe00640 1 -- 172.21.15.67:0/3218531926 learned_addr learned my addr 172.21.15.67:0/3218531926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4ef400640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f00c1090 0x7fc4f01ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4efe00640 1 -- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f0152420 msgr2=0x7fc4f00c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.259+0000 7fc4efe00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f0152420 0x7fc4f00c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4efe00640 1 -- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f00c1090 msgr2=0x7fc4f01ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4efe00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f00c1090 0x7fc4f01ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4efe00640 1 -- 172.21.15.67:0/3218531926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0152d20 con 0x7fc4f00bfa30 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4efe00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 0x7fc4f01539a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c430 tx=0x7fc4e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00052d0 con 0x7fc4f00bfa30 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e0005490 con 0x7fc4f00bfa30 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f01eeb20 con 0x7fc4f00bfa30 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0007260 con 0x7fc4f00bfa30 2024-02-26T10:39:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.260+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f00c3860 con 0x7fc4f00bfa30 2024-02-26T10:39:23.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.262+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f0074a30 con 0x7fc4f00bfa30 2024-02-26T10:39:23.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.262+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4e0007400 con 0x7fc4f00bfa30 2024-02-26T10:39:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.264+0000 7fc4e7e00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc4d006b500 0x7fc4d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.264+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc4e0089760 con 0x7fc4f00bfa30 2024-02-26T10:39:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.264+0000 7fc4ef400640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc4d006b500 0x7fc4d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.266+0000 7fc4ef400640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc4d006b500 0x7fc4d006d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0159120 tx=0x7fc4dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.268+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4e0057e70 con 0x7fc4f00bfa30 2024-02-26T10:39:23.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.467+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4f006e440 con 0x7fc4f00bfa30 2024-02-26T10:39:23.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.469+0000 7fc4e7e00640 1 -- 172.21.15.67:0/3218531926 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16782 (secure 0 0 0) 0x7fc4e005bf10 con 0x7fc4f00bfa30 2024-02-26T10:39:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:23.471 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":23,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:21.453133+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc4d006b500 msgr2=0x7fc4d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc4d006b500 0x7fc4d006d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0159120 tx=0x7fc4dc008040 comp rx=0 tx=0).stop 2024-02-26T10:39:23.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 msgr2=0x7fc4f01539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 0x7fc4f01539a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c430 tx=0x7fc4e000c8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 shutdown_connections 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc4f0152420 0x7fc4f00c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc4d006b500 0x7fc4d006d9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f00c1090 0x7fc4f01ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 --2- 172.21.15.67:0/3218531926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4f00bfa30 0x7fc4f01539a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 >> 172.21.15.67:0/3218531926 conn(0x7fc4f00a84a0 msgr2=0x7fc4f00a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 shutdown_connections 2024-02-26T10:39:23.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.472+0000 7fc4f6c00640 1 -- 172.21.15.67:0/3218531926 wait complete. 2024-02-26T10:39:23.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:38:32.893594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:38:33.645319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:23.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.716+0000 7f31de200640 1 Processor -- start 2024-02-26T10:39:23.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.716+0000 7f31de200640 1 -- start start 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.717+0000 7f31de200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f31d81562f0 0x7f31d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.717+0000 7f31de200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31d806ce30 0x7f31d80af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.717+0000 7f31de200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d806c110 con 0x7f31d80c5290 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.717+0000 7f31de200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d80c72c0 con 0x7f31d81562f0 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31de200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80c75a0 con 0x7f31d806ce30 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31dce00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f31d80c5290 0x7f31d8155880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36172/0 (socket says 172.21.15.67:36172) 2024-02-26T10:39:23.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31dce00640 1 -- 172.21.15.67:0/693797604 learned_addr learned my addr 172.21.15.67:0/693797604 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:23.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31d7e00640 1 --2- 172.21.15.67:0/693797604 >> v2:172.21.15.82:3300/0 conn(0x7f31d81562f0 0x7f31d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31dd800640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31d806ce30 0x7f31d80af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948154552 0 0) 0x7f31d806c110 con 0x7f31d80c5290 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.718+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31bc003520 con 0x7f31d80c5290 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286038019 0 0) 0x7f31bc003520 con 0x7f31d80c5290 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31d806ce30 msgr2=0x7f31d80af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31d806ce30 0x7f31d80af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 >> v2:172.21.15.82:3300/0 conn(0x7f31d81562f0 msgr2=0x7f31d806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 --2- 172.21.15.67:0/693797604 >> v2:172.21.15.82:3300/0 conn(0x7f31d81562f0 0x7f31d806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80c7880 con 0x7f31d80c5290 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f31c0001670 con 0x7f31d80c5290 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 >> v1:172.21.15.67:6789/0 conn(0x7f31d80c5290 legacy=0x7f31d8155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31bc0039b0 0x7f31bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31bc004800 0x7f31bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc0051c0 0x7f31bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80c72c0 con 0x7f31bc004800 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80c75a0 con 0x7f31bc0039b0 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc00bf00 con 0x7f31bc0051c0 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.719+0000 7f31d7e00640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31bc004800 0x7f31bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31dce00640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31bc0039b0 0x7f31bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31dd800640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc0051c0 0x7f31bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31d7e00640 1 -- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc0051c0 msgr2=0x7f31bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31d7e00640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc0051c0 0x7f31bc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31d7e00640 1 -- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31bc0039b0 msgr2=0x7f31bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31d7e00640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31bc0039b0 0x7f31bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31d7e00640 1 -- 172.21.15.67:0/693797604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc003520 con 0x7f31bc004800 2024-02-26T10:39:23.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.720+0000 7f31dce00640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31bc0039b0 0x7f31bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.721+0000 7f31d7e00640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31bc004800 0x7f31bc004c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fc0 tx=0x7f31bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0992b7c0d8a5be4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.721+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8039070 con 0x7f31bc004800 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.721+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c802cc40 con 0x7f31bc004800 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.721+0000 7f31d7400640 1 -- 172.21.15.67:0/693797604 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8009c10 con 0x7f31bc004800 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31de200640 1 -- 172.21.15.67:0/693797604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31bc004800 msgr2=0x7f31bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31de200640 1 --2- 172.21.15.67:0/693797604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31bc004800 0x7f31bc004c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fc0 tx=0x7f31bc005830 comp rx=0 tx=0).stop 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31dd800640 1 -- 172.21.15.67:0/693797604 reap_dead start 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31de200640 1 -- 172.21.15.67:0/693797604 shutdown_connections 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31de200640 1 -- 172.21.15.67:0/693797604 >> 172.21.15.67:0/693797604 conn(0x7f31d80ac230 msgr2=0x7f31d80ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31de200640 1 -- 172.21.15.67:0/693797604 shutdown_connections 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.722+0000 7f31de200640 1 -- 172.21.15.67:0/693797604 wait complete. 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.724+0000 7f31de200640 1 Processor -- start 2024-02-26T10:39:23.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.724+0000 7f31de200640 1 -- start start 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31de200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc004800 0x7f31d8158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31de200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 0x7f31d81eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31de200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d81ebbe0 0x7f31d814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31de200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80c57f0 con 0x7f31d81ee420 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31de200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80c5670 con 0x7f31d81ebbe0 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31de200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80bddc0 con 0x7f31bc004800 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 0x7f31d81eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 0x7f31d81eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60604/0 (socket says 172.21.15.67:60604) 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 -- 172.21.15.67:0/1138354378 learned_addr learned my addr 172.21.15.67:0/1138354378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31dce00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc004800 0x7f31d8158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31dd800640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d81ebbe0 0x7f31d814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 -- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc004800 msgr2=0x7f31d8158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc004800 0x7f31d8158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 -- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d81ebbe0 msgr2=0x7f31d814ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d81ebbe0 0x7f31d814ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31d7e00640 1 -- 172.21.15.67:0/1138354378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d81ea340 con 0x7f31d81ee420 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.725+0000 7f31dce00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc004800 0x7f31d8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.726+0000 7f31d7e00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 0x7f31d81eb6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31c802c710 tx=0x7f31c8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.726+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8039050 con 0x7f31d81ee420 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.726+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c803a040 con 0x7f31d81ee420 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.726+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d81ed940 con 0x7f31d81ee420 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.726+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8003990 con 0x7f31d81ee420 2024-02-26T10:39:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.726+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d81edf00 con 0x7f31d81ee420 2024-02-26T10:39:23.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.728+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d8074a30 con 0x7f31d81ee420 2024-02-26T10:39:23.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.728+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31c8047030 con 0x7f31d81ee420 2024-02-26T10:39:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.730+0000 7f31d4c00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f31b406b4d0 0x7f31b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.730+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f31c80ac6b0 con 0x7f31d81ee420 2024-02-26T10:39:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.730+0000 7f31dce00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f31b406b4d0 0x7f31b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.732+0000 7f31dce00640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f31b406b4d0 0x7f31b406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31c0007970 tx=0x7f31c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:23.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.734+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31c804e020 con 0x7f31d81ee420 2024-02-26T10:39:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.936+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31d80c4ad0 con 0x7f31d81ee420 2024-02-26T10:39:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.936+0000 7f31d4c00640 1 -- 172.21.15.67:0/1138354378 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31c807b5c0 con 0x7f31d81ee420 2024-02-26T10:39:23.938 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f31b406b4d0 msgr2=0x7f31b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f31b406b4d0 0x7f31b406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31c0007970 tx=0x7f31c0009040 comp rx=0 tx=0).stop 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 msgr2=0x7f31d81eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 0x7f31d81eb6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31c802c710 tx=0x7f31c8009750 comp rx=0 tx=0).stop 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 shutdown_connections 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f31b406b4d0 0x7f31b406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d81ebbe0 0x7f31d814ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31d81ee420 0x7f31d81eb6a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 --2- 172.21.15.67:0/1138354378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31bc004800 0x7f31d8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.939+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 >> 172.21.15.67:0/1138354378 conn(0x7f31d80ac230 msgr2=0x7f31d80ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.940+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 shutdown_connections 2024-02-26T10:39:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:23.940+0000 7f31de200640 1 -- 172.21.15.67:0/1138354378 wait complete. 2024-02-26T10:39:23.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.172+0000 7f9b27e00640 1 Processor -- start 2024-02-26T10:39:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.172+0000 7f9b27e00640 1 -- start start 2024-02-26T10:39:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.174+0000 7f9b27e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b201503d0 0x7f9b2006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.174+0000 7f9b27e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.174+0000 7f9b27e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b200a6eb0 con 0x7f9b200c2950 2024-02-26T10:39:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.174+0000 7f9b27e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200a7190 con 0x7f9b201503d0 2024-02-26T10:39:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.174+0000 7f9b27e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200a7470 con 0x7f9b2006e390 2024-02-26T10:39:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b201503d0 0x7f9b2006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b24a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b201503d0 0x7f9b2006de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48714/0 (socket says 172.21.15.67:48714) 2024-02-26T10:39:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 -- 172.21.15.67:0/2416544741 learned_addr learned my addr 172.21.15.67:0/2416544741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 -- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 msgr2=0x7f9b201533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 --2- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 -- 172.21.15.67:0/2416544741 >> v1:172.21.15.82:6789/0 conn(0x7f9b200c2950 legacy=0x7f9b20156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.175+0000 7f9b25400640 1 -- 172.21.15.67:0/2416544741 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b200a7750 con 0x7f9b201503d0 2024-02-26T10:39:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.176+0000 7f9b24a00640 1 --2- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:39:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.177+0000 7f9b25400640 1 --2- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b201503d0 0x7f9b2006de50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009870 tx=0x7f9b1402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaff0b28f072f809 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.177+0000 7f9b1fe00640 1 -- 172.21.15.67:0/2416544741 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b140317c0 con 0x7f9b201503d0 2024-02-26T10:39:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.177+0000 7f9b1fe00640 1 -- 172.21.15.67:0/2416544741 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b14037070 con 0x7f9b201503d0 2024-02-26T10:39:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.177+0000 7f9b1fe00640 1 -- 172.21.15.67:0/2416544741 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14031b80 con 0x7f9b201503d0 2024-02-26T10:39:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 -- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b201503d0 msgr2=0x7f9b2006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b201503d0 0x7f9b2006de50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009870 tx=0x7f9b1402b860 comp rx=0 tx=0).stop 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 -- 172.21.15.67:0/2416544741 shutdown_connections 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2416544741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b201503d0 0x7f9b2006de50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 -- 172.21.15.67:0/2416544741 >> 172.21.15.67:0/2416544741 conn(0x7f9b200c6da0 msgr2=0x7f9b200c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 -- 172.21.15.67:0/2416544741 shutdown_connections 2024-02-26T10:39:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.178+0000 7f9b27e00640 1 -- 172.21.15.67:0/2416544741 wait complete. 2024-02-26T10:39:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.180+0000 7f9b27e00640 1 Processor -- start 2024-02-26T10:39:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.181+0000 7f9b27e00640 1 -- start start 2024-02-26T10:39:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b27e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201eb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b27e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b200c2950 0x7f9b200c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 0x7f9b20152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b27e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b201507b0 con 0x7f9b201503d0 2024-02-26T10:39:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b27e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2006ef00 con 0x7f9b200c2950 2024-02-26T10:39:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b27e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200c0ef0 con 0x7f9b2006e390 2024-02-26T10:39:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.182+0000 7f9b25e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 0x7f9b20152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 0x7f9b20152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60616/0 (socket says 172.21.15.67:60616) 2024-02-26T10:39:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 -- 172.21.15.67:0/2385440899 learned_addr learned my addr 172.21.15.67:0/2385440899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b24a00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b200c2950 0x7f9b200c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25400640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 -- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 msgr2=0x7f9b201eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201eb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 -- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b200c2950 msgr2=0x7f9b200c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b200c2950 0x7f9b200c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 -- 172.21.15.67:0/2385440899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b201eaef0 con 0x7f9b201503d0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.183+0000 7f9b25e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 0x7f9b20152360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800c7a0 tx=0x7f9b1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.184+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18003b70 con 0x7f9b201503d0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.184+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b18003d30 con 0x7f9b201503d0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.184+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800aa90 con 0x7f9b201503d0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.184+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b201eef00 con 0x7f9b201503d0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.184+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b201ef1e0 con 0x7f9b201503d0 2024-02-26T10:39:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.185+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b20150930 con 0x7f9b201503d0 2024-02-26T10:39:24.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.186+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b1800ac30 con 0x7f9b201503d0 2024-02-26T10:39:24.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.187+0000 7f9b1d600640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9b0006b4d0 0x7f9b0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.187+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9b1808a1e0 con 0x7f9b201503d0 2024-02-26T10:39:24.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.187+0000 7f9b25400640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9b0006b4d0 0x7f9b0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.188+0000 7f9b25400640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9b0006b4d0 0x7f9b0006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b14036000 tx=0x7f9b1403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:24.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.191+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b20150930 con 0x7f9b201503d0 2024-02-26T10:39:24.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.394+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b20074a30 con 0x7f9b201503d0 2024-02-26T10:39:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.394+0000 7f9b1d600640 1 -- 172.21.15.67:0/2385440899 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b18058fd0 con 0x7f9b201503d0 2024-02-26T10:39:24.396 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9b0006b4d0 msgr2=0x7f9b0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9b0006b4d0 0x7f9b0006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b14036000 tx=0x7f9b1403d040 comp rx=0 tx=0).stop 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 msgr2=0x7f9b20152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 0x7f9b20152360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800c7a0 tx=0x7f9b1800cc60 comp rx=0 tx=0).stop 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 shutdown_connections 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201503d0 0x7f9b20152360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9b0006b4d0 0x7f9b0006d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b200c2950 0x7f9b200c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 --2- 172.21.15.67:0/2385440899 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b2006e390 0x7f9b201eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.397+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 >> 172.21.15.67:0/2385440899 conn(0x7f9b200c6da0 msgr2=0x7f9b20153230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.398+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 shutdown_connections 2024-02-26T10:39:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.398+0000 7f9b27e00640 1 -- 172.21.15.67:0/2385440899 wait complete. 2024-02-26T10:39:24.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:24.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.643+0000 7fe70ea00640 1 Processor -- start 2024-02-26T10:39:24.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.644+0000 7fe70ea00640 1 -- start start 2024-02-26T10:39:24.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7081562f0 0x7fe70806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe70806ce30 0x7fe7080af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70806c110 con 0x7fe7080c5290 2024-02-26T10:39:24.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7080c72c0 con 0x7fe7081562f0 2024-02-26T10:39:24.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70ea00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7080c75a0 con 0x7fe70806ce30 2024-02-26T10:39:24.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70d600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe7080c5290 0x7fe708155880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36184/0 (socket says 172.21.15.67:36184) 2024-02-26T10:39:24.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70d600640 1 -- 172.21.15.67:0/1728665851 learned_addr learned my addr 172.21.15.67:0/1728665851 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:24.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70cc00640 1 --2- 172.21.15.67:0/1728665851 >> v2:172.21.15.67:3300/0 conn(0x7fe7081562f0 0x7fe70806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe70e000640 1 --2- 172.21.15.67:0/1728665851 >> v2:172.21.15.82:3300/0 conn(0x7fe70806ce30 0x7fe7080af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe707e00640 1 -- 172.21.15.67:0/1728665851 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562026760 0 0) 0x7fe70806c110 con 0x7fe7080c5290 2024-02-26T10:39:24.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.645+0000 7fe707e00640 1 -- 172.21.15.67:0/1728665851 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f0003520 con 0x7fe7080c5290 2024-02-26T10:39:24.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.646+0000 7fe70cc00640 1 -- 172.21.15.67:0/1728665851 >> v2:172.21.15.82:3300/0 conn(0x7fe70806ce30 msgr2=0x7fe7080af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.646+0000 7fe70cc00640 1 --2- 172.21.15.67:0/1728665851 >> v2:172.21.15.82:3300/0 conn(0x7fe70806ce30 0x7fe7080af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.646+0000 7fe70cc00640 1 -- 172.21.15.67:0/1728665851 >> v1:172.21.15.67:6789/0 conn(0x7fe7080c5290 legacy=0x7fe708155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.646+0000 7fe70cc00640 1 -- 172.21.15.67:0/1728665851 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080c7880 con 0x7fe7081562f0 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.647+0000 7fe70cc00640 1 --2- 172.21.15.67:0/1728665851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe70806c8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe708159490 tx=0x7fe6f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b213dae5a32636 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.647+0000 7fe707e00640 1 -- 172.21.15.67:0/1728665851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80315a0 con 0x7fe7081562f0 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.647+0000 7fe707e00640 1 -- 172.21.15.67:0/1728665851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8037070 con 0x7fe7081562f0 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.647+0000 7fe707e00640 1 -- 172.21.15.67:0/1728665851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8031960 con 0x7fe7081562f0 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 -- 172.21.15.67:0/1728665851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 msgr2=0x7fe70806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1728665851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe70806c8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe708159490 tx=0x7fe6f802b840 comp rx=0 tx=0).stop 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 -- 172.21.15.67:0/1728665851 shutdown_connections 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1728665851 >> v2:172.21.15.82:3300/0 conn(0x7fe70806ce30 0x7fe7080af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1728665851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe70806c8f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 -- 172.21.15.67:0/1728665851 >> 172.21.15.67:0/1728665851 conn(0x7fe7080ac230 msgr2=0x7fe7080ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:24.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 -- 172.21.15.67:0/1728665851 shutdown_connections 2024-02-26T10:39:24.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.648+0000 7fe70ea00640 1 -- 172.21.15.67:0/1728665851 wait complete. 2024-02-26T10:39:24.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.649+0000 7fe70ea00640 1 Processor -- start 2024-02-26T10:39:24.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.650+0000 7fe70ea00640 1 -- start start 2024-02-26T10:39:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.650+0000 7fe70ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7080c5290 0x7fe7080c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe7080c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7080c3960 0x7fe70814eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080c57f0 con 0x7fe7081562f0 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080c5670 con 0x7fe7080c5290 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080bddc0 con 0x7fe7080c3960 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe7080c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe7080c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60620/0 (socket says 172.21.15.67:60620) 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 -- 172.21.15.67:0/1051272946 learned_addr learned my addr 172.21.15.67:0/1051272946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 -- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7080c3960 msgr2=0x7fe70814eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7080c3960 0x7fe70814eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 -- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7080c5290 msgr2=0x7fe7080c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7080c5290 0x7fe7080c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.651+0000 7fe70cc00640 1 -- 172.21.15.67:0/1051272946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080c3f10 con 0x7fe7081562f0 2024-02-26T10:39:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.652+0000 7fe70cc00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe7080c85b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009890 tx=0x7fe6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.652+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8032630 con 0x7fe7081562f0 2024-02-26T10:39:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.652+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f80327d0 con 0x7fe7081562f0 2024-02-26T10:39:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.652+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8032bc0 con 0x7fe7081562f0 2024-02-26T10:39:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.652+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080c41f0 con 0x7fe7081562f0 2024-02-26T10:39:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.652+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7080c2130 con 0x7fe7081562f0 2024-02-26T10:39:24.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.653+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe708074a30 con 0x7fe7081562f0 2024-02-26T10:39:24.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.654+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6f804c020 con 0x7fe7081562f0 2024-02-26T10:39:24.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.655+0000 7fe705600640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe6ec06b4d0 0x7fe6ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:24.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.655+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe6f802f020 con 0x7fe7081562f0 2024-02-26T10:39:24.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.656+0000 7fe70d600640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe6ec06b4d0 0x7fe6ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:24.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.657+0000 7fe70d600640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe6ec06b4d0 0x7fe6ec06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc000f60 tx=0x7fe6fc0078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:24.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.660+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6f8040070 con 0x7fe7081562f0 2024-02-26T10:39:24.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.862+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe70806d950 con 0x7fe7081562f0 2024-02-26T10:39:24.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.863+0000 7fe705600640 1 -- 172.21.15.67:0/1051272946 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6f807ac60 con 0x7fe7081562f0 2024-02-26T10:39:24.864 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.865+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe6ec06b4d0 msgr2=0x7fe6ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.865+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe6ec06b4d0 0x7fe6ec06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc000f60 tx=0x7fe6fc0078f0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.865+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 msgr2=0x7fe7080c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.865+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe7080c85b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009890 tx=0x7fe6f80046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.865+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 shutdown_connections 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7080c3960 0x7fe70814eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7081562f0 0x7fe7080c85b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe6ec06b4d0 0x7fe6ec06d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 --2- 172.21.15.67:0/1051272946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7080c5290 0x7fe7080c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 >> 172.21.15.67:0/1051272946 conn(0x7fe7080ac230 msgr2=0x7fe7080c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 shutdown_connections 2024-02-26T10:39:24.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:24.866+0000 7fe70ea00640 1 -- 172.21.15.67:0/1051272946 wait complete. 2024-02-26T10:39:24.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.110+0000 7f2852c00640 1 Processor -- start 2024-02-26T10:39:25.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.110+0000 7f2852c00640 1 -- start start 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.111+0000 7f2852c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f284c0c2310 0x7f284c153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f2852c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c155f10 con 0x7f284c06ac20 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f2852c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c0c28d0 con 0x7f284c06b000 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f2852c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c06b430 con 0x7f284c0c2310 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f2851800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f284c06ac20 0x7f284c152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36186/0 (socket says 172.21.15.67:36186) 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f2851800640 1 -- 172.21.15.67:0/2697830191 learned_addr learned my addr 172.21.15.67:0/2697830191 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:25.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f2850e00640 1 --2- 172.21.15.67:0/2697830191 >> v2:172.21.15.67:3300/0 conn(0x7f284c0c2310 0x7f284c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f284be00640 1 -- 172.21.15.67:0/2697830191 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081420649 0 0) 0x7f284c155f10 con 0x7f284c06ac20 2024-02-26T10:39:25.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.112+0000 7f284be00640 1 -- 172.21.15.67:0/2697830191 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2834005730 con 0x7f284c06ac20 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.113+0000 7f2850e00640 1 -- 172.21.15.67:0/2697830191 >> v1:172.21.15.82:6789/0 conn(0x7f284c06b000 legacy=0x7f284c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.113+0000 7f2850e00640 1 -- 172.21.15.67:0/2697830191 >> v1:172.21.15.67:6789/0 conn(0x7f284c06ac20 legacy=0x7f284c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.113+0000 7f2850e00640 1 -- 172.21.15.67:0/2697830191 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c155150 con 0x7f284c0c2310 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.114+0000 7f2850e00640 1 --2- 172.21.15.67:0/2697830191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c0c2310 0x7f284c153630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28400097a0 tx=0x7f284002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42266d86c95ee2b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.114+0000 7f284be00640 1 -- 172.21.15.67:0/2697830191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400315a0 con 0x7f284c0c2310 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.114+0000 7f284be00640 1 -- 172.21.15.67:0/2697830191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840037070 con 0x7f284c0c2310 2024-02-26T10:39:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.114+0000 7f284be00640 1 -- 172.21.15.67:0/2697830191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840031960 con 0x7f284c0c2310 2024-02-26T10:39:25.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 -- 172.21.15.67:0/2697830191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c0c2310 msgr2=0x7f284c153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 --2- 172.21.15.67:0/2697830191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c0c2310 0x7f284c153630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28400097a0 tx=0x7f284002b840 comp rx=0 tx=0).stop 2024-02-26T10:39:25.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 -- 172.21.15.67:0/2697830191 shutdown_connections 2024-02-26T10:39:25.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 --2- 172.21.15.67:0/2697830191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c0c2310 0x7f284c153630 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 -- 172.21.15.67:0/2697830191 >> 172.21.15.67:0/2697830191 conn(0x7f284c0c63f0 msgr2=0x7f284c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:25.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 -- 172.21.15.67:0/2697830191 shutdown_connections 2024-02-26T10:39:25.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.115+0000 7f2852c00640 1 -- 172.21.15.67:0/2697830191 wait complete. 2024-02-26T10:39:25.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.116+0000 7f2852c00640 1 Processor -- start 2024-02-26T10:39:25.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 -- start start 2024-02-26T10:39:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f284c06ac20 0x7f284c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 0x7f284c151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f284c0c2310 0x7f284c141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c06e1b0 con 0x7f284c06b000 2024-02-26T10:39:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c06e030 con 0x7f284c0c2310 2024-02-26T10:39:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.117+0000 7f2852c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c06e330 con 0x7f284c06ac20 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 0x7f284c151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 0x7f284c151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60638/0 (socket says 172.21.15.67:60638) 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 -- 172.21.15.67:0/2404255098 learned_addr learned my addr 172.21.15.67:0/2404255098 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 -- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f284c06ac20 msgr2=0x7f284c14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f284c06ac20 0x7f284c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 -- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f284c0c2310 msgr2=0x7f284c141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f284c0c2310 0x7f284c141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 -- 172.21.15.67:0/2404255098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c1ddec0 con 0x7f284c06b000 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.118+0000 7f2850e00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 0x7f284c151880 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f28400098c0 tx=0x7f28400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.119+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840032630 con 0x7f284c06b000 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.119+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28400327f0 con 0x7f284c06b000 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.119+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c14fda0 con 0x7f284c06b000 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.119+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840032be0 con 0x7f284c06b000 2024-02-26T10:39:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.119+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c152de0 con 0x7f284c06b000 2024-02-26T10:39:25.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.120+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c074a30 con 0x7f284c06b000 2024-02-26T10:39:25.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.121+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f284004e430 con 0x7f284c06b000 2024-02-26T10:39:25.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.122+0000 7f2849600640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f283006b4d0 0x7f283006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.122+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f284002f030 con 0x7f284c06b000 2024-02-26T10:39:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.123+0000 7f2851800640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f283006b4d0 0x7f283006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.124+0000 7f2851800640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f283006b4d0 0x7f283006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f283c001610 tx=0x7f283c007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.127+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2840040070 con 0x7f284c06b000 2024-02-26T10:39:25.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.364+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f284c155420 con 0x7f284c06b000 2024-02-26T10:39:25.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.365+0000 7f2849600640 1 -- 172.21.15.67:0/2404255098 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f284007af60 con 0x7f284c06b000 2024-02-26T10:39:25.367 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:25.367 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:39:25.367 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f283006b4d0 msgr2=0x7f283006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f283006b4d0 0x7f283006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f283c001610 tx=0x7f283c007930 comp rx=0 tx=0).stop 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 msgr2=0x7f284c151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 0x7f284c151880 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f28400098c0 tx=0x7f28400046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 shutdown_connections 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f283006b4d0 0x7f283006d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f284c0c2310 0x7f284c141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f284c06b000 0x7f284c151880 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 --2- 172.21.15.67:0/2404255098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f284c06ac20 0x7f284c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.368+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 >> 172.21.15.67:0/2404255098 conn(0x7f284c0c63f0 msgr2=0x7f284c06e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.369+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 shutdown_connections 2024-02-26T10:39:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.369+0000 7f2852c00640 1 -- 172.21.15.67:0/2404255098 wait complete. 2024-02-26T10:39:25.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:25.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.611+0000 7f1198600640 1 Processor -- start 2024-02-26T10:39:25.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.611+0000 7f1198600640 1 -- start start 2024-02-26T10:39:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1198600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11900c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1198600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f119006e410 0x7f11900c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1198600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1190152e50 con 0x7f1190156230 2024-02-26T10:39:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1198600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11901506e0 con 0x7f119006e410 2024-02-26T10:39:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1198600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1190153a60 con 0x7f1190150300 2024-02-26T10:39:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1196600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1190156230 0x7f11900aaba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36192/0 (socket says 172.21.15.67:36192) 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1196600640 1 -- 172.21.15.67:0/2952167225 learned_addr learned my addr 172.21.15.67:0/2952167225 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1195200640 1 --2- 172.21.15.67:0/2952167225 >> v2:172.21.15.82:3300/0 conn(0x7f119006e410 0x7f11900c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.613+0000 7f1195c00640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11900c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958359486 0 0) 0x7f1190152e50 con 0x7f1190156230 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1178003520 con 0x7f1190156230 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 37876927 0 0) 0x7f1178003520 con 0x7f1190156230 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 msgr2=0x7f11900c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11900c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 >> v2:172.21.15.82:3300/0 conn(0x7f119006e410 msgr2=0x7f11900c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 --2- 172.21.15.67:0/2952167225 >> v2:172.21.15.82:3300/0 conn(0x7f119006e410 0x7f11900c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190153130 con 0x7f1190156230 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f11880019f0 con 0x7f1190156230 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 >> v1:172.21.15.67:6789/0 conn(0x7f1190156230 legacy=0x7f11900aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11780039b0 0x7f1178003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1178004800 0x7f1178004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11901506e0 con 0x7f1178004800 2024-02-26T10:39:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1190153a60 con 0x7f11780039b0 2024-02-26T10:39:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.614+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117800ba40 con 0x7f1190150300 2024-02-26T10:39:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195200640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1178004800 0x7f1178004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195c00640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11780039b0 0x7f1178003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1196600640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195200640 1 -- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 msgr2=0x7f11780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195200640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195200640 1 -- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11780039b0 msgr2=0x7f1178003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195200640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11780039b0 0x7f1178003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1195200640 1 -- 172.21.15.67:0/2952167225 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178003520 con 0x7f1178004800 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.615+0000 7f1196600640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1190150300 0x7f11780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.616+0000 7f1195200640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1178004800 0x7f1178004c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f11900c3a70 tx=0x7f1178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05db906d0c88110 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.616+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c039070 con 0x7f1178004800 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c02c680 con 0x7f1178004800 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f118fe00640 1 -- 172.21.15.67:0/2952167225 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c009690 con 0x7f1178004800 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1198600640 1 -- 172.21.15.67:0/2952167225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1178004800 msgr2=0x7f1178004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1198600640 1 --2- 172.21.15.67:0/2952167225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1178004800 0x7f1178004c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f11900c3a70 tx=0x7f1178005830 comp rx=0 tx=0).stop 2024-02-26T10:39:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1196600640 1 -- 172.21.15.67:0/2952167225 reap_dead start 2024-02-26T10:39:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1198600640 1 -- 172.21.15.67:0/2952167225 shutdown_connections 2024-02-26T10:39:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1198600640 1 -- 172.21.15.67:0/2952167225 >> 172.21.15.67:0/2952167225 conn(0x7f11900a84a0 msgr2=0x7f11900a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1198600640 1 -- 172.21.15.67:0/2952167225 shutdown_connections 2024-02-26T10:39:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.617+0000 7f1198600640 1 -- 172.21.15.67:0/2952167225 wait complete. 2024-02-26T10:39:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.619+0000 7f1198600640 1 Processor -- start 2024-02-26T10:39:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.619+0000 7f1198600640 1 -- start start 2024-02-26T10:39:25.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.620+0000 7f1198600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1178004800 0x7f11901edc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.620+0000 7f1198600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11900c4c00 0x7f11900bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.620+0000 7f1198600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 0x7f11901ea700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.620+0000 7f1198600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119006dd80 con 0x7f11900c53d0 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.620+0000 7f1198600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11901536f0 con 0x7f1178004800 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.620+0000 7f1198600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11900c2ff0 con 0x7f11900c4c00 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 0x7f11901ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 0x7f11901ea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60668/0 (socket says 172.21.15.67:60668) 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 -- 172.21.15.67:0/327444628 learned_addr learned my addr 172.21.15.67:0/327444628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1195c00640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1178004800 0x7f11901edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1195200640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11900c4c00 0x7f11900bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 -- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11900c4c00 msgr2=0x7f11900bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11900c4c00 0x7f11900bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 -- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1178004800 msgr2=0x7f11901edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1178004800 0x7f11901edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 -- 172.21.15.67:0/327444628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11901ee1b0 con 0x7f11900c53d0 2024-02-26T10:39:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1195200640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11900c4c00 0x7f11900bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.621+0000 7f1196600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 0x7f11901ea700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1188000f80 tx=0x7f118800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.622+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188004d60 con 0x7f11900c53d0 2024-02-26T10:39:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.622+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118800bb20 con 0x7f11900c53d0 2024-02-26T10:39:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.622+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188007110 con 0x7f11900c53d0 2024-02-26T10:39:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.622+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11900c0a10 con 0x7f11900c53d0 2024-02-26T10:39:25.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.622+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1190072de0 con 0x7f11900c53d0 2024-02-26T10:39:25.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.623+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1190074a30 con 0x7f11900c53d0 2024-02-26T10:39:25.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.624+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f118801b020 con 0x7f11900c53d0 2024-02-26T10:39:25.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.625+0000 7f118d600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f117006b500 0x7f117006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:25.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.625+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f118808a350 con 0x7f11900c53d0 2024-02-26T10:39:25.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.625+0000 7f1195c00640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f117006b500 0x7f117006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.626+0000 7f1195c00640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f117006b500 0x7f117006d9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f11900c75e0 tx=0x7f1184008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.629+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11880591e0 con 0x7f11900c53d0 2024-02-26T10:39:25.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.828+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f119006e0d0 con 0x7f11900c53d0 2024-02-26T10:39:25.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.830+0000 7f118d600640 1 -- 172.21.15.67:0/327444628 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f1188058ae0 con 0x7f11900c53d0 2024-02-26T10:39:25.832 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:25.832 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":23,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:21.453133+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f117006b500 msgr2=0x7f117006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f117006b500 0x7f117006d9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f11900c75e0 tx=0x7f1184008040 comp rx=0 tx=0).stop 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 msgr2=0x7f11901ea700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 0x7f11901ea700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1188000f80 tx=0x7f118800b4a0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 shutdown_connections 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11900c53d0 0x7f11901ea700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11900c4c00 0x7f11900bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f117006b500 0x7f117006d9c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 --2- 172.21.15.67:0/327444628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1178004800 0x7f11901edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 >> 172.21.15.67:0/327444628 conn(0x7f11900a84a0 msgr2=0x7f119014e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 shutdown_connections 2024-02-26T10:39:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:25.833+0000 7f1198600640 1 -- 172.21.15.67:0/327444628 wait complete. 2024-02-26T10:39:25.853 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T10:39:25.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:26.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.071+0000 7f4643200640 1 Processor -- start 2024-02-26T10:39:26.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.071+0000 7f4643200640 1 -- start start 2024-02-26T10:39:26.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4643200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46440a94a0 0x7f464406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4643200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f464406db00 0x7f46440abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4643200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464406df90 con 0x7f464406ac20 2024-02-26T10:39:26.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4643200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46440a9a80 con 0x7f464406db00 2024-02-26T10:39:26.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4643200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46440a9d60 con 0x7f46440a94a0 2024-02-26T10:39:26.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4642800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f464406db00 0x7f46440abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f463be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46440a94a0 0x7f464406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4642800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f464406db00 0x7f46440abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48794/0 (socket says 172.21.15.67:48794) 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4642800640 1 -- 172.21.15.67:0/2827183636 learned_addr learned my addr 172.21.15.67:0/2827183636 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.073+0000 7f4641e00640 1 --1- 172.21.15.67:0/2827183636 >> v1:172.21.15.82:6789/0 conn(0x7f464406ac20 0x7f4644153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:37176/0 (socket says 172.21.15.67:37176) 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.074+0000 7f463be00640 1 -- 172.21.15.67:0/2827183636 >> v2:172.21.15.82:3300/0 conn(0x7f464406db00 msgr2=0x7f46440abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.074+0000 7f463be00640 1 --2- 172.21.15.67:0/2827183636 >> v2:172.21.15.82:3300/0 conn(0x7f464406db00 0x7f46440abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.074+0000 7f463be00640 1 -- 172.21.15.67:0/2827183636 >> v1:172.21.15.82:6789/0 conn(0x7f464406ac20 legacy=0x7f4644153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.074+0000 7f463be00640 1 -- 172.21.15.67:0/2827183636 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46440ab750 con 0x7f46440a94a0 2024-02-26T10:39:26.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.075+0000 7f463be00640 1 --2- 172.21.15.67:0/2827183636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46440a94a0 0x7f464406b000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f462c0097a0 tx=0x7f462c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb03608169874a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.076+0000 7f4641400640 1 -- 172.21.15.67:0/2827183636 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c035020 con 0x7f46440a94a0 2024-02-26T10:39:26.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.076+0000 7f4641400640 1 -- 172.21.15.67:0/2827183636 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c031830 con 0x7f46440a94a0 2024-02-26T10:39:26.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.076+0000 7f4641400640 1 -- 172.21.15.67:0/2827183636 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c031c00 con 0x7f46440a94a0 2024-02-26T10:39:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 -- 172.21.15.67:0/2827183636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46440a94a0 msgr2=0x7f464406b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 --2- 172.21.15.67:0/2827183636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46440a94a0 0x7f464406b000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f462c0097a0 tx=0x7f462c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 -- 172.21.15.67:0/2827183636 shutdown_connections 2024-02-26T10:39:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 --2- 172.21.15.67:0/2827183636 >> v2:172.21.15.82:3300/0 conn(0x7f464406db00 0x7f46440abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 --2- 172.21.15.67:0/2827183636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46440a94a0 0x7f464406b000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 -- 172.21.15.67:0/2827183636 >> 172.21.15.67:0/2827183636 conn(0x7f46440c63f0 msgr2=0x7f46440c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 -- 172.21.15.67:0/2827183636 shutdown_connections 2024-02-26T10:39:26.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.077+0000 7f4643200640 1 -- 172.21.15.67:0/2827183636 wait complete. 2024-02-26T10:39:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.079+0000 7f4643200640 1 Processor -- start 2024-02-26T10:39:26.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.079+0000 7f4643200640 1 -- start start 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4643200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464406ac20 0x7f4644147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4643200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f464406db00 0x7f46440c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4643200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 0x7f4644149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4643200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644150410 con 0x7f46440a94a0 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4643200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46440ac490 con 0x7f464406ac20 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4643200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4644150590 con 0x7f464406db00 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4642800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 0x7f4644149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4642800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 0x7f4644149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60678/0 (socket says 172.21.15.67:60678) 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.080+0000 7f4642800640 1 -- 172.21.15.67:0/3496645231 learned_addr learned my addr 172.21.15.67:0/3496645231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4641e00640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464406ac20 0x7f4644147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f463be00640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f464406db00 0x7f46440c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4642800640 1 -- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f464406db00 msgr2=0x7f46440c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4642800640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f464406db00 0x7f46440c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4642800640 1 -- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464406ac20 msgr2=0x7f4644147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4642800640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464406ac20 0x7f4644147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4642800640 1 -- 172.21.15.67:0/3496645231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644147b90 con 0x7f46440a94a0 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f463be00640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f464406db00 0x7f46440c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4641e00640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464406ac20 0x7f4644147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.081+0000 7f4642800640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 0x7f4644149e90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f463c00efc0 tx=0x7f463c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.082+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c00cd20 con 0x7f46440a94a0 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.082+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c003990 con 0x7f46440a94a0 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.082+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46440c0360 con 0x7f46440a94a0 2024-02-26T10:39:26.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.082+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c003d80 con 0x7f46440a94a0 2024-02-26T10:39:26.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.082+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644072810 con 0x7f46440a94a0 2024-02-26T10:39:26.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.083+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4644074a30 con 0x7f46440a94a0 2024-02-26T10:39:26.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.084+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f463c012050 con 0x7f46440a94a0 2024-02-26T10:39:26.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.085+0000 7f463a000640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462006b500 0x7f462006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.085+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f463c08a740 con 0x7f46440a94a0 2024-02-26T10:39:26.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.086+0000 7f4641e00640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462006b500 0x7f462006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.087+0000 7f4641e00640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462006b500 0x7f462006d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4634007fd0 tx=0x7f4634007d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.090+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f463c010040 con 0x7f46440a94a0 2024-02-26T10:39:26.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.327+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46440c2ae0 con 0x7f46440a94a0 2024-02-26T10:39:26.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.328+0000 7f463a000640 1 -- 172.21.15.67:0/3496645231 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f463c0595d0 con 0x7f46440a94a0 2024-02-26T10:39:26.330 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:26.330 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:39:26.330 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462006b500 msgr2=0x7f462006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462006b500 0x7f462006d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4634007fd0 tx=0x7f4634007d80 comp rx=0 tx=0).stop 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 msgr2=0x7f4644149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 0x7f4644149e90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f463c00efc0 tx=0x7f463c00c490 comp rx=0 tx=0).stop 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 shutdown_connections 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46440a94a0 0x7f4644149e90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f464406db00 0x7f46440c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462006b500 0x7f462006d9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 --2- 172.21.15.67:0/3496645231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464406ac20 0x7f4644147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 >> 172.21.15.67:0/3496645231 conn(0x7f46440c63f0 msgr2=0x7f464414e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 shutdown_connections 2024-02-26T10:39:26.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.331+0000 7f4643200640 1 -- 172.21.15.67:0/3496645231 wait complete. 2024-02-26T10:39:26.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:26.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.569+0000 7f0837e00640 1 Processor -- start 2024-02-26T10:39:26.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.569+0000 7f0837e00640 1 -- start start 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.570+0000 7f0837e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f083806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0837e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0838153630 0x7f08381524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0837e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0838155f10 con 0x7f08380c2310 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0837e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0838153aa0 con 0x7f0838153630 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0837e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380c28d0 con 0x7f083806ac20 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0836000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f08380c2310 0x7f0838152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36204/0 (socket says 172.21.15.67:36204) 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0836000640 1 -- 172.21.15.67:0/1971581374 learned_addr learned my addr 172.21.15.67:0/1971581374 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0837400640 1 --2- 172.21.15.67:0/1971581374 >> v2:172.21.15.67:3300/0 conn(0x7f0838153630 0x7f08381524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.571+0000 7f0836a00640 1 --2- 172.21.15.67:0/1971581374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f083806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.572+0000 7f0835600640 1 -- 172.21.15.67:0/1971581374 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306816396 0 0) 0x7f0838155f10 con 0x7f08380c2310 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.572+0000 7f0835600640 1 -- 172.21.15.67:0/1971581374 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f081c005730 con 0x7f08380c2310 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.572+0000 7f0837400640 1 -- 172.21.15.67:0/1971581374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 msgr2=0x7f083806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.572+0000 7f0837400640 1 --2- 172.21.15.67:0/1971581374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f083806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.572+0000 7f0837400640 1 -- 172.21.15.67:0/1971581374 >> v1:172.21.15.67:6789/0 conn(0x7f08380c2310 legacy=0x7f0838152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.572+0000 7f0837400640 1 -- 172.21.15.67:0/1971581374 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838155150 con 0x7f0838153630 2024-02-26T10:39:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.573+0000 7f0837400640 1 --2- 172.21.15.67:0/1971581374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381524b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0830009d90 tx=0x7f083002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97699df0505c1fe0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0835600640 1 -- 172.21.15.67:0/1971581374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830039070 con 0x7f0838153630 2024-02-26T10:39:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0835600640 1 -- 172.21.15.67:0/1971581374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083002cb30 con 0x7f0838153630 2024-02-26T10:39:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0835600640 1 -- 172.21.15.67:0/1971581374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830034440 con 0x7f0838153630 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 -- 172.21.15.67:0/1971581374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 msgr2=0x7f08381524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 --2- 172.21.15.67:0/1971581374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381524b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0830009d90 tx=0x7f083002bd80 comp rx=0 tx=0).stop 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 -- 172.21.15.67:0/1971581374 shutdown_connections 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 --2- 172.21.15.67:0/1971581374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381524b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 --2- 172.21.15.67:0/1971581374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f083806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 -- 172.21.15.67:0/1971581374 >> 172.21.15.67:0/1971581374 conn(0x7f08380c63f0 msgr2=0x7f08380c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 -- 172.21.15.67:0/1971581374 shutdown_connections 2024-02-26T10:39:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.574+0000 7f0837e00640 1 -- 172.21.15.67:0/1971581374 wait complete. 2024-02-26T10:39:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.576+0000 7f0837e00640 1 Processor -- start 2024-02-26T10:39:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.576+0000 7f0837e00640 1 -- start start 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f08381edd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08380c2310 0x7f08381df1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381df710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08381559f0 con 0x7f0838153630 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838155710 con 0x7f08380c2310 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0838148850 con 0x7f083806ac20 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381df710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60690/0 (socket says 172.21.15.67:60690) 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.577+0000 7f0837400640 1 -- 172.21.15.67:0/1919279855 learned_addr learned my addr 172.21.15.67:0/1919279855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0836a00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f08381edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0836000640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08380c2310 0x7f08381df1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837400640 1 -- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 msgr2=0x7f08381edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837400640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f08381edd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837400640 1 -- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08380c2310 msgr2=0x7f08381df1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837400640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08380c2310 0x7f08381df1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837400640 1 -- 172.21.15.67:0/1919279855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380ab180 con 0x7f0838153630 2024-02-26T10:39:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837400640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381df710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0830009eb0 tx=0x7f0830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830039050 con 0x7f0838153630 2024-02-26T10:39:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0830045070 con 0x7f0838153630 2024-02-26T10:39:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830009450 con 0x7f0838153630 2024-02-26T10:39:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.578+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08381ee690 con 0x7f0838153630 2024-02-26T10:39:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.579+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08380ab940 con 0x7f0838153630 2024-02-26T10:39:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.580+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0838074a30 con 0x7f0838153630 2024-02-26T10:39:26.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.580+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f083003e850 con 0x7f0838153630 2024-02-26T10:39:26.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.581+0000 7f082f400640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f081806b4d0 0x7f081806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:26.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.581+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f08300ad080 con 0x7f0838153630 2024-02-26T10:39:26.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.582+0000 7f0836a00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f081806b4d0 0x7f081806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.583+0000 7f0836a00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f081806b4d0 0x7f081806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0828009a30 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.586+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0830032e20 con 0x7f0838153630 2024-02-26T10:39:26.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.812+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08380c20f0 con 0x7f0838153630 2024-02-26T10:39:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.813+0000 7f082f400640 1 -- 172.21.15.67:0/1919279855 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f083007bf90 con 0x7f0838153630 2024-02-26T10:39:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:26.814 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2134165620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2134165620},{"type":"v1","addr":"172.21.15.82:6835","nonce":2134165620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/2807186934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2807186934},{"type":"v1","addr":"172.21.15.67:6833","nonce":2807186934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2238876145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2238876145},{"type":"v1","addr":"172.21.15.67:6835","nonce":2238876145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:38:34.632128+0000","modified":"2024-02-26T10:38:37.689492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/1820988808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1820988808},{"type":"v1","addr":"172.21.15.82:6837","nonce":1820988808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-26T10:39:26.814 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-02-26T10:39:26.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f081806b4d0 msgr2=0x7f081806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f081806b4d0 0x7f081806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0828009a30 tx=0x7f0828008040 comp rx=0 tx=0).stop 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 msgr2=0x7f08381df710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381df710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0830009eb0 tx=0x7f0830004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 shutdown_connections 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0838153630 0x7f08381df710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f081806b4d0 0x7f081806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08380c2310 0x7f08381df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 --2- 172.21.15.67:0/1919279855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f083806ac20 0x7f08381edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 >> 172.21.15.67:0/1919279855 conn(0x7f08380c63f0 msgr2=0x7f08380c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 shutdown_connections 2024-02-26T10:39:26.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:26.815+0000 7f0837e00640 1 -- 172.21.15.67:0/1919279855 wait complete. 2024-02-26T10:39:26.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:27.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.049+0000 7f4c4d000640 1 Processor -- start 2024-02-26T10:39:27.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.049+0000 7f4c4d000640 1 -- start start 2024-02-26T10:39:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c4d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4c480c2310 0x7f4c48153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c4d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c48155f10 con 0x7f4c4806b000 2024-02-26T10:39:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c4d000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480c28d0 con 0x7f4c4806ac20 2024-02-26T10:39:27.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c4d000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4806b430 con 0x7f4c480c2310 2024-02-26T10:39:27.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c47e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4c4806b000 0x7f4c4815c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36206/0 (socket says 172.21.15.67:36206) 2024-02-26T10:39:27.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c47e00640 1 -- 172.21.15.67:0/780409147 learned_addr learned my addr 172.21.15.67:0/780409147 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:27.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.051+0000 7f4c3fe00640 1 --2- 172.21.15.67:0/780409147 >> v2:172.21.15.82:3300/0 conn(0x7f4c480c2310 0x7f4c48153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.052+0000 7f4c46a00640 1 -- 172.21.15.67:0/780409147 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115352069 0 0) 0x7f4c48155f10 con 0x7f4c4806b000 2024-02-26T10:39:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.052+0000 7f4c46a00640 1 -- 172.21.15.67:0/780409147 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c2c005730 con 0x7f4c4806b000 2024-02-26T10:39:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.052+0000 7f4c3fe00640 1 -- 172.21.15.67:0/780409147 >> v1:172.21.15.82:6789/0 conn(0x7f4c4806ac20 legacy=0x7f4c48152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.052+0000 7f4c3fe00640 1 -- 172.21.15.67:0/780409147 >> v1:172.21.15.67:6789/0 conn(0x7f4c4806b000 legacy=0x7f4c4815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.052+0000 7f4c3fe00640 1 -- 172.21.15.67:0/780409147 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48155150 con 0x7f4c480c2310 2024-02-26T10:39:27.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.053+0000 7f4c3fe00640 1 --2- 172.21.15.67:0/780409147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c48153630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c300097a0 tx=0x7f4c3002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1eafa1a0beb5363 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.054+0000 7f4c46a00640 1 -- 172.21.15.67:0/780409147 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300315a0 con 0x7f4c480c2310 2024-02-26T10:39:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.054+0000 7f4c46a00640 1 -- 172.21.15.67:0/780409147 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c30037070 con 0x7f4c480c2310 2024-02-26T10:39:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.054+0000 7f4c46a00640 1 -- 172.21.15.67:0/780409147 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30031940 con 0x7f4c480c2310 2024-02-26T10:39:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 -- 172.21.15.67:0/780409147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 msgr2=0x7f4c48153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 --2- 172.21.15.67:0/780409147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c48153630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c300097a0 tx=0x7f4c3002b840 comp rx=0 tx=0).stop 2024-02-26T10:39:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 -- 172.21.15.67:0/780409147 shutdown_connections 2024-02-26T10:39:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 --2- 172.21.15.67:0/780409147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c48153630 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 -- 172.21.15.67:0/780409147 >> 172.21.15.67:0/780409147 conn(0x7f4c480c63f0 msgr2=0x7f4c480c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 -- 172.21.15.67:0/780409147 shutdown_connections 2024-02-26T10:39:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.055+0000 7f4c4d000640 1 -- 172.21.15.67:0/780409147 wait complete. 2024-02-26T10:39:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.057+0000 7f4c4d000640 1 Processor -- start 2024-02-26T10:39:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.057+0000 7f4c4d000640 1 -- start start 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c4d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 0x7f4c481dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c4d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c4806b000 0x7f4c481e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c4d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c480714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c4d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806e1b0 con 0x7f4c4806ac20 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c4d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806e030 con 0x7f4c480c2310 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c4d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4806e330 con 0x7f4c4806b000 2024-02-26T10:39:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 0x7f4c481dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 0x7f4c481dd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60700/0 (socket says 172.21.15.67:60700) 2024-02-26T10:39:27.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 -- 172.21.15.67:0/1446649769 learned_addr learned my addr 172.21.15.67:0/1446649769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c3fe00640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c4806b000 0x7f4c481e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 -- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c4806b000 msgr2=0x7f4c481e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47e00640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c480714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c4806b000 0x7f4c481e9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 -- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 msgr2=0x7f4c480714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.058+0000 7f4c47400640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c480714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c47400640 1 -- 172.21.15.67:0/1446649769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c481ea340 con 0x7f4c4806ac20 2024-02-26T10:39:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c3fe00640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c4806b000 0x7f4c481e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c47400640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 0x7f4c481dd6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c480 tx=0x7f4c3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c380052d0 con 0x7f4c4806ac20 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38005470 con 0x7f4c4806ac20 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38012870 con 0x7f4c4806ac20 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c481ea620 con 0x7f4c4806ac20 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.059+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c481ea040 con 0x7f4c4806ac20 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.060+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c48074a30 con 0x7f4c4806ac20 2024-02-26T10:39:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.061+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c38012a10 con 0x7f4c4806ac20 2024-02-26T10:39:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.062+0000 7f4c3f400640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c2406b4d0 0x7f4c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.062+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4c38089e80 con 0x7f4c4806ac20 2024-02-26T10:39:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.063+0000 7f4c3fe00640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c2406b4d0 0x7f4c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.064+0000 7f4c3fe00640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c2406b4d0 0x7f4c2406d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c30036000 tx=0x7f4c30008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:27.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.067+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c38058d10 con 0x7f4c4806ac20 2024-02-26T10:39:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.258+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c480c20f0 con 0x7f4c4806ac20 2024-02-26T10:39:27.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.260+0000 7f4c3f400640 1 -- 172.21.15.67:0/1446649769 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16782 (secure 0 0 0) 0x7f4c38058610 con 0x7f4c4806ac20 2024-02-26T10:39:27.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:27.262 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":23,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:21.453133+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:38:32.893594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-26T10:38:33.645319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:27.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c2406b4d0 msgr2=0x7f4c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c2406b4d0 0x7f4c2406d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c30036000 tx=0x7f4c30008030 comp rx=0 tx=0).stop 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 msgr2=0x7f4c481dd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 0x7f4c481dd6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800c480 tx=0x7f4c3800c940 comp rx=0 tx=0).stop 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 shutdown_connections 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c2406b4d0 0x7f4c2406d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c480c2310 0x7f4c480714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c4806b000 0x7f4c481e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 --2- 172.21.15.67:0/1446649769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c4806ac20 0x7f4c481dd6b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 >> 172.21.15.67:0/1446649769 conn(0x7f4c480c63f0 msgr2=0x7f4c480c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 shutdown_connections 2024-02-26T10:39:27.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.263+0000 7f4c4d000640 1 -- 172.21.15.67:0/1446649769 wait complete. 2024-02-26T10:39:27.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T10:39:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.497+0000 7faf19800640 1 Processor -- start 2024-02-26T10:39:27.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.497+0000 7faf19800640 1 -- start start 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf19800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf140a94a0 0x7faf1406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf19800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf1406db00 0x7faf140abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf19800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1406df90 con 0x7faf1406ac20 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf19800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf140a9a80 con 0x7faf1406db00 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf19800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140a9d60 con 0x7faf140a94a0 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf13e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faf1406ac20 0x7faf14153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36212/0 (socket says 172.21.15.67:36212) 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf18e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf1406db00 0x7faf140abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.499+0000 7faf13e00640 1 -- 172.21.15.67:0/1897847688 learned_addr learned my addr 172.21.15.67:0/1897847688 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.500+0000 7faf13400640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf140a94a0 0x7faf1406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.500+0000 7faf18e00640 1 -- 172.21.15.67:0/1897847688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf140a94a0 msgr2=0x7faf1406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.500+0000 7faf18e00640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf140a94a0 0x7faf1406b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.500+0000 7faf18e00640 1 -- 172.21.15.67:0/1897847688 >> v1:172.21.15.67:6789/0 conn(0x7faf1406ac20 legacy=0x7faf14153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.500+0000 7faf18e00640 1 -- 172.21.15.67:0/1897847688 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140ab750 con 0x7faf1406db00 2024-02-26T10:39:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.500+0000 7faf13400640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf140a94a0 0x7faf1406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.501+0000 7faf18e00640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf1406db00 0x7faf140abf50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009d90 tx=0x7faf0c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f48b1943deae8f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.501+0000 7faf12a00640 1 -- 172.21.15.67:0/1897847688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c039070 con 0x7faf1406db00 2024-02-26T10:39:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.501+0000 7faf12a00640 1 -- 172.21.15.67:0/1897847688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c02cc50 con 0x7faf1406db00 2024-02-26T10:39:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.501+0000 7faf12a00640 1 -- 172.21.15.67:0/1897847688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c034440 con 0x7faf1406db00 2024-02-26T10:39:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 -- 172.21.15.67:0/1897847688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf1406db00 msgr2=0x7faf140abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf1406db00 0x7faf140abf50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009d90 tx=0x7faf0c02bea0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 -- 172.21.15.67:0/1897847688 shutdown_connections 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf1406db00 0x7faf140abf50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 --2- 172.21.15.67:0/1897847688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf140a94a0 0x7faf1406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 -- 172.21.15.67:0/1897847688 >> 172.21.15.67:0/1897847688 conn(0x7faf140c63f0 msgr2=0x7faf140c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 -- 172.21.15.67:0/1897847688 shutdown_connections 2024-02-26T10:39:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.502+0000 7faf19800640 1 -- 172.21.15.67:0/1897847688 wait complete. 2024-02-26T10:39:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.504+0000 7faf19800640 1 Processor -- start 2024-02-26T10:39:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.504+0000 7faf19800640 1 -- start start 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf19800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1406ac20 0x7faf1414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf19800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf1406db00 0x7faf1414eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf19800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 0x7faf141eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf19800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14150410 con 0x7faf140a94a0 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf19800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140ac490 con 0x7faf1406ac20 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf19800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf14150590 con 0x7faf1406db00 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf18e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 0x7faf141eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf18e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 0x7faf141eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60720/0 (socket says 172.21.15.67:60720) 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.505+0000 7faf18e00640 1 -- 172.21.15.67:0/932564237 learned_addr learned my addr 172.21.15.67:0/932564237 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf13e00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1406ac20 0x7faf1414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf13400640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf1406db00 0x7faf1414eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf18e00640 1 -- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf1406db00 msgr2=0x7faf1414eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf18e00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf1406db00 0x7faf1414eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf18e00640 1 -- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1406ac20 msgr2=0x7faf1414e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf18e00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1406ac20 0x7faf1414e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf18e00640 1 -- 172.21.15.67:0/932564237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1414b2a0 con 0x7faf140a94a0 2024-02-26T10:39:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.506+0000 7faf18e00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 0x7faf141eaa90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009d90 tx=0x7faf0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.509+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c039050 con 0x7faf140a94a0 2024-02-26T10:39:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.509+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c045070 con 0x7faf140a94a0 2024-02-26T10:39:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.509+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c03e610 con 0x7faf140a94a0 2024-02-26T10:39:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.509+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf141ee880 con 0x7faf140a94a0 2024-02-26T10:39:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.509+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1414f1a0 con 0x7faf140a94a0 2024-02-26T10:39:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.510+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf141ee670 con 0x7faf140a94a0 2024-02-26T10:39:27.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.511+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf0c03e7b0 con 0x7faf140a94a0 2024-02-26T10:39:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.512+0000 7faf0be00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faef406b500 0x7faef406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.512+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faf0c0ace50 con 0x7faf140a94a0 2024-02-26T10:39:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.513+0000 7faf13e00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faef406b500 0x7faef406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:27.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.514+0000 7faf13e00640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faef406b500 0x7faef406d9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf140c62e0 tx=0x7faf04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.516+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf0c032e20 con 0x7faf140a94a0 2024-02-26T10:39:27.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.753+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf14074a30 con 0x7faf140a94a0 2024-02-26T10:39:27.756 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:39:27.755+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:39:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.757+0000 7faf0be00640 1 -- 172.21.15.67:0/932564237 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf0c07bce0 con 0x7faf140a94a0 2024-02-26T10:39:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faef406b500 msgr2=0x7faef406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faef406b500 0x7faef406d9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf140c62e0 tx=0x7faf04009040 comp rx=0 tx=0).stop 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 msgr2=0x7faf141eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 0x7faf141eaa90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009d90 tx=0x7faf0c004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 shutdown_connections 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf140a94a0 0x7faf141eaa90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf1406db00 0x7faf1414eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faef406b500 0x7faef406d9c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 --2- 172.21.15.67:0/932564237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1406ac20 0x7faf1414e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.761+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 >> 172.21.15.67:0/932564237 conn(0x7faf140c63f0 msgr2=0x7faf140c23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:27.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.762+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 shutdown_connections 2024-02-26T10:39:27.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:27.762+0000 7faf19800640 1 -- 172.21.15.67:0/932564237 wait complete. 2024-02-26T10:39:27.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T10:39:27.798 INFO:tasks.ceph.mds.b.smithi082.stderr: -6209> 2024-02-26T10:38:39.610+0000 7f7f62e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:39:27.879 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-02-26T10:39:28.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.005+0000 7fbb3a800640 1 Processor -- start 2024-02-26T10:39:28.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.005+0000 7fbb3a800640 1 -- start start 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.007+0000 7fbb3a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb3406ed80 0x7fbb34070f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.007+0000 7fbb3a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3415a2b0 con 0x7fbb3415c920 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.007+0000 7fbb3a800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb34072bb0 con 0x7fbb340c3290 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.007+0000 7fbb3a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb34072e90 con 0x7fbb3406ed80 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.008+0000 7fbb32a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb3406ed80 0x7fbb34070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.008+0000 7fbb33e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbb3415c920 0x7fbb3415cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36218/0 (socket says 172.21.15.67:36218) 2024-02-26T10:39:28.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.008+0000 7fbb33e00640 1 -- 172.21.15.67:0/1467537299 learned_addr learned my addr 172.21.15.67:0/1467537299 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.008+0000 7fbb32a00640 1 -- 172.21.15.67:0/1467537299 >> v1:172.21.15.82:6789/0 conn(0x7fbb340c3290 legacy=0x7fbb34159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.008+0000 7fbb32a00640 1 -- 172.21.15.67:0/1467537299 >> v1:172.21.15.67:6789/0 conn(0x7fbb3415c920 legacy=0x7fbb3415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.008+0000 7fbb32a00640 1 -- 172.21.15.67:0/1467537299 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34073170 con 0x7fbb3406ed80 2024-02-26T10:39:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.009+0000 7fbb32a00640 1 --2- 172.21.15.67:0/1467537299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb34070f10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0097a0 tx=0x7fbb1c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb84e78d36ec2b89 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.010+0000 7fbb32000640 1 -- 172.21.15.67:0/1467537299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c034070 con 0x7fbb3406ed80 2024-02-26T10:39:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.010+0000 7fbb32000640 1 -- 172.21.15.67:0/1467537299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c030610 con 0x7fbb3406ed80 2024-02-26T10:39:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.010+0000 7fbb32000640 1 -- 172.21.15.67:0/1467537299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0309e0 con 0x7fbb3406ed80 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.010+0000 7fbb3a800640 1 -- 172.21.15.67:0/1467537299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 msgr2=0x7fbb34070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.010+0000 7fbb3a800640 1 --2- 172.21.15.67:0/1467537299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb34070f10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0097a0 tx=0x7fbb1c02b960 comp rx=0 tx=0).stop 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.011+0000 7fbb3a800640 1 -- 172.21.15.67:0/1467537299 shutdown_connections 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.011+0000 7fbb3a800640 1 --2- 172.21.15.67:0/1467537299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb34070f10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.011+0000 7fbb3a800640 1 -- 172.21.15.67:0/1467537299 >> 172.21.15.67:0/1467537299 conn(0x7fbb340a84a0 msgr2=0x7fbb340a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.011+0000 7fbb3a800640 1 -- 172.21.15.67:0/1467537299 shutdown_connections 2024-02-26T10:39:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.011+0000 7fbb3a800640 1 -- 172.21.15.67:0/1467537299 wait complete. 2024-02-26T10:39:28.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.012+0000 7fbb3a800640 1 Processor -- start 2024-02-26T10:39:28.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 -- start start 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb340aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb340c3290 0x7fbb3414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb3415c920 0x7fbb3414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c3670 con 0x7fbb3406ed80 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34159330 con 0x7fbb3415c920 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.013+0000 7fbb3a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c37f0 con 0x7fbb340c3290 2024-02-26T10:39:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb340aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb340aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60732/0 (socket says 172.21.15.67:60732) 2024-02-26T10:39:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 -- 172.21.15.67:0/2909596743 learned_addr learned my addr 172.21.15.67:0/2909596743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33e00640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb3415c920 0x7fbb3414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb32a00640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb340c3290 0x7fbb3414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 -- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb340c3290 msgr2=0x7fbb3414d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb340c3290 0x7fbb3414d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 -- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb3415c920 msgr2=0x7fbb3414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb3415c920 0x7fbb3414dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.014+0000 7fbb33400640 1 -- 172.21.15.67:0/2909596743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb341f2bd0 con 0x7fbb3406ed80 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.015+0000 7fbb33400640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb340aaa30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c400 tx=0x7fbb2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.015+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240052d0 con 0x7fbb3406ed80 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.015+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24005490 con 0x7fbb3406ed80 2024-02-26T10:39:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.015+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24007260 con 0x7fbb3406ed80 2024-02-26T10:39:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.015+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb34153b40 con 0x7fbb3406ed80 2024-02-26T10:39:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.015+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34154100 con 0x7fbb3406ed80 2024-02-26T10:39:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.016+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb34074a30 con 0x7fbb3406ed80 2024-02-26T10:39:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.017+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb24007400 con 0x7fbb3406ed80 2024-02-26T10:39:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.018+0000 7fbb2b400640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb1006b4d0 0x7fbb1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.018+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fbb24089830 con 0x7fbb3406ed80 2024-02-26T10:39:28.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.019+0000 7fbb32a00640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb1006b4d0 0x7fbb1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:28.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.020+0000 7fbb32a00640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb1006b4d0 0x7fbb1006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c036000 tx=0x7fbb1c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:28.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.023+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb240585d0 con 0x7fbb3406ed80 2024-02-26T10:39:28.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.265+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb3406e360 con 0x7fbb3406ed80 2024-02-26T10:39:28.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.767+0000 7fbb2b400640 1 -- 172.21.15.67:0/2909596743 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb24057ed0 con 0x7fbb3406ed80 2024-02-26T10:39:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb1006b4d0 msgr2=0x7fbb1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb1006b4d0 0x7fbb1006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c036000 tx=0x7fbb1c03b040 comp rx=0 tx=0).stop 2024-02-26T10:39:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 msgr2=0x7fbb340aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb340aaa30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c400 tx=0x7fbb2400c8c0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 shutdown_connections 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbb1006b4d0 0x7fbb1006d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb3415c920 0x7fbb3414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb340c3290 0x7fbb3414d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.772+0000 7fbb3a800640 1 --2- 172.21.15.67:0/2909596743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb3406ed80 0x7fbb340aaa30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.773+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 >> 172.21.15.67:0/2909596743 conn(0x7fbb340a84a0 msgr2=0x7fbb3415c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.773+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 shutdown_connections 2024-02-26T10:39:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:28.773+0000 7fbb3a800640 1 -- 172.21.15.67:0/2909596743 wait complete. 2024-02-26T10:39:28.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T10:39:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.030+0000 7f58eb200640 1 Processor -- start 2024-02-26T10:39:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.030+0000 7f58eb200640 1 -- start start 2024-02-26T10:39:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58eb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58ec06ac20 0x7f58ec06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58eb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06b000 0x7f58ec0abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58eb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec0a99b0 con 0x7f58ec0a94a0 2024-02-26T10:39:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58eb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec06b490 con 0x7f58ec06ac20 2024-02-26T10:39:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58eb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0aba30 con 0x7f58ec06b000 2024-02-26T10:39:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58e9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58ec06ac20 0x7f58ec06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58e9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f58ec0a94a0 0x7f58ec153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36226/0 (socket says 172.21.15.67:36226) 2024-02-26T10:39:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.032+0000 7f58e9400640 1 -- 172.21.15.67:0/3833196904 learned_addr learned my addr 172.21.15.67:0/3833196904 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58ea800640 1 --2- 172.21.15.67:0/3833196904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06b000 0x7f58ec0abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58e8a00640 1 -- 172.21.15.67:0/3833196904 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774465156 0 0) 0x7f58ec0a99b0 con 0x7f58ec0a94a0 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58e8a00640 1 -- 172.21.15.67:0/3833196904 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58d0003520 con 0x7f58ec0a94a0 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58e9e00640 1 -- 172.21.15.67:0/3833196904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06b000 msgr2=0x7f58ec0abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58e9e00640 1 --2- 172.21.15.67:0/3833196904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06b000 0x7f58ec0abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58e9e00640 1 -- 172.21.15.67:0/3833196904 >> v1:172.21.15.67:6789/0 conn(0x7f58ec0a94a0 legacy=0x7f58ec153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.033+0000 7f58e9e00640 1 -- 172.21.15.67:0/3833196904 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0a9c90 con 0x7f58ec06ac20 2024-02-26T10:39:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.034+0000 7f58e9e00640 1 --2- 172.21.15.67:0/3833196904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec06ac20 0x7f58ec06db00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0097a0 tx=0x7f58dc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d059ccbc2dc1664 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:29.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58e8a00640 1 -- 172.21.15.67:0/3833196904 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc031780 con 0x7f58ec06ac20 2024-02-26T10:39:29.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58e8a00640 1 -- 172.21.15.67:0/3833196904 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc037070 con 0x7f58ec06ac20 2024-02-26T10:39:29.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58e8a00640 1 -- 172.21.15.67:0/3833196904 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc031b40 con 0x7f58ec06ac20 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58eb200640 1 -- 172.21.15.67:0/3833196904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec06ac20 msgr2=0x7f58ec06db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58eb200640 1 --2- 172.21.15.67:0/3833196904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec06ac20 0x7f58ec06db00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0097a0 tx=0x7f58dc02b820 comp rx=0 tx=0).stop 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58eb200640 1 -- 172.21.15.67:0/3833196904 shutdown_connections 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58eb200640 1 --2- 172.21.15.67:0/3833196904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06b000 0x7f58ec0abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58eb200640 1 --2- 172.21.15.67:0/3833196904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec06ac20 0x7f58ec06db00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.035+0000 7f58eb200640 1 -- 172.21.15.67:0/3833196904 >> 172.21.15.67:0/3833196904 conn(0x7f58ec0c63f0 msgr2=0x7f58ec0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.036+0000 7f58eb200640 1 -- 172.21.15.67:0/3833196904 shutdown_connections 2024-02-26T10:39:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.036+0000 7f58eb200640 1 -- 172.21.15.67:0/3833196904 wait complete. 2024-02-26T10:39:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.037+0000 7f58eb200640 1 Processor -- start 2024-02-26T10:39:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.037+0000 7f58eb200640 1 -- start start 2024-02-26T10:39:29.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58eb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06ac20 0x7f58ec14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58eb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ec06b000 0x7f58ec1eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58eb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 0x7f58ec1eb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58eb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0a9f70 con 0x7f58ec0a94a0 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58eb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1504f0 con 0x7f58ec06b000 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58eb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0aa0f0 con 0x7f58ec06ac20 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.038+0000 7f58ea800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 0x7f58ec1eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 0x7f58ec1eb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60750/0 (socket says 172.21.15.67:60750) 2024-02-26T10:39:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 -- 172.21.15.67:0/2730717381 learned_addr learned my addr 172.21.15.67:0/2730717381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58e9e00640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06ac20 0x7f58ec14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 -- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06ac20 msgr2=0x7f58ec14f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06ac20 0x7f58ec14f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 -- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ec06b000 msgr2=0x7f58ec1eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58e9400640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ec06b000 0x7f58ec1eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ec06b000 0x7f58ec1eb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 -- 172.21.15.67:0/2730717381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec1ebe90 con 0x7f58ec0a94a0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.039+0000 7f58ea800640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 0x7f58ec1eb950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58e4007e40 tx=0x7f58e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.040+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4014620 con 0x7f58ec0a94a0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.040+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e4019070 con 0x7f58ec0a94a0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.040+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4014a20 con 0x7f58ec0a94a0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.040+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec1eaf10 con 0x7f58ec0a94a0 2024-02-26T10:39:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.040+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec0c5cd0 con 0x7f58ec0a94a0 2024-02-26T10:39:29.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.041+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec075020 con 0x7f58ec0a94a0 2024-02-26T10:39:29.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.042+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58e4014bc0 con 0x7f58ec0a94a0 2024-02-26T10:39:29.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.043+0000 7f58e2000640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58cc06b4d0 0x7f58cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:29.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.043+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f58e408abf0 con 0x7f58ec0a94a0 2024-02-26T10:39:29.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.044+0000 7f58e9e00640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58cc06b4d0 0x7f58cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:29.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.045+0000 7f58e9e00640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58cc06b4d0 0x7f58cc06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58dc036000 tx=0x7f58dc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:29.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.047+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58ec075020 con 0x7f58ec0a94a0 2024-02-26T10:39:29.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.250+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58ec074a30 con 0x7f58ec0a94a0 2024-02-26T10:39:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.772+0000 7f58e2000640 1 -- 172.21.15.67:0/2730717381 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f58ec1ebe90 con 0x7f58ec0a94a0 2024-02-26T10:39:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58cc06b4d0 msgr2=0x7f58cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58cc06b4d0 0x7f58cc06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58dc036000 tx=0x7f58dc008030 comp rx=0 tx=0).stop 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 msgr2=0x7f58ec1eb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 0x7f58ec1eb950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f58e4007e40 tx=0x7f58e400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 shutdown_connections 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58ec0a94a0 0x7f58ec1eb950 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58cc06b4d0 0x7f58cc06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ec06b000 0x7f58ec1eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 --2- 172.21.15.67:0/2730717381 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ec06ac20 0x7f58ec14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.776+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 >> 172.21.15.67:0/2730717381 conn(0x7f58ec0c63f0 msgr2=0x7f58ec14b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.777+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 shutdown_connections 2024-02-26T10:39:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:29.777+0000 7f58eb200640 1 -- 172.21.15.67:0/2730717381 wait complete. 2024-02-26T10:39:29.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T10:39:29.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:39:30.001 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:39:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.018+0000 7fd296200640 1 Processor -- start 2024-02-26T10:39:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.018+0000 7fd296200640 1 -- start start 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.019+0000 7fd296200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd29006bc30 0x7fd29006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.019+0000 7fd296200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c14a0 0x7fd29014f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.019+0000 7fd296200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2901538d0 con 0x7fd2900c49a0 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.019+0000 7fd296200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900c30e0 con 0x7fd29006bc30 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.020+0000 7fd296200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2901535f0 con 0x7fd2900c14a0 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.020+0000 7fd28fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c14a0 0x7fd29014f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.020+0000 7fd294e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd29006bc30 0x7fd29006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.020+0000 7fd295800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd2900c49a0 0x7fd29014fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:51652/0 (socket says 172.21.15.67:51652) 2024-02-26T10:39:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.020+0000 7fd295800640 1 -- 172.21.15.67:0/2698506416 learned_addr learned my addr 172.21.15.67:0/2698506416 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd28f400640 1 -- 172.21.15.67:0/2698506416 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211634437 0 0) 0x7fd2901538d0 con 0x7fd2900c49a0 2024-02-26T10:39:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd28f400640 1 -- 172.21.15.67:0/2698506416 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd274003520 con 0x7fd2900c49a0 2024-02-26T10:39:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd294e00640 1 -- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c14a0 msgr2=0x7fd29014f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd294e00640 1 --2- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c14a0 0x7fd29014f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd294e00640 1 -- 172.21.15.67:0/2698506416 >> v1:172.21.15.82:6789/0 conn(0x7fd2900c49a0 legacy=0x7fd29014fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:30.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd294e00640 1 -- 172.21.15.67:0/2698506416 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900c2d70 con 0x7fd29006bc30 2024-02-26T10:39:30.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.021+0000 7fd28fe00640 1 --2- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c14a0 0x7fd29014f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:39:30.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.022+0000 7fd294e00640 1 --2- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd29006c010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd280009d90 tx=0x7fd28002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e89349ac9b34a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:30.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.023+0000 7fd28f400640 1 -- 172.21.15.67:0/2698506416 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280039070 con 0x7fd29006bc30 2024-02-26T10:39:30.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.023+0000 7fd28f400640 1 -- 172.21.15.67:0/2698506416 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28002cc50 con 0x7fd29006bc30 2024-02-26T10:39:30.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.023+0000 7fd28f400640 1 -- 172.21.15.67:0/2698506416 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280034440 con 0x7fd29006bc30 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.023+0000 7fd296200640 1 -- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 msgr2=0x7fd29006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.023+0000 7fd296200640 1 --2- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd29006c010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd280009d90 tx=0x7fd28002bea0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.024+0000 7fd296200640 1 -- 172.21.15.67:0/2698506416 shutdown_connections 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.024+0000 7fd296200640 1 --2- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c14a0 0x7fd29014f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.024+0000 7fd296200640 1 --2- 172.21.15.67:0/2698506416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd29006c010 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.024+0000 7fd296200640 1 -- 172.21.15.67:0/2698506416 >> 172.21.15.67:0/2698506416 conn(0x7fd2900c35a0 msgr2=0x7fd2900c39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.024+0000 7fd296200640 1 -- 172.21.15.67:0/2698506416 shutdown_connections 2024-02-26T10:39:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.024+0000 7fd296200640 1 -- 172.21.15.67:0/2698506416 wait complete. 2024-02-26T10:39:30.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.025+0000 7fd296200640 1 Processor -- start 2024-02-26T10:39:30.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.026+0000 7fd296200640 1 -- start start 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.026+0000 7fd296200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd290152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd296200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 0x7fd2901ea840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd296200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c49a0 0x7fd2901ead80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd296200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900725a0 con 0x7fd2900c14a0 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd296200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290072420 con 0x7fd29006bc30 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd296200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290072720 con 0x7fd2900c49a0 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 0x7fd2901ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 0x7fd2901ea840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49604/0 (socket says 172.21.15.67:49604) 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 -- 172.21.15.67:0/1230431969 learned_addr learned my addr 172.21.15.67:0/1230431969 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd295800640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c49a0 0x7fd2901ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd294e00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd290152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 -- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c49a0 msgr2=0x7fd2901ead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c49a0 0x7fd2901ead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 -- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 msgr2=0x7fd290152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd290152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd28fe00640 1 -- 172.21.15.67:0/1230431969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2901edd90 con 0x7fd2900c14a0 2024-02-26T10:39:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd295800640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c49a0 0x7fd2901ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.027+0000 7fd294e00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd290152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.028+0000 7fd28fe00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 0x7fd2901ea840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd27800ebe0 tx=0x7fd27800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.028+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278017070 con 0x7fd2900c14a0 2024-02-26T10:39:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.028+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27800cc30 con 0x7fd2900c14a0 2024-02-26T10:39:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.028+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2901ee070 con 0x7fd2900c14a0 2024-02-26T10:39:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.028+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278013470 con 0x7fd2900c14a0 2024-02-26T10:39:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.028+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2901ee630 con 0x7fd2900c14a0 2024-02-26T10:39:30.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.030+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd290074a30 con 0x7fd2900c14a0 2024-02-26T10:39:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.030+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd27801a070 con 0x7fd2900c14a0 2024-02-26T10:39:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.031+0000 7fd28cc00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd26c06b4d0 0x7fd26c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.031+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fd27808a440 con 0x7fd2900c14a0 2024-02-26T10:39:30.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.032+0000 7fd294e00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd26c06b4d0 0x7fd26c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:30.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.033+0000 7fd294e00640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd26c06b4d0 0x7fd26c06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd280036000 tx=0x7fd280030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:30.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.036+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd278059390 con 0x7fd2900c14a0 2024-02-26T10:39:30.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.238+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2900bff70 con 0x7fd2900c14a0 2024-02-26T10:39:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.784+0000 7fd28cc00640 1 -- 172.21.15.67:0/1230431969 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd278058c90 con 0x7fd2900c14a0 2024-02-26T10:39:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-02-26T10:39:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd26c06b4d0 msgr2=0x7fd26c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd26c06b4d0 0x7fd26c06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd280036000 tx=0x7fd280030000 comp rx=0 tx=0).stop 2024-02-26T10:39:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 msgr2=0x7fd2901ea840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 0x7fd2901ea840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd27800ebe0 tx=0x7fd27800c520 comp rx=0 tx=0).stop 2024-02-26T10:39:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 shutdown_connections 2024-02-26T10:39:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2900c49a0 0x7fd2901ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2900c14a0 0x7fd2901ea840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd26c06b4d0 0x7fd26c06d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 --2- 172.21.15.67:0/1230431969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd29006bc30 0x7fd290152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.789+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 >> 172.21.15.67:0/1230431969 conn(0x7fd2900c35a0 msgr2=0x7fd2900c0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.790+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 shutdown_connections 2024-02-26T10:39:30.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:30.791+0000 7fd296200640 1 -- 172.21.15.67:0/1230431969 wait complete. 2024-02-26T10:39:30.820 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-26T10:39:30.820 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-26T10:39:30.821 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-26T10:39:30.821 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:39:30.821 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-26T10:39:30.821 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-26T10:39:30.822 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-26T10:39:30.822 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:39:30.822 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-26T10:39:30.822 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-26T10:39:30.822 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-26T10:39:30.822 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:39:30.822 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-26T10:39:30.822 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-26T10:39:30.823 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-26T10:39:30.823 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:39:30.839 INFO:tasks.ceph.mds.c:Stopped 2024-02-26T10:39:30.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-26T10:39:30.845 INFO:tasks.ceph.mds.a:Stopped 2024-02-26T10:39:30.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-26T10:39:30.846 INFO:tasks.ceph.mds.d:Stopped 2024-02-26T10:39:30.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-26T10:39:30.852 INFO:tasks.ceph.mds.b:Stopped 2024-02-26T10:39:30.852 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-26T10:39:30.892 INFO:tasks.ceph.mds.c:Started 2024-02-26T10:39:30.892 DEBUG:teuthology.parallel:result is None 2024-02-26T10:39:30.898 INFO:tasks.ceph.mds.a:Started 2024-02-26T10:39:30.898 DEBUG:teuthology.parallel:result is None 2024-02-26T10:39:30.899 INFO:tasks.ceph.mds.d:Started 2024-02-26T10:39:30.899 DEBUG:teuthology.parallel:result is None 2024-02-26T10:39:30.905 INFO:tasks.ceph.mds.b:Started 2024-02-26T10:39:30.905 DEBUG:teuthology.parallel:result is None 2024-02-26T10:39:30.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-26T10:39:31.049 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-02-26T10:39:31.050 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-02-26T10:39:31.051 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:39:31.057 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-02-26T10:39:31.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.095+0000 7f101b800640 1 Processor -- start 2024-02-26T10:39:31.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.095+0000 7f101b800640 1 -- start start 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.096+0000 7f101b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1014150300 0x7f10140c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.096+0000 7f101b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f101406e410 0x7f10140c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.096+0000 7f101b800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1014152e50 con 0x7f1014156230 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.096+0000 7f101b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10141506e0 con 0x7f1014150300 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.096+0000 7f101b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1014153a60 con 0x7f101406e410 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f1019800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1014156230 0x7f10140aaba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40094/0 (socket says 172.21.15.67:40094) 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f1019800640 1 -- 172.21.15.67:0/2290201790 learned_addr learned my addr 172.21.15.67:0/2290201790 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f100be00640 1 --2- 172.21.15.67:0/2290201790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f101406e410 0x7f10140c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:31.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f1018e00640 1 --2- 172.21.15.67:0/2290201790 >> v2:172.21.15.82:3300/0 conn(0x7f1014150300 0x7f10140c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f100be00640 1 -- 172.21.15.67:0/2290201790 >> v2:172.21.15.82:3300/0 conn(0x7f1014150300 msgr2=0x7f10140c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f100be00640 1 --2- 172.21.15.67:0/2290201790 >> v2:172.21.15.82:3300/0 conn(0x7f1014150300 0x7f10140c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f100be00640 1 -- 172.21.15.67:0/2290201790 >> v1:172.21.15.67:6789/0 conn(0x7f1014156230 legacy=0x7f10140aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f1013e00640 1 -- 172.21.15.67:0/2290201790 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973788608 0 0) 0x7f1014152e50 con 0x7f1014156230 2024-02-26T10:39:31.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.097+0000 7f100be00640 1 -- 172.21.15.67:0/2290201790 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014153130 con 0x7f101406e410 2024-02-26T10:39:31.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.098+0000 7f100be00640 1 --2- 172.21.15.67:0/2290201790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f101406e410 0x7f10140c3530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0097a0 tx=0x7f0ffc02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96b3a5377da6205 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f1013e00640 1 -- 172.21.15.67:0/2290201790 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc038070 con 0x7f101406e410 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f1013e00640 1 -- 172.21.15.67:0/2290201790 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc031700 con 0x7f101406e410 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f1013e00640 1 -- 172.21.15.67:0/2290201790 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc031ad0 con 0x7f101406e410 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 -- 172.21.15.67:0/2290201790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f101406e410 msgr2=0x7f10140c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 --2- 172.21.15.67:0/2290201790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f101406e410 0x7f10140c3530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0097a0 tx=0x7f0ffc02b7a0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 -- 172.21.15.67:0/2290201790 shutdown_connections 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 --2- 172.21.15.67:0/2290201790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f101406e410 0x7f10140c3530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 --2- 172.21.15.67:0/2290201790 >> v2:172.21.15.82:3300/0 conn(0x7f1014150300 0x7f10140c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 -- 172.21.15.67:0/2290201790 >> 172.21.15.67:0/2290201790 conn(0x7f10140a84a0 msgr2=0x7f10140a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 -- 172.21.15.67:0/2290201790 shutdown_connections 2024-02-26T10:39:31.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.099+0000 7f101b800640 1 -- 172.21.15.67:0/2290201790 wait complete. 2024-02-26T10:39:31.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.101+0000 7f101b800640 1 Processor -- start 2024-02-26T10:39:31.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.101+0000 7f101b800640 1 -- start start 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f101b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 0x7f10141474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f101b800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1014150300 0x7f101414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f101b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1014156230 0x7f1014146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f101b800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101406dd80 con 0x7f1014150300 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f101b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141536f0 con 0x7f101406e410 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f101b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10140ab2b0 con 0x7f1014156230 2024-02-26T10:39:31.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f100be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1014150300 0x7f101414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.102+0000 7f1018e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 0x7f10141474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 0x7f10141474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34378/0 (socket says 172.21.15.67:34378) 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 -- 172.21.15.67:0/194373030 learned_addr learned my addr 172.21.15.67:0/194373030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1019800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1014156230 0x7f1014146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 -- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1014156230 msgr2=0x7f1014146840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1014156230 0x7f1014146840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 -- 172.21.15.67:0/194373030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1014150300 msgr2=0x7f101414e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1014150300 0x7f101414e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 -- 172.21.15.67:0/194373030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014072260 con 0x7f101406e410 2024-02-26T10:39:31.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1018e00640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 0x7f10141474e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f100400e9e0 tx=0x7f100400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.103+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004017070 con 0x7f101406e410 2024-02-26T10:39:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.104+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100400c9d0 con 0x7f101406e410 2024-02-26T10:39:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.104+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100400cdc0 con 0x7f101406e410 2024-02-26T10:39:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.104+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1014072540 con 0x7f101406e410 2024-02-26T10:39:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.104+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1014072b00 con 0x7f101406e410 2024-02-26T10:39:31.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.105+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014074a30 con 0x7f101406e410 2024-02-26T10:39:31.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.106+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f100401a070 con 0x7f101406e410 2024-02-26T10:39:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.107+0000 7f1011600640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ff006b500 0x7f0ff006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.107+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f100408a200 con 0x7f101406e410 2024-02-26T10:39:31.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.107+0000 7f100be00640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ff006b500 0x7f0ff006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:31.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.108+0000 7f100be00640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ff006b500 0x7f0ff006d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc035000 tx=0x7f0ffc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:31.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.112+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1004058d30 con 0x7f101406e410 2024-02-26T10:39:31.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.332+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f101406e0d0 con 0x7f101406e410 2024-02-26T10:39:31.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.812+0000 7f1011600640 1 -- 172.21.15.67:0/194373030 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f100405cdd0 con 0x7f101406e410 2024-02-26T10:39:31.814 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.816+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ff006b500 msgr2=0x7f0ff006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.816+0000 7f101b800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ff006b500 0x7f0ff006d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc035000 tx=0x7f0ffc008030 comp rx=0 tx=0).stop 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.816+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 msgr2=0x7f10141474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.816+0000 7f101b800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 0x7f10141474e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f100400e9e0 tx=0x7f100400eea0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.816+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 shutdown_connections 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.816+0000 7f101b800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1014156230 0x7f1014146840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.817+0000 7f101b800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1014150300 0x7f101414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.817+0000 7f101b800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0ff006b500 0x7f0ff006d9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.817+0000 7f101b800640 1 --2- 172.21.15.67:0/194373030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f101406e410 0x7f10141474e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.817+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 >> 172.21.15.67:0/194373030 conn(0x7f10140a84a0 msgr2=0x7f10140aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.817+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 shutdown_connections 2024-02-26T10:39:31.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:31.817+0000 7f101b800640 1 -- 172.21.15.67:0/194373030 wait complete. 2024-02-26T10:39:31.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-26T10:39:32.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.035+0000 7fce33e00640 1 Processor -- start 2024-02-26T10:39:32.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.035+0000 7fce33e00640 1 -- start start 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce33e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fce340c2310 0x7fce34153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce33e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce3406b000 0x7fce341524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce33e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce34155f10 con 0x7fce3406ac20 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce33e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3406b470 con 0x7fce340c2310 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce33e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce340c28d0 con 0x7fce3406b000 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce32a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fce3406ac20 0x7fce34152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40098/0 (socket says 172.21.15.67:40098) 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce32a00640 1 -- 172.21.15.67:0/3587239711 learned_addr learned my addr 172.21.15.67:0/3587239711 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:32.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce32000640 1 --2- 172.21.15.67:0/3587239711 >> v2:172.21.15.67:3300/0 conn(0x7fce340c2310 0x7fce34153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.037+0000 7fce33400640 1 --2- 172.21.15.67:0/3587239711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce3406b000 0x7fce341524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.038+0000 7fce31600640 1 -- 172.21.15.67:0/3587239711 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786036045 0 0) 0x7fce34155f10 con 0x7fce3406ac20 2024-02-26T10:39:32.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.038+0000 7fce31600640 1 -- 172.21.15.67:0/3587239711 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce18003520 con 0x7fce3406ac20 2024-02-26T10:39:32.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.038+0000 7fce32000640 1 -- 172.21.15.67:0/3587239711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce3406b000 msgr2=0x7fce341524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:32.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.038+0000 7fce32000640 1 --2- 172.21.15.67:0/3587239711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce3406b000 0x7fce341524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.038+0000 7fce32000640 1 -- 172.21.15.67:0/3587239711 >> v1:172.21.15.67:6789/0 conn(0x7fce3406ac20 legacy=0x7fce34152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.038+0000 7fce32000640 1 -- 172.21.15.67:0/3587239711 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34155150 con 0x7fce340c2310 2024-02-26T10:39:32.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.039+0000 7fce32000640 1 --2- 172.21.15.67:0/3587239711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce340c2310 0x7fce34153630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce200097a0 tx=0x7fce2002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa48018011c0b2a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:32.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.039+0000 7fce31600640 1 -- 172.21.15.67:0/3587239711 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20038070 con 0x7fce340c2310 2024-02-26T10:39:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.039+0000 7fce31600640 1 -- 172.21.15.67:0/3587239711 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce20031700 con 0x7fce340c2310 2024-02-26T10:39:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce31600640 1 -- 172.21.15.67:0/3587239711 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20031ad0 con 0x7fce340c2310 2024-02-26T10:39:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce33e00640 1 -- 172.21.15.67:0/3587239711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce340c2310 msgr2=0x7fce34153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce33e00640 1 --2- 172.21.15.67:0/3587239711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce340c2310 0x7fce34153630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce200097a0 tx=0x7fce2002b7a0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce33e00640 1 -- 172.21.15.67:0/3587239711 shutdown_connections 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce33e00640 1 --2- 172.21.15.67:0/3587239711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce3406b000 0x7fce341524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce33e00640 1 --2- 172.21.15.67:0/3587239711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce340c2310 0x7fce34153630 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.040+0000 7fce33e00640 1 -- 172.21.15.67:0/3587239711 >> 172.21.15.67:0/3587239711 conn(0x7fce340c63f0 msgr2=0x7fce340c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.041+0000 7fce33e00640 1 -- 172.21.15.67:0/3587239711 shutdown_connections 2024-02-26T10:39:32.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.041+0000 7fce33e00640 1 -- 172.21.15.67:0/3587239711 wait complete. 2024-02-26T10:39:32.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.042+0000 7fce33e00640 1 Processor -- start 2024-02-26T10:39:32.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.042+0000 7fce33e00640 1 -- start start 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce33e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 0x7fce340abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce33e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3406b000 0x7fce341ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce33e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce340c2310 0x7fce34147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce33e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce341559f0 con 0x7fce3406ac20 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce33e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34155710 con 0x7fce3406b000 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce33e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce34148850 con 0x7fce340c2310 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce32a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 0x7fce340abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce32a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 0x7fce340abed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49652/0 (socket says 172.21.15.67:49652) 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.043+0000 7fce32a00640 1 -- 172.21.15.67:0/760546190 learned_addr learned my addr 172.21.15.67:0/760546190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce33400640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce340c2310 0x7fce34147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32a00640 1 -- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce340c2310 msgr2=0x7fce34147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32a00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce340c2310 0x7fce34147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32000640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3406b000 0x7fce341ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32a00640 1 -- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3406b000 msgr2=0x7fce341ed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32a00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3406b000 0x7fce341ed9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32a00640 1 -- 172.21.15.67:0/760546190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340ab710 con 0x7fce3406ac20 2024-02-26T10:39:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce32a00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 0x7fce340abed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c390 tx=0x7fce2400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.044+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24017070 con 0x7fce3406ac20 2024-02-26T10:39:32.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.045+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24005ce0 con 0x7fce3406ac20 2024-02-26T10:39:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.045+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240072f0 con 0x7fce3406ac20 2024-02-26T10:39:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.045+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3413e630 con 0x7fce3406ac20 2024-02-26T10:39:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.045+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3413f5b0 con 0x7fce3406ac20 2024-02-26T10:39:32.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.046+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce34074a30 con 0x7fce3406ac20 2024-02-26T10:39:32.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.047+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce2401a070 con 0x7fce3406ac20 2024-02-26T10:39:32.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.048+0000 7fce2aa00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fce1406b4d0 0x7fce1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.048+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce24089230 con 0x7fce3406ac20 2024-02-26T10:39:32.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.049+0000 7fce32000640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fce1406b4d0 0x7fce1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.050+0000 7fce32000640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fce1406b4d0 0x7fce1406d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce20035000 tx=0x7fce20008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:32.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.053+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce24058490 con 0x7fce3406ac20 2024-02-26T10:39:32.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.324+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fce340c20f0 con 0x7fce3406ac20 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.325+0000 7fce2aa00640 1 -- 172.21.15.67:0/760546190 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6065 (secure 0 0 0) 0x7fce24057d90 con 0x7fce3406ac20 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-02-26T10:39:32.326 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlrPwnMBAADC1UqiHSleXCZZ8LRLN/MQ==", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlXEpIGBAANii7fywaaL0+vjyqLX6Atw==", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxl2Z1dBhAAT9aRD1Te0TwHE93ZvNdInA==", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlDx1gJxAAYGDhukySlrxjHaHWn+707g==", 2024-02-26T10:39:32.328 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDzadxlki0qBBAAWmQAFvxfJJ5kYMrk0ChwLQ==", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.329 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD2adxl4X5OHRAAHtaf00M+kXgvYOn1/ddr7A==", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD5adxlQqxnNxAAWzDQ/1DzO3r1ztTrociQzQ==", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD9adxlgCXOFRAAHdACX7JyyLLY0KPVHOycGw==", 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAEatxlJ8FqFhAA2XFN8d8NUYW1rITUu3tQnA==", 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.331 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAHatxlO9YRMBAA6z8hYbTwHnU0BV3OMxKSjA==", 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAKatxlzGhIORAANuWy2QCgtFcJRO4vQV4EkQ==", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAOatxlUwuJGBAAYZlIHvgD3suXzh5kyeUWqQ==", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw==", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw==", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:39:32.334 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow r", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rwx" 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDsadxlvPgjIxAAClmsDQ2hetAdBGNLgj50lg==", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.335 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-mds", 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAXatxl/I/ZAhAA7sBHfVw5wRjiU4s967qmbw==", 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-mds" 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-mgr", 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAXatxltZnZAhAAunoQzI1nODkpm+qnqTaBBA==", 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-mgr" 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-osd", 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAXatxlq6LZAhAAQ1o6vx/tXY1d9WP/KKf98g==", 2024-02-26T10:39:32.336 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-osd" 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rbd", 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAXatxlPq3ZAhAAm/ciXoDIWaY6dC2ldAkXPg==", 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rbd" 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAXatxleLbZAhAAi9yEX1JeNOnulZh2qK69WQ==", 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.bootstrap-rgw", 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAXatxla7/ZAhAA1ag9pYVvTv5pBtFzOcnk7g==", 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile bootstrap-rgw" 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlQMXpIxAA0W1caz5FAL2wDMrNG3j5Mg==", 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.338 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlMa/TFxAAAa7erEK+tXM+k5KOfx2JgA==", 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-02-26T10:39:32.339 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fce1406b4d0 msgr2=0x7fce1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fce1406b4d0 0x7fce1406d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce20035000 tx=0x7fce20008030 comp rx=0 tx=0).stop 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 msgr2=0x7fce340abed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 0x7fce340abed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c390 tx=0x7fce2400c850 comp rx=0 tx=0).stop 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 shutdown_connections 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce340c2310 0x7fce34147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fce1406b4d0 0x7fce1406d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3406b000 0x7fce341ed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 --2- 172.21.15.67:0/760546190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce3406ac20 0x7fce340abed0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 >> 172.21.15.67:0/760546190 conn(0x7fce340c63f0 msgr2=0x7fce34146230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:32.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 shutdown_connections 2024-02-26T10:39:32.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.328+0000 7fce33e00640 1 -- 172.21.15.67:0/760546190 wait complete. 2024-02-26T10:39:32.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-02-26T10:39:32.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.569+0000 7f909b200640 1 Processor -- start 2024-02-26T10:39:32.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.569+0000 7f909b200640 1 -- start start 2024-02-26T10:39:32.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.570+0000 7f909b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f909c0c1c10 0x7f909c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f909b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c155cf0 con 0x7f909c0c0850 2024-02-26T10:39:32.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f909b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c06dca0 con 0x7f909c06e180 2024-02-26T10:39:32.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f909b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0c2a70 con 0x7f909c0c1c10 2024-02-26T10:39:32.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f9099400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f909c0c0850 0x7f909c06ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40102/0 (socket says 172.21.15.67:40102) 2024-02-26T10:39:32.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f9099400640 1 -- 172.21.15.67:0/3402788417 learned_addr learned my addr 172.21.15.67:0/3402788417 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f9099e00640 1 --2- 172.21.15.67:0/3402788417 >> v2:172.21.15.82:3300/0 conn(0x7f909c0c1c10 0x7f909c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860936874 0 0) 0x7f9084001110 con 0x7f909c0c0850 2024-02-26T10:39:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.571+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9080003520 con 0x7f909c0c0850 2024-02-26T10:39:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973366451 0 0) 0x7f9080003520 con 0x7f909c0c0850 2024-02-26T10:39:32.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 >> v2:172.21.15.82:3300/0 conn(0x7f909c0c1c10 msgr2=0x7f909c1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 --2- 172.21.15.67:0/3402788417 >> v2:172.21.15.82:3300/0 conn(0x7f909c0c1c10 0x7f909c1557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 >> v1:172.21.15.82:6789/0 conn(0x7f909c06e180 legacy=0x7f909c15f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0c21d0 con 0x7f909c0c0850 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f9084001840 con 0x7f909c0c0850 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 >> v1:172.21.15.67:6789/0 conn(0x7f909c0c0850 legacy=0x7f909c06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9080003a00 0x7f9080003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9080004800 0x7f9080004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90800052e0 0x7f908000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c2a70 con 0x7f90800052e0 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908000c1e0 con 0x7f9080004800 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908000c3e0 con 0x7f9080003a00 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.572+0000 7f909a800640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90800052e0 0x7f908000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f9099e00640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9080003a00 0x7f9080003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f9099400640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9080004800 0x7f9080004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f909a800640 1 -- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9080003a00 msgr2=0x7f9080003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:32.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f909a800640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9080003a00 0x7f9080003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f909a800640 1 -- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9080004800 msgr2=0x7f9080004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f909a800640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9080004800 0x7f9080004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f909a800640 1 -- 172.21.15.67:0/3402788417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080003520 con 0x7f90800052e0 2024-02-26T10:39:32.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.573+0000 7f9099400640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9080004800 0x7f9080004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.574+0000 7f909a800640 1 -- 172.21.15.67:0/3402788417 reap_dead start 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.574+0000 7f909a800640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90800052e0 0x7f908000b9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9094001e50 tx=0x7f9080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a115e9f0c726c01 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.574+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094001a70 con 0x7f90800052e0 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.574+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9094003bb0 con 0x7f90800052e0 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.574+0000 7f9098a00640 1 -- 172.21.15.67:0/3402788417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940050d0 con 0x7f90800052e0 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 -- 172.21.15.67:0/3402788417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90800052e0 msgr2=0x7f908000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90800052e0 0x7f908000b9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9094001e50 tx=0x7f9080005830 comp rx=0 tx=0).stop 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 -- 172.21.15.67:0/3402788417 shutdown_connections 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 --2- 172.21.15.67:0/3402788417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90800052e0 0x7f908000b9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 -- 172.21.15.67:0/3402788417 >> 172.21.15.67:0/3402788417 conn(0x7f909c0c63f0 msgr2=0x7f909c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 -- 172.21.15.67:0/3402788417 shutdown_connections 2024-02-26T10:39:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.575+0000 7f909b200640 1 -- 172.21.15.67:0/3402788417 wait complete. 2024-02-26T10:39:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.577+0000 7f909b200640 1 Processor -- start 2024-02-26T10:39:32.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.577+0000 7f909b200640 1 -- start start 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f90800052e0 0x7f909c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 0x7f909c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f909c1ee560 0x7f909c1ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c0df0 con 0x7f909c14d710 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c2790 con 0x7f909c1ee560 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c0f70 con 0x7f90800052e0 2024-02-26T10:39:32.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 0x7f909c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 0x7f909c14db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49680/0 (socket says 172.21.15.67:49680) 2024-02-26T10:39:32.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099400640 1 -- 172.21.15.67:0/1490026854 learned_addr learned my addr 172.21.15.67:0/1490026854 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099e00640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f90800052e0 0x7f909c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f909a800640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f909c1ee560 0x7f909c1ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099400640 1 -- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f90800052e0 msgr2=0x7f909c151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099400640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f90800052e0 0x7f909c151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.578+0000 7f9099400640 1 -- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f909c1ee560 msgr2=0x7f909c1ee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9099400640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f909c1ee560 0x7f909c1ee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9099400640 1 -- 172.21.15.67:0/1490026854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c152090 con 0x7f909c14d710 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9099e00640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f90800052e0 0x7f909c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9099400640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 0x7f909c14db90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f908400d7a0 tx=0x7f908400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084019070 con 0x7f909c14d710 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908400de50 con 0x7f909c14d710 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c152370 con 0x7f909c14d710 2024-02-26T10:39:32.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840047e0 con 0x7f909c14d710 2024-02-26T10:39:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.579+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c1ee040 con 0x7f909c14d710 2024-02-26T10:39:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.580+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c074a30 con 0x7f909c14d710 2024-02-26T10:39:32.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.581+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f908401c070 con 0x7f909c14d710 2024-02-26T10:39:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.582+0000 7f9092000640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f907806b4d0 0x7f907806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:32.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.582+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f90840899a0 con 0x7f909c14d710 2024-02-26T10:39:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.583+0000 7f9099e00640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f907806b4d0 0x7f907806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.584+0000 7f9099e00640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f907806b4d0 0x7f907806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f909c14e680 tx=0x7f908c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.586+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f909c155cf0 con 0x7f909c14d710 2024-02-26T10:39:32.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.868+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f909c0c60e0 con 0x7f909c14d710 2024-02-26T10:39:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.873+0000 7f9092000640 1 -- 172.21.15.67:0/1490026854 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9084058b60 con 0x7f909c14d710 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.876+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f907806b4d0 msgr2=0x7f907806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.876+0000 7f909b200640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f907806b4d0 0x7f907806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f909c14e680 tx=0x7f908c008040 comp rx=0 tx=0).stop 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 msgr2=0x7f909c14db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 0x7f909c14db90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f908400d7a0 tx=0x7f908400dc60 comp rx=0 tx=0).stop 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 shutdown_connections 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f907806b4d0 0x7f907806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f909c1ee560 0x7f909c1ee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f909c14d710 0x7f909c14db90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 --2- 172.21.15.67:0/1490026854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f90800052e0 0x7f909c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 >> 172.21.15.67:0/1490026854 conn(0x7f909c0c63f0 msgr2=0x7f909c06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 shutdown_connections 2024-02-26T10:39:32.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:32.877+0000 7f909b200640 1 -- 172.21.15.67:0/1490026854 wait complete. 2024-02-26T10:39:32.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-02-26T10:39:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.118+0000 7ff78a200640 1 Processor -- start 2024-02-26T10:39:33.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.119+0000 7ff78a200640 1 -- start start 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.120+0000 7ff78a200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff7840a94a0 0x7ff78406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.120+0000 7ff78a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff7840abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.120+0000 7ff78a200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78406df90 con 0x7ff78406ac20 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.120+0000 7ff78a200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7840a9a80 con 0x7ff7840a94a0 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.120+0000 7ff78a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840a9d60 con 0x7ff78406db00 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.121+0000 7ff788e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff78406ac20 0x7ff784153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:51688/0 (socket says 172.21.15.67:51688) 2024-02-26T10:39:33.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.121+0000 7ff788e00640 1 -- 172.21.15.67:0/1126592004 learned_addr learned my addr 172.21.15.67:0/1126592004 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.121+0000 7ff789800640 1 --2- 172.21.15.67:0/1126592004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff7840abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.121+0000 7ff783e00640 1 --2- 172.21.15.67:0/1126592004 >> v2:172.21.15.82:3300/0 conn(0x7ff7840a94a0 0x7ff78406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.122+0000 7ff783400640 1 -- 172.21.15.67:0/1126592004 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125872287 0 0) 0x7ff78406df90 con 0x7ff78406ac20 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.122+0000 7ff783400640 1 -- 172.21.15.67:0/1126592004 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff768003520 con 0x7ff78406ac20 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.122+0000 7ff789800640 1 -- 172.21.15.67:0/1126592004 >> v2:172.21.15.82:3300/0 conn(0x7ff7840a94a0 msgr2=0x7ff78406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.122+0000 7ff789800640 1 --2- 172.21.15.67:0/1126592004 >> v2:172.21.15.82:3300/0 conn(0x7ff7840a94a0 0x7ff78406b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.122+0000 7ff789800640 1 -- 172.21.15.67:0/1126592004 >> v1:172.21.15.82:6789/0 conn(0x7ff78406ac20 legacy=0x7ff784153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.122+0000 7ff789800640 1 -- 172.21.15.67:0/1126592004 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840ab750 con 0x7ff78406db00 2024-02-26T10:39:33.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.124+0000 7ff789800640 1 --2- 172.21.15.67:0/1126592004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff7840abf50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0097a0 tx=0x7ff77c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d7dc311216128a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.124+0000 7ff783400640 1 -- 172.21.15.67:0/1126592004 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c035020 con 0x7ff78406db00 2024-02-26T10:39:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.124+0000 7ff783400640 1 -- 172.21.15.67:0/1126592004 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c031830 con 0x7ff78406db00 2024-02-26T10:39:33.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.124+0000 7ff783400640 1 -- 172.21.15.67:0/1126592004 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c031be0 con 0x7ff78406db00 2024-02-26T10:39:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 -- 172.21.15.67:0/1126592004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 msgr2=0x7ff7840abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 --2- 172.21.15.67:0/1126592004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff7840abf50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0097a0 tx=0x7ff77c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 -- 172.21.15.67:0/1126592004 shutdown_connections 2024-02-26T10:39:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 --2- 172.21.15.67:0/1126592004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff7840abf50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 --2- 172.21.15.67:0/1126592004 >> v2:172.21.15.82:3300/0 conn(0x7ff7840a94a0 0x7ff78406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 -- 172.21.15.67:0/1126592004 >> 172.21.15.67:0/1126592004 conn(0x7ff7840c63f0 msgr2=0x7ff7840c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:33.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 -- 172.21.15.67:0/1126592004 shutdown_connections 2024-02-26T10:39:33.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.125+0000 7ff78a200640 1 -- 172.21.15.67:0/1126592004 wait complete. 2024-02-26T10:39:33.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.127+0000 7ff78a200640 1 Processor -- start 2024-02-26T10:39:33.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.127+0000 7ff78a200640 1 -- start start 2024-02-26T10:39:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff78a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78406ac20 0x7ff78414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff78a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff784156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff78a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 0x7ff784154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff78a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784150410 con 0x7ff7840a94a0 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff78a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840ac490 con 0x7ff78406ac20 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff78a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff784150590 con 0x7ff78406db00 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 0x7ff784154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 0x7ff784154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49682/0 (socket says 172.21.15.67:49682) 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 -- 172.21.15.67:0/2283672304 learned_addr learned my addr 172.21.15.67:0/2283672304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff783e00640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff784156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff788e00640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78406ac20 0x7ff78414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 -- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 msgr2=0x7ff784156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff784156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 -- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78406ac20 msgr2=0x7ff78414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78406ac20 0x7ff78414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.128+0000 7ff789800640 1 -- 172.21.15.67:0/2283672304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78414df00 con 0x7ff7840a94a0 2024-02-26T10:39:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.129+0000 7ff789800640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 0x7ff784154260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098c0 tx=0x7ff77c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.129+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c035040 con 0x7ff7840a94a0 2024-02-26T10:39:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.129+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c03f070 con 0x7ff7840a94a0 2024-02-26T10:39:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.129+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78414d380 con 0x7ff7840a94a0 2024-02-26T10:39:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.129+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c03ba60 con 0x7ff7840a94a0 2024-02-26T10:39:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.129+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78414dc20 con 0x7ff7840a94a0 2024-02-26T10:39:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.130+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78406b750 con 0x7ff7840a94a0 2024-02-26T10:39:33.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.131+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff77c038020 con 0x7ff7840a94a0 2024-02-26T10:39:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.132+0000 7ff780c00640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff76406b4d0 0x7ff76406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.132+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff77c0ab9b0 con 0x7ff7840a94a0 2024-02-26T10:39:33.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.133+0000 7ff788e00640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff76406b4d0 0x7ff76406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.134+0000 7ff788e00640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff76406b4d0 0x7ff76406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff774005c00 tx=0x7ff774009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:33.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.136+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff78406b750 con 0x7ff7840a94a0 2024-02-26T10:39:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.396+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff78406bea0 con 0x7ff7840a94a0 2024-02-26T10:39:33.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.401+0000 7ff780c00640 1 -- 172.21.15.67:0/2283672304 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff78406bea0 con 0x7ff7840a94a0 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff76406b4d0 msgr2=0x7ff76406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff76406b4d0 0x7ff76406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff774005c00 tx=0x7ff774009040 comp rx=0 tx=0).stop 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 msgr2=0x7ff784154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 0x7ff784154260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098c0 tx=0x7ff77c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 shutdown_connections 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7840a94a0 0x7ff784154260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff78406db00 0x7ff784156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff76406b4d0 0x7ff76406d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 --2- 172.21.15.67:0/2283672304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78406ac20 0x7ff78414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 >> 172.21.15.67:0/2283672304 conn(0x7ff7840c63f0 msgr2=0x7ff78414e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 shutdown_connections 2024-02-26T10:39:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.404+0000 7ff78a200640 1 -- 172.21.15.67:0/2283672304 wait complete. 2024-02-26T10:39:33.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-02-26T10:39:33.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.638+0000 7f75d2a00640 1 Processor -- start 2024-02-26T10:39:33.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.639+0000 7f75d2a00640 1 -- start start 2024-02-26T10:39:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.640+0000 7f75d2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75cc0c2310 0x7f75cc153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.640+0000 7f75d2a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc155f10 con 0x7f75cc06ac20 2024-02-26T10:39:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.640+0000 7f75d2a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0c28d0 con 0x7f75cc06b000 2024-02-26T10:39:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.640+0000 7f75d2a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc06b430 con 0x7f75cc0c2310 2024-02-26T10:39:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75d1600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f75cc06ac20 0x7f75cc152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40118/0 (socket says 172.21.15.67:40118) 2024-02-26T10:39:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75d1600640 1 -- 172.21.15.67:0/1511806651 learned_addr learned my addr 172.21.15.67:0/1511806651 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088348116 0 0) 0x7f75cc155f10 con 0x7f75cc06ac20 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b4003520 con 0x7f75cc06ac20 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75d0c00640 1 --2- 172.21.15.67:0/1511806651 >> v2:172.21.15.82:3300/0 conn(0x7f75cc0c2310 0x7f75cc153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3658022640 0 0) 0x7f75b4003520 con 0x7f75cc06ac20 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 >> v2:172.21.15.82:3300/0 conn(0x7f75cc0c2310 msgr2=0x7f75cc153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.641+0000 7f75cbe00640 1 --2- 172.21.15.67:0/1511806651 >> v2:172.21.15.82:3300/0 conn(0x7f75cc0c2310 0x7f75cc153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 >> v1:172.21.15.82:6789/0 conn(0x7f75cc06b000 legacy=0x7f75cc15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc155150 con 0x7f75cc06ac20 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f75c0001670 con 0x7f75cc06ac20 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 >> v1:172.21.15.67:6789/0 conn(0x7f75cc06ac20 legacy=0x7f75cc152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40053c0 0x7f75b400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc06b430 con 0x7f75b4004800 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400e460 con 0x7f75b40053c0 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400e660 con 0x7f75b40039b0 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d0c00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d2000640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40053c0 0x7f75b400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d1600640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d0c00640 1 -- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b40039b0 msgr2=0x7f75b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d0c00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d0c00640 1 -- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40053c0 msgr2=0x7f75b400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d0c00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40053c0 0x7f75b400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d0c00640 1 -- 172.21.15.67:0/1511806651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4003520 con 0x7f75b4004800 2024-02-26T10:39:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.642+0000 7f75d1600640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.643+0000 7f75d2000640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40053c0 0x7f75b400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.643+0000 7f75d2000640 1 -- 172.21.15.67:0/1511806651 reap_dead start 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75d0c00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75b4004c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ba30 tx=0x7f75b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27118781439d2a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8036020 con 0x7f75b4004800 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b8032630 con 0x7f75b4004800 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75cbe00640 1 -- 172.21.15.67:0/1511806651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b800b600 con 0x7f75b4004800 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75d2a00640 1 -- 172.21.15.67:0/1511806651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 msgr2=0x7f75b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75d2a00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75b4004c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ba30 tx=0x7f75b4005830 comp rx=0 tx=0).stop 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75d2a00640 1 -- 172.21.15.67:0/1511806651 shutdown_connections 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75d2a00640 1 --2- 172.21.15.67:0/1511806651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75b4004c80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.644+0000 7f75d2a00640 1 -- 172.21.15.67:0/1511806651 >> 172.21.15.67:0/1511806651 conn(0x7f75cc0c63f0 msgr2=0x7f75cc0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.645+0000 7f75d2a00640 1 -- 172.21.15.67:0/1511806651 shutdown_connections 2024-02-26T10:39:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.645+0000 7f75d2a00640 1 -- 172.21.15.67:0/1511806651 wait complete. 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.646+0000 7f75d2a00640 1 Processor -- start 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.646+0000 7f75d2a00640 1 -- start start 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75cc0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75cc0c5ea0 0x7f75cc071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75cc1ee020 0x7f75cc071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1559f0 con 0x7f75b4004800 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc155710 con 0x7f75cc0c5ea0 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc148850 con 0x7f75cc1ee020 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75cc0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75cc0bfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49698/0 (socket says 172.21.15.67:49698) 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.647+0000 7f75d1600640 1 -- 172.21.15.67:0/3871011010 learned_addr learned my addr 172.21.15.67:0/3871011010 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d1600640 1 -- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75cc1ee020 msgr2=0x7f75cc071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d1600640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75cc1ee020 0x7f75cc071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d1600640 1 -- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75cc0c5ea0 msgr2=0x7f75cc071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d0c00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75cc0c5ea0 0x7f75cc071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d1600640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75cc0c5ea0 0x7f75cc071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d1600640 1 -- 172.21.15.67:0/3871011010 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc0c0480 con 0x7f75b4004800 2024-02-26T10:39:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d1600640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75cc0bfa90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75c000d7a0 tx=0x7f75c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0019070 con 0x7f75b4004800 2024-02-26T10:39:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c00074e0 con 0x7f75b4004800 2024-02-26T10:39:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00049e0 con 0x7f75b4004800 2024-02-26T10:39:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.648+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc1eae00 con 0x7f75b4004800 2024-02-26T10:39:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.649+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1eb3c0 con 0x7f75b4004800 2024-02-26T10:39:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.650+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc074a30 con 0x7f75b4004800 2024-02-26T10:39:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.651+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75c001c070 con 0x7f75b4004800 2024-02-26T10:39:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.652+0000 7f75c9600640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.652+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75c0089810 con 0x7f75b4004800 2024-02-26T10:39:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.652+0000 7f75d0c00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.653+0000 7f75d0c00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ba00 tx=0x7f75b8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:33.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.656+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75c0058a70 con 0x7f75b4004800 2024-02-26T10:39:33.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.918+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f75cc0c20f0 con 0x7f75b4004800 2024-02-26T10:39:33.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.923+0000 7f75c9600640 1 -- 172.21.15.67:0/3871011010 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f75c0058370 con 0x7f75b4004800 2024-02-26T10:39:33.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 msgr2=0x7f75ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ba00 tx=0x7f75b8030000 comp rx=0 tx=0).stop 2024-02-26T10:39:33.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 msgr2=0x7f75cc0bfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:33.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75cc0bfa90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75c000d7a0 tx=0x7f75c000dc60 comp rx=0 tx=0).stop 2024-02-26T10:39:33.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 shutdown_connections 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75cc1ee020 0x7f75cc071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75cc0c5ea0 0x7f75cc071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 --2- 172.21.15.67:0/3871011010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b4004800 0x7f75cc0bfa90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 >> 172.21.15.67:0/3871011010 conn(0x7f75cc0c63f0 msgr2=0x7f75cc0c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 shutdown_connections 2024-02-26T10:39:33.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:33.926+0000 7f75d2a00640 1 -- 172.21.15.67:0/3871011010 wait complete. 2024-02-26T10:39:33.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-02-26T10:39:34.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.159+0000 7fb8bd000640 1 Processor -- start 2024-02-26T10:39:34.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.159+0000 7fb8bd000640 1 -- start start 2024-02-26T10:39:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8bd000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb8b806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8bd000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80a9a80 con 0x7fb8b806ac20 2024-02-26T10:39:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8bd000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80a9d60 con 0x7fb8b80a94a0 2024-02-26T10:39:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8bd000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b806b490 con 0x7fb8b806b000 2024-02-26T10:39:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8b7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb8b806ac20 0x7fb8b8153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40128/0 (socket says 172.21.15.67:40128) 2024-02-26T10:39:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8b7400640 1 -- 172.21.15.67:0/745025632 learned_addr learned my addr 172.21.15.67:0/745025632 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855927848 0 0) 0x7fb8b80a9a80 con 0x7fb8b806ac20 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.161+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb89c003520 con 0x7fb8b806ac20 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b7e00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb8b806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169717690 0 0) 0x7fb89c003520 con 0x7fb8b806ac20 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 msgr2=0x7fb8b806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb8b806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 >> v1:172.21.15.82:6789/0 conn(0x7fb8b80a94a0 legacy=0x7fb8b80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80ab750 con 0x7fb8b806ac20 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fb8a8001670 con 0x7fb8b806ac20 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 >> v1:172.21.15.67:6789/0 conn(0x7fb8b806ac20 legacy=0x7fb8b8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb89c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb89c004060 0x7fb89c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb89c004d40 0x7fb89c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b806b490 con 0x7fb89c004d40 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c00bc00 con 0x7fb89c004060 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c00be00 con 0x7fb8b806b000 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.162+0000 7fb8b7e00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb89c004d40 0x7fb89c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7400640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb89c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7e00640 1 -- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 msgr2=0x7fb89c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7e00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb89c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7e00640 1 -- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb89c004060 msgr2=0x7fb89c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b6a00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb89c004060 0x7fb89c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7e00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb89c004060 0x7fb89c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7e00640 1 -- 172.21.15.67:0/745025632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c003520 con 0x7fb89c004d40 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b7400640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806b000 0x7fb89c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.163+0000 7fb8b6a00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb89c004060 0x7fb89c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.164+0000 7fb8b7e00640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb89c004d40 0x7fb89c0051c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007360 tx=0x7fb89c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e95e86aad7e55f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:34.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.164+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0039070 con 0x7fb89c004d40 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.164+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b002ccb0 con 0x7fb89c004d40 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.164+0000 7fb8b6000640 1 -- 172.21.15.67:0/745025632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0009be0 con 0x7fb89c004d40 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8bd000640 1 -- 172.21.15.67:0/745025632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb89c004d40 msgr2=0x7fb89c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8bd000640 1 --2- 172.21.15.67:0/745025632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb89c004d40 0x7fb89c0051c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007360 tx=0x7fb89c005830 comp rx=0 tx=0).stop 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8b7e00640 1 -- 172.21.15.67:0/745025632 reap_dead start 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8bd000640 1 -- 172.21.15.67:0/745025632 shutdown_connections 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8bd000640 1 -- 172.21.15.67:0/745025632 >> 172.21.15.67:0/745025632 conn(0x7fb8b80c63f0 msgr2=0x7fb8b80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8bd000640 1 -- 172.21.15.67:0/745025632 shutdown_connections 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.165+0000 7fb8bd000640 1 -- 172.21.15.67:0/745025632 wait complete. 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.166+0000 7fb8bd000640 1 Processor -- start 2024-02-26T10:39:34.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.167+0000 7fb8bd000640 1 -- start start 2024-02-26T10:39:34.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.167+0000 7fb8bd000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b8145c50 0x7fb8b8143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.167+0000 7fb8bd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 0x7fb8b8143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.167+0000 7fb8bd000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b81444f0 0x7fb8b8142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.167+0000 7fb8bd000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8150590 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.167+0000 7fb8bd000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8150410 con 0x7fb8b8145c50 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8bd000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b806db00 con 0x7fb8b81444f0 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 0x7fb8b8143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 0x7fb8b8143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49710/0 (socket says 172.21.15.67:49710) 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 -- 172.21.15.67:0/2576352483 learned_addr learned my addr 172.21.15.67:0/2576352483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b7400640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b8145c50 0x7fb8b8143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 -- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b81444f0 msgr2=0x7fb8b8142230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b7e00640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b81444f0 0x7fb8b8142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b81444f0 0x7fb8b8142230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 -- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b8145c50 msgr2=0x7fb8b8143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b8145c50 0x7fb8b8143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b6a00640 1 -- 172.21.15.67:0/2576352483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b814e5d0 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.168+0000 7fb8b7400640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b8145c50 0x7fb8b8143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:34.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.169+0000 7fb8b6a00640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 0x7fb8b8143fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4000f60 tx=0x7fb8a400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.169+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4005480 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.169+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a4017070 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.169+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b814e8b0 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.169+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4003810 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.169+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b8147e30 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.170+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b814ffc0 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.171+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8a40039b0 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.172+0000 7fb8af400640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b500 0x7fb89806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.172+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb8a4089c80 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.172+0000 7fb8b7400640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b500 0x7fb89806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.174+0000 7fb8b7400640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b500 0x7fb89806d9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8007a70 tx=0x7fb8a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:34.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.176+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8b814ffc0 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.449+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fb8b8074a30 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.454+0000 7fb8af400640 1 -- 172.21.15.67:0/2576352483 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb8a4058e40 con 0x7fb8b81ec9e0 2024-02-26T10:39:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b500 msgr2=0x7fb89806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b500 0x7fb89806d9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8007a70 tx=0x7fb8a8009040 comp rx=0 tx=0).stop 2024-02-26T10:39:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 msgr2=0x7fb8b8143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 0x7fb8b8143fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4000f60 tx=0x7fb8a400d9f0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 shutdown_connections 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b81444f0 0x7fb8b8142230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b81ec9e0 0x7fb8b8143fb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b500 0x7fb89806d9c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 --2- 172.21.15.67:0/2576352483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b8145c50 0x7fb8b8143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 >> 172.21.15.67:0/2576352483 conn(0x7fb8b80c63f0 msgr2=0x7fb8b80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 shutdown_connections 2024-02-26T10:39:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.457+0000 7fb8bd000640 1 -- 172.21.15.67:0/2576352483 wait complete. 2024-02-26T10:39:34.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-02-26T10:39:34.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.694+0000 7f8aa9800640 1 Processor -- start 2024-02-26T10:39:34.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.694+0000 7f8aa9800640 1 -- start start 2024-02-26T10:39:34.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.696+0000 7f8aa9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa406ac20 0x7f8aa40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.696+0000 7f8aa9800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8aa406b000 0x7f8aa406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.696+0000 7f8aa9800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa406b460 con 0x7f8aa4153610 2024-02-26T10:39:34.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.696+0000 7f8aa9800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa40a9a80 con 0x7f8aa406b000 2024-02-26T10:39:34.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.696+0000 7f8aa9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa40a9d60 con 0x7f8aa406ac20 2024-02-26T10:39:34.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.697+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa406ac20 0x7f8aa40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.697+0000 7f8aa8e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8aa4153610 0x7f8aa40abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:51714/0 (socket says 172.21.15.67:51714) 2024-02-26T10:39:34.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.697+0000 7f8aa8e00640 1 -- 172.21.15.67:0/2873233251 learned_addr learned my addr 172.21.15.67:0/2873233251 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:34.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.697+0000 7f8aa3400640 1 --2- 172.21.15.67:0/2873233251 >> v2:172.21.15.82:3300/0 conn(0x7f8aa406b000 0x7f8aa406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa2a00640 1 -- 172.21.15.67:0/2873233251 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24763125 0 0) 0x7f8aa406b460 con 0x7f8aa4153610 2024-02-26T10:39:34.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa2a00640 1 -- 172.21.15.67:0/2873233251 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a88003520 con 0x7f8aa4153610 2024-02-26T10:39:34.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa3e00640 1 -- 172.21.15.67:0/2873233251 >> v2:172.21.15.82:3300/0 conn(0x7f8aa406b000 msgr2=0x7f8aa406db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:34.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa3e00640 1 --2- 172.21.15.67:0/2873233251 >> v2:172.21.15.82:3300/0 conn(0x7f8aa406b000 0x7f8aa406db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa3e00640 1 -- 172.21.15.67:0/2873233251 >> v1:172.21.15.82:6789/0 conn(0x7f8aa4153610 legacy=0x7f8aa40abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa3e00640 1 -- 172.21.15.67:0/2873233251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40ab750 con 0x7f8aa406ac20 2024-02-26T10:39:34.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.698+0000 7f8aa3400640 1 --2- 172.21.15.67:0/2873233251 >> v2:172.21.15.82:3300/0 conn(0x7f8aa406b000 0x7f8aa406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:39:34.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.700+0000 7f8aa3e00640 1 --2- 172.21.15.67:0/2873233251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa406ac20 0x7f8aa40a94a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0097a0 tx=0x7f8a8c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad14744eff70ca90 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.700+0000 7f8aa2a00640 1 -- 172.21.15.67:0/2873233251 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c035020 con 0x7f8aa406ac20 2024-02-26T10:39:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa2a00640 1 -- 172.21.15.67:0/2873233251 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c031830 con 0x7f8aa406ac20 2024-02-26T10:39:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa2a00640 1 -- 172.21.15.67:0/2873233251 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c031c00 con 0x7f8aa406ac20 2024-02-26T10:39:34.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa9800640 1 -- 172.21.15.67:0/2873233251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa406ac20 msgr2=0x7f8aa40a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa9800640 1 --2- 172.21.15.67:0/2873233251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa406ac20 0x7f8aa40a94a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0097a0 tx=0x7f8a8c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa9800640 1 -- 172.21.15.67:0/2873233251 shutdown_connections 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa9800640 1 --2- 172.21.15.67:0/2873233251 >> v2:172.21.15.82:3300/0 conn(0x7f8aa406b000 0x7f8aa406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa9800640 1 --2- 172.21.15.67:0/2873233251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa406ac20 0x7f8aa40a94a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.701+0000 7f8aa9800640 1 -- 172.21.15.67:0/2873233251 >> 172.21.15.67:0/2873233251 conn(0x7f8aa40c63f0 msgr2=0x7f8aa40c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.702+0000 7f8aa9800640 1 -- 172.21.15.67:0/2873233251 shutdown_connections 2024-02-26T10:39:34.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.702+0000 7f8aa9800640 1 -- 172.21.15.67:0/2873233251 wait complete. 2024-02-26T10:39:34.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.703+0000 7f8aa9800640 1 Processor -- start 2024-02-26T10:39:34.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.703+0000 7f8aa9800640 1 -- start start 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.704+0000 7f8aa9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8aa406ac20 0x7f8aa4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.704+0000 7f8aa9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 0x7f8aa41deee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.704+0000 7f8aa9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa4153610 0x7f8aa41df420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.704+0000 7f8aa9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4150410 con 0x7f8aa406b000 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.704+0000 7f8aa9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4153aa0 con 0x7f8aa406ac20 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.704+0000 7f8aa9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa4150590 con 0x7f8aa4153610 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 0x7f8aa41deee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 0x7f8aa41deee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49724/0 (socket says 172.21.15.67:49724) 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 -- 172.21.15.67:0/3684257709 learned_addr learned my addr 172.21.15.67:0/3684257709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3e00640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8aa406ac20 0x7f8aa4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa8e00640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa4153610 0x7f8aa41df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 -- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa4153610 msgr2=0x7f8aa41df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa4153610 0x7f8aa41df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 -- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8aa406ac20 msgr2=0x7f8aa4152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8aa406ac20 0x7f8aa4152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 -- 172.21.15.67:0/3684257709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4154e30 con 0x7f8aa406b000 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.705+0000 7f8aa3400640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 0x7f8aa41deee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400efc0 tx=0x7f8a9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.706+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94018020 con 0x7f8aa406b000 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.706+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94014680 con 0x7f8aa406b000 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.706+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94014a50 con 0x7f8aa406b000 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.706+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa4157040 con 0x7f8aa406b000 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.706+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa41e8870 con 0x7f8aa406b000 2024-02-26T10:39:34.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.707+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa4074a30 con 0x7f8aa406b000 2024-02-26T10:39:34.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.708+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a94005d10 con 0x7f8aa406b000 2024-02-26T10:39:34.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.709+0000 7f8a9be00640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:34.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.709+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a9408ad90 con 0x7f8aa406b000 2024-02-26T10:39:34.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.709+0000 7f8aa3e00640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:34.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.711+0000 7f8aa3e00640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a8006b4d0 0x7f8a8006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c036000 tx=0x7f8a8c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:34.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.713+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a94022030 con 0x7f8aa406b000 2024-02-26T10:39:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.986+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8aa40c0460 con 0x7f8aa406b000 2024-02-26T10:39:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.991+0000 7f8a9be00640 1 -- 172.21.15.67:0/3684257709 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8aa4154e30 con 0x7f8aa406b000 2024-02-26T10:39:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a8006b4d0 msgr2=0x7f8a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a8006b4d0 0x7f8a8006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c036000 tx=0x7f8a8c03c040 comp rx=0 tx=0).stop 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 msgr2=0x7f8aa41deee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 0x7f8aa41deee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400efc0 tx=0x7f8a9400c490 comp rx=0 tx=0).stop 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 shutdown_connections 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8aa4153610 0x7f8aa41df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa406b000 0x7f8aa41deee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 --2- 172.21.15.67:0/3684257709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8aa406ac20 0x7f8aa4152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 >> 172.21.15.67:0/3684257709 conn(0x7f8aa40c63f0 msgr2=0x7f8aa40abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 shutdown_connections 2024-02-26T10:39:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:34.994+0000 7f8aa9800640 1 -- 172.21.15.67:0/3684257709 wait complete. 2024-02-26T10:39:35.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-02-26T10:39:35.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.228+0000 7f78d8e00640 1 Processor -- start 2024-02-26T10:39:35.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.228+0000 7f78d8e00640 1 -- start start 2024-02-26T10:39:35.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.229+0000 7f78d8e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78d4156230 0x7f78d40ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.229+0000 7f78d8e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d4152e50 con 0x7f78d4150300 2024-02-26T10:39:35.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.229+0000 7f78d8e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d4153130 con 0x7f78d406e410 2024-02-26T10:39:35.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.229+0000 7f78d8e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d41506e0 con 0x7f78d4156230 2024-02-26T10:39:35.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.230+0000 7f78d1e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f78d4150300 0x7f78d40c2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40144/0 (socket says 172.21.15.67:40144) 2024-02-26T10:39:35.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.230+0000 7f78d1e00640 1 -- 172.21.15.67:0/4254066832 learned_addr learned my addr 172.21.15.67:0/4254066832 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.230+0000 7f78d2800640 1 --2- 172.21.15.67:0/4254066832 >> v2:172.21.15.82:3300/0 conn(0x7f78d4156230 0x7f78d40ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.230+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564972492 0 0) 0x7f78d4152e50 con 0x7f78d4150300 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.230+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b8003520 con 0x7f78d4150300 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2016149398 0 0) 0x7f78b8003520 con 0x7f78d4150300 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 >> v2:172.21.15.82:3300/0 conn(0x7f78d4156230 msgr2=0x7f78d40ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 --2- 172.21.15.67:0/4254066832 >> v2:172.21.15.82:3300/0 conn(0x7f78d4156230 0x7f78d40ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 >> v1:172.21.15.82:6789/0 conn(0x7f78d406e410 legacy=0x7f78d40aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d4153410 con 0x7f78d4150300 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f78bc001670 con 0x7f78d4150300 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 >> v1:172.21.15.67:6789/0 conn(0x7f78d4150300 legacy=0x7f78d40c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8003a00 0x7f78b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b8004800 0x7f78b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b80052e0 0x7f78b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d41506e0 con 0x7f78b80052e0 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b800c1e0 con 0x7f78b8003a00 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b800c3e0 con 0x7f78b8004800 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.231+0000 7f78d2800640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b80052e0 0x7f78b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d1400640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b8004800 0x7f78b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d1e00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8003a00 0x7f78b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d2800640 1 -- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b8004800 msgr2=0x7f78b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d2800640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b8004800 0x7f78b8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d2800640 1 -- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8003a00 msgr2=0x7f78b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d2800640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8003a00 0x7f78b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d2800640 1 -- 172.21.15.67:0/4254066832 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8003520 con 0x7f78b80052e0 2024-02-26T10:39:35.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.232+0000 7f78d1e00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8003a00 0x7f78b8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.233+0000 7f78d2800640 1 -- 172.21.15.67:0/4254066832 reap_dead start 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.233+0000 7f78d2800640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b80052e0 0x7f78b800b9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009fc0 tx=0x7f78b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f79dc79d50e87fb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.233+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc009c10 con 0x7f78b80052e0 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.233+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc02cc60 con 0x7f78b80052e0 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.233+0000 7f78d0a00640 1 -- 172.21.15.67:0/4254066832 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc003bd0 con 0x7f78b80052e0 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 -- 172.21.15.67:0/4254066832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b80052e0 msgr2=0x7f78b800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b80052e0 0x7f78b800b9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009fc0 tx=0x7f78b8005830 comp rx=0 tx=0).stop 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 -- 172.21.15.67:0/4254066832 shutdown_connections 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 --2- 172.21.15.67:0/4254066832 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b80052e0 0x7f78b800b9c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 -- 172.21.15.67:0/4254066832 >> 172.21.15.67:0/4254066832 conn(0x7f78d40a84a0 msgr2=0x7f78d40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 -- 172.21.15.67:0/4254066832 shutdown_connections 2024-02-26T10:39:35.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.234+0000 7f78d8e00640 1 -- 172.21.15.67:0/4254066832 wait complete. 2024-02-26T10:39:35.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.236+0000 7f78d8e00640 1 Processor -- start 2024-02-26T10:39:35.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.236+0000 7f78d8e00640 1 -- start start 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b80052e0 0x7f78d414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 0x7f78d40bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d4151e20 0x7f78d40c0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d8e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d406dd80 con 0x7f78d41489e0 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d8e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d4153a60 con 0x7f78d4151e20 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40c2ff0 con 0x7f78b80052e0 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 0x7f78d40bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 0x7f78d40bfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49742/0 (socket says 172.21.15.67:49742) 2024-02-26T10:39:35.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 -- 172.21.15.67:0/1977630030 learned_addr learned my addr 172.21.15.67:0/1977630030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 -- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b80052e0 msgr2=0x7f78d414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b80052e0 0x7f78d414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d2800640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d4151e20 0x7f78d40c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b80052e0 0x7f78d414e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 -- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d4151e20 msgr2=0x7f78d40c0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d4151e20 0x7f78d40c0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.237+0000 7f78d1400640 1 -- 172.21.15.67:0/1977630030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d41e8b60 con 0x7f78d41489e0 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.238+0000 7f78d1400640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 0x7f78d40bfd10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f78c400d480 tx=0x7f78c400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.238+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4005480 con 0x7f78d41489e0 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.238+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4017070 con 0x7f78d41489e0 2024-02-26T10:39:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.238+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d41e8e40 con 0x7f78d41489e0 2024-02-26T10:39:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.238+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4003830 con 0x7f78d41489e0 2024-02-26T10:39:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.238+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d41e9400 con 0x7f78d41489e0 2024-02-26T10:39:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.239+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d40c63b0 con 0x7f78d41489e0 2024-02-26T10:39:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.240+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78c40039d0 con 0x7f78d41489e0 2024-02-26T10:39:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.241+0000 7f78ca000640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f78b006b500 0x7f78b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.241+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f78c4089c80 con 0x7f78d41489e0 2024-02-26T10:39:35.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.242+0000 7f78d1e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f78b006b500 0x7f78b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.243+0000 7f78d1e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f78b006b500 0x7f78b006d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f78bc001200 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:35.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.246+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78d40c63b0 con 0x7f78d41489e0 2024-02-26T10:39:35.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.516+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f78d4074a30 con 0x7f78d41489e0 2024-02-26T10:39:35.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.520+0000 7f78ca000640 1 -- 172.21.15.67:0/1977630030 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f78c4058f00 con 0x7f78d41489e0 2024-02-26T10:39:35.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f78b006b500 msgr2=0x7f78b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f78b006b500 0x7f78b006d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f78bc001200 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2024-02-26T10:39:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 msgr2=0x7f78d40bfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 0x7f78d40bfd10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f78c400d480 tx=0x7f78c400d940 comp rx=0 tx=0).stop 2024-02-26T10:39:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 shutdown_connections 2024-02-26T10:39:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f78b006b500 0x7f78b006d9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d4151e20 0x7f78d40c0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78d41489e0 0x7f78d40bfd10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 --2- 172.21.15.67:0/1977630030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78b80052e0 0x7f78d414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 >> 172.21.15.67:0/1977630030 conn(0x7f78d40a84a0 msgr2=0x7f78d406b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 shutdown_connections 2024-02-26T10:39:35.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.523+0000 7f78d8e00640 1 -- 172.21.15.67:0/1977630030 wait complete. 2024-02-26T10:39:35.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.749+0000 7f8eefa00640 1 Processor -- start 2024-02-26T10:39:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.749+0000 7f8eefa00640 1 -- start start 2024-02-26T10:39:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8eefa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 0x7f8ee8155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8eefa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ee81556a0 0x7f8ee8071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8eefa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee8155c00 con 0x7f8ee814f3d0 2024-02-26T10:39:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8eefa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee806dd80 con 0x7f8ee81556a0 2024-02-26T10:39:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8eefa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8155ee0 con 0x7f8ee80c2870 2024-02-26T10:39:35.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8ee7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 0x7f8ee8155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8ee7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 0x7f8ee8155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:47440/0 (socket says 172.21.15.67:47440) 2024-02-26T10:39:35.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8ee7e00640 1 -- 172.21.15.67:0/1397308921 learned_addr learned my addr 172.21.15.67:0/1397308921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:35.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.751+0000 7f8eed000640 1 --1- 172.21.15.67:0/1397308921 >> v1:172.21.15.82:6789/0 conn(0x7f8ee814f3d0 0x7f8ee806e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:51730/0 (socket says 172.21.15.67:51730) 2024-02-26T10:39:35.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.752+0000 7f8eeda00640 1 --2- 172.21.15.67:0/1397308921 >> v2:172.21.15.82:3300/0 conn(0x7f8ee81556a0 0x7f8ee8071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.752+0000 7f8ee7e00640 1 -- 172.21.15.67:0/1397308921 >> v2:172.21.15.82:3300/0 conn(0x7f8ee81556a0 msgr2=0x7f8ee8071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:35.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.752+0000 7f8ee7e00640 1 --2- 172.21.15.67:0/1397308921 >> v2:172.21.15.82:3300/0 conn(0x7f8ee81556a0 0x7f8ee8071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.752+0000 7f8ee7400640 1 -- 172.21.15.67:0/1397308921 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704638544 0 0) 0x7f8ee8155c00 con 0x7f8ee814f3d0 2024-02-26T10:39:35.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.752+0000 7f8ee7e00640 1 -- 172.21.15.67:0/1397308921 >> v1:172.21.15.82:6789/0 conn(0x7f8ee814f3d0 legacy=0x7f8ee806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.753+0000 7f8ee7e00640 1 -- 172.21.15.67:0/1397308921 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee806dfd0 con 0x7f8ee80c2870 2024-02-26T10:39:35.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.754+0000 7f8ee7e00640 1 --2- 172.21.15.67:0/1397308921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 0x7f8ee8155160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009870 tx=0x7f8ed002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ada77b20e7b909 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:35.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.755+0000 7f8ee7400640 1 -- 172.21.15.67:0/1397308921 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0035020 con 0x7f8ee80c2870 2024-02-26T10:39:35.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.755+0000 7f8ee7400640 1 -- 172.21.15.67:0/1397308921 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed0031640 con 0x7f8ee80c2870 2024-02-26T10:39:35.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.755+0000 7f8ee7400640 1 -- 172.21.15.67:0/1397308921 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002d040 con 0x7f8ee80c2870 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 -- 172.21.15.67:0/1397308921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 msgr2=0x7f8ee8155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 --2- 172.21.15.67:0/1397308921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 0x7f8ee8155160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009870 tx=0x7f8ed002b8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 -- 172.21.15.67:0/1397308921 shutdown_connections 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 --2- 172.21.15.67:0/1397308921 >> v2:172.21.15.82:3300/0 conn(0x7f8ee81556a0 0x7f8ee8071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 --2- 172.21.15.67:0/1397308921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee80c2870 0x7f8ee8155160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 -- 172.21.15.67:0/1397308921 >> 172.21.15.67:0/1397308921 conn(0x7f8ee80a84a0 msgr2=0x7f8ee80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:35.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 -- 172.21.15.67:0/1397308921 shutdown_connections 2024-02-26T10:39:35.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.756+0000 7f8eefa00640 1 -- 172.21.15.67:0/1397308921 wait complete. 2024-02-26T10:39:35.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.758+0000 7f8eefa00640 1 Processor -- start 2024-02-26T10:39:35.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.758+0000 7f8eefa00640 1 -- start start 2024-02-26T10:39:35.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.758+0000 7f8eefa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 0x7f8ee8143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eefa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee814f3d0 0x7f8ee813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eefa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee81556a0 0x7f8ee813f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eefa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c2cb0 con 0x7f8ee80c2870 2024-02-26T10:39:35.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eefa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c1810 con 0x7f8ee81556a0 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eefa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c2e30 con 0x7f8ee814f3d0 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 0x7f8ee8143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 0x7f8ee8143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49744/0 (socket says 172.21.15.67:49744) 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 -- 172.21.15.67:0/4084827454 learned_addr learned my addr 172.21.15.67:0/4084827454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8ee7e00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee814f3d0 0x7f8ee813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eeda00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee81556a0 0x7f8ee813f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 -- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee814f3d0 msgr2=0x7f8ee813ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee814f3d0 0x7f8ee813ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 -- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee81556a0 msgr2=0x7f8ee813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee81556a0 0x7f8ee813f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.759+0000 7f8eed000640 1 -- 172.21.15.67:0/4084827454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee813f7b0 con 0x7f8ee80c2870 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.760+0000 7f8eed000640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 0x7f8ee8143150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c7f0 tx=0x7f8ed800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.760+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8018070 con 0x7f8ee80c2870 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.760+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed800a640 con 0x7f8ee80c2870 2024-02-26T10:39:35.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.760+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800aa30 con 0x7f8ee80c2870 2024-02-26T10:39:35.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.760+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee80ab360 con 0x7f8ee80c2870 2024-02-26T10:39:35.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.760+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee80a9c10 con 0x7f8ee80c2870 2024-02-26T10:39:35.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.761+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee80ab880 con 0x7f8ee80c2870 2024-02-26T10:39:35.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.762+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ed801b070 con 0x7f8ee80c2870 2024-02-26T10:39:35.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.763+0000 7f8ee4c00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b4d0 0x7f8ec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:35.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.763+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8ed80894b0 con 0x7f8ee80c2870 2024-02-26T10:39:35.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.764+0000 7f8ee7e00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b4d0 0x7f8ec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:35.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.765+0000 7f8ee7e00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b4d0 0x7f8ec406d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0036000 tx=0x7f8ed003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:35.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.767+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ed8058670 con 0x7f8ee80c2870 2024-02-26T10:39:35.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.965+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ee8074a30 con 0x7f8ee80c2870 2024-02-26T10:39:35.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.967+0000 7f8ee4c00640 1 -- 172.21.15.67:0/4084827454 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12819 (secure 0 0 0) 0x7f8ed8057f70 con 0x7f8ee80c2870 2024-02-26T10:39:35.968 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:35.969 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:31.820810+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:35.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b4d0 msgr2=0x7f8ec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b4d0 0x7f8ec406d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0036000 tx=0x7f8ed003c040 comp rx=0 tx=0).stop 2024-02-26T10:39:35.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 msgr2=0x7f8ee8143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:35.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 0x7f8ee8143150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c7f0 tx=0x7f8ed800ccb0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 shutdown_connections 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ec406b4d0 0x7f8ec406d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee81556a0 0x7f8ee813f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ee814f3d0 0x7f8ee813ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 --2- 172.21.15.67:0/4084827454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee80c2870 0x7f8ee8143150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 >> 172.21.15.67:0/4084827454 conn(0x7f8ee80a84a0 msgr2=0x7f8ee80a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 shutdown_connections 2024-02-26T10:39:35.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:35.970+0000 7f8eefa00640 1 -- 172.21.15.67:0/4084827454 wait complete. 2024-02-26T10:39:35.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:35.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:36.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.213+0000 7fca4d000640 1 Processor -- start 2024-02-26T10:39:36.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.213+0000 7fca4d000640 1 -- start start 2024-02-26T10:39:36.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca4d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca480a9430 0x7fca480ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca4d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca4806e2d0 0x7fca480c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca4d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca48150780 con 0x7fca481503a0 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca4d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca480a98c0 con 0x7fca480a9430 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca4d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480a9ba0 con 0x7fca4806e2d0 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca3ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca480a9430 0x7fca480ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca3ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca480a9430 0x7fca480ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49754/0 (socket says 172.21.15.67:49754) 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca3ea00640 1 -- 172.21.15.67:0/1840004175 learned_addr learned my addr 172.21.15.67:0/1840004175 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.215+0000 7fca47e00640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca4806e2d0 0x7fca480c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.216+0000 7fca3ea00640 1 -- 172.21.15.67:0/1840004175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca4806e2d0 msgr2=0x7fca480c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.216+0000 7fca3ea00640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca4806e2d0 0x7fca480c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.216+0000 7fca3ea00640 1 -- 172.21.15.67:0/1840004175 >> v1:172.21.15.67:6789/0 conn(0x7fca481503a0 legacy=0x7fca481535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.216+0000 7fca3ea00640 1 -- 172.21.15.67:0/1840004175 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480a9e80 con 0x7fca480a9430 2024-02-26T10:39:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.216+0000 7fca47e00640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca4806e2d0 0x7fca480c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.217+0000 7fca3ea00640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca480a9430 0x7fca480ab480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca30009e50 tx=0x7fca3002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e0d923a41c95d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.217+0000 7fca46a00640 1 -- 172.21.15.67:0/1840004175 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3002cb00 con 0x7fca480a9430 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.217+0000 7fca46a00640 1 -- 172.21.15.67:0/1840004175 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3002cca0 con 0x7fca480a9430 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.217+0000 7fca46a00640 1 -- 172.21.15.67:0/1840004175 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30034610 con 0x7fca480a9430 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 -- 172.21.15.67:0/1840004175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca480a9430 msgr2=0x7fca480ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca480a9430 0x7fca480ab480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca30009e50 tx=0x7fca3002bf60 comp rx=0 tx=0).stop 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 -- 172.21.15.67:0/1840004175 shutdown_connections 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca4806e2d0 0x7fca480c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 --2- 172.21.15.67:0/1840004175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca480a9430 0x7fca480ab480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 -- 172.21.15.67:0/1840004175 >> 172.21.15.67:0/1840004175 conn(0x7fca480c6380 msgr2=0x7fca480c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 -- 172.21.15.67:0/1840004175 shutdown_connections 2024-02-26T10:39:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.218+0000 7fca4d000640 1 -- 172.21.15.67:0/1840004175 wait complete. 2024-02-26T10:39:36.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.220+0000 7fca4d000640 1 Processor -- start 2024-02-26T10:39:36.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.220+0000 7fca4d000640 1 -- start start 2024-02-26T10:39:36.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca4d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 0x7fca48149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca4d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca481503a0 0x7fca481ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca4d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca481ec180 0x7fca481ec630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca4d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c0f80 con 0x7fca4806e2d0 2024-02-26T10:39:36.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca4d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4806ddb0 con 0x7fca481ec180 2024-02-26T10:39:36.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca4d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4806e720 con 0x7fca481503a0 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 0x7fca48149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 0x7fca48149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49758/0 (socket says 172.21.15.67:49758) 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 -- 172.21.15.67:0/2056106866 learned_addr learned my addr 172.21.15.67:0/2056106866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 -- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca481503a0 msgr2=0x7fca481ede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca481503a0 0x7fca481ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 -- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca481ec180 msgr2=0x7fca481ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca481ec180 0x7fca481ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.221+0000 7fca47400640 1 -- 172.21.15.67:0/2056106866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca481ee670 con 0x7fca4806e2d0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.222+0000 7fca47400640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 0x7fca48149d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c3c0 tx=0x7fca3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.222+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380052d0 con 0x7fca4806e2d0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.222+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38005490 con 0x7fca4806e2d0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.222+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38007260 con 0x7fca4806e2d0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.222+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca481ee950 con 0x7fca4806e2d0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.222+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca481ee370 con 0x7fca4806e2d0 2024-02-26T10:39:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.223+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca480678e0 con 0x7fca4806e2d0 2024-02-26T10:39:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.224+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca38007400 con 0x7fca4806e2d0 2024-02-26T10:39:36.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.225+0000 7fca3fe00640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.225+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fca38089310 con 0x7fca4806e2d0 2024-02-26T10:39:36.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.225+0000 7fca3ea00640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.227+0000 7fca3ea00640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca2406b500 0x7fca2406d9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca30035000 tx=0x7fca3002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:36.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.229+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca380584d0 con 0x7fca4806e2d0 2024-02-26T10:39:36.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.423+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca48074a30 con 0x7fca4806e2d0 2024-02-26T10:39:36.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.424+0000 7fca3fe00640 1 -- 172.21.15.67:0/2056106866 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca38057dd0 con 0x7fca4806e2d0 2024-02-26T10:39:36.425 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca2406b500 msgr2=0x7fca2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca2406b500 0x7fca2406d9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca30035000 tx=0x7fca3002e000 comp rx=0 tx=0).stop 2024-02-26T10:39:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 msgr2=0x7fca48149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 0x7fca48149d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c3c0 tx=0x7fca3800c880 comp rx=0 tx=0).stop 2024-02-26T10:39:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 shutdown_connections 2024-02-26T10:39:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca481ec180 0x7fca481ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca481503a0 0x7fca481ede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 --2- 172.21.15.67:0/2056106866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca4806e2d0 0x7fca48149d60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 >> 172.21.15.67:0/2056106866 conn(0x7fca480c6380 msgr2=0x7fca480c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 shutdown_connections 2024-02-26T10:39:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.426+0000 7fca4d000640 1 -- 172.21.15.67:0/2056106866 wait complete. 2024-02-26T10:39:36.445 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:39:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.656+0000 7f22de600640 1 Processor -- start 2024-02-26T10:39:36.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.656+0000 7f22de600640 1 -- start start 2024-02-26T10:39:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.657+0000 7f22de600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22d80a94a0 0x7f22d806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.657+0000 7f22de600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d806df70 con 0x7f22d806ac20 2024-02-26T10:39:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22de600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80a9a80 con 0x7f22d806db00 2024-02-26T10:39:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22de600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80a9d60 con 0x7f22d80a94a0 2024-02-26T10:39:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22dd200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f22d806ac20 0x7f22d8153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40160/0 (socket says 172.21.15.67:40160) 2024-02-26T10:39:36.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22dd200640 1 -- 172.21.15.67:0/551952394 learned_addr learned my addr 172.21.15.67:0/551952394 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22d7e00640 1 --2- 172.21.15.67:0/551952394 >> v2:172.21.15.82:3300/0 conn(0x7f22d80a94a0 0x7f22d806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638693819 0 0) 0x7f22d806df70 con 0x7f22d806ac20 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.658+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c0003520 con 0x7f22d806ac20 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177634824 0 0) 0x7f22c0003520 con 0x7f22d806ac20 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 >> v2:172.21.15.82:3300/0 conn(0x7f22d80a94a0 msgr2=0x7f22d806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 --2- 172.21.15.67:0/551952394 >> v2:172.21.15.82:3300/0 conn(0x7f22d80a94a0 0x7f22d806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 >> v1:172.21.15.82:6789/0 conn(0x7f22d806db00 legacy=0x7f22d80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80ab750 con 0x7f22d806ac20 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f22c8001670 con 0x7f22d806ac20 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 >> v1:172.21.15.67:6789/0 conn(0x7f22d806ac20 legacy=0x7f22d8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c0003a00 0x7f22c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22c0004800 0x7f22c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c00051c0 0x7f22c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80a9d60 con 0x7f22c00051c0 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c000c0c0 con 0x7f22c0003a00 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c000c2c0 con 0x7f22c0004800 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.659+0000 7f22ddc00640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c00051c0 0x7f22c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22dd200640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c0003a00 0x7f22c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22d7e00640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22c0004800 0x7f22c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22ddc00640 1 -- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22c0004800 msgr2=0x7f22c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22ddc00640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22c0004800 0x7f22c0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22ddc00640 1 -- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c0003a00 msgr2=0x7f22c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:36.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22ddc00640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c0003a00 0x7f22c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22ddc00640 1 -- 172.21.15.67:0/551952394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0003520 con 0x7f22c00051c0 2024-02-26T10:39:36.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.660+0000 7f22dd200640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c0003a00 0x7f22c0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22ddc00640 1 -- 172.21.15.67:0/551952394 reap_dead start 2024-02-26T10:39:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22ddc00640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c00051c0 0x7f22c000b9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f22d0001e50 tx=0x7f22c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2756bd5541f5e538 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0001a70 con 0x7f22c00051c0 2024-02-26T10:39:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0003bb0 con 0x7f22c00051c0 2024-02-26T10:39:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22d7400640 1 -- 172.21.15.67:0/551952394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00050d0 con 0x7f22c00051c0 2024-02-26T10:39:36.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22de600640 1 -- 172.21.15.67:0/551952394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c00051c0 msgr2=0x7f22c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.661+0000 7f22de600640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c00051c0 0x7f22c000b9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f22d0001e50 tx=0x7f22c0005830 comp rx=0 tx=0).stop 2024-02-26T10:39:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.662+0000 7f22de600640 1 -- 172.21.15.67:0/551952394 shutdown_connections 2024-02-26T10:39:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.662+0000 7f22de600640 1 --2- 172.21.15.67:0/551952394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c00051c0 0x7f22c000b9c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.662+0000 7f22de600640 1 -- 172.21.15.67:0/551952394 >> 172.21.15.67:0/551952394 conn(0x7f22d80c63f0 msgr2=0x7f22d80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.662+0000 7f22de600640 1 -- 172.21.15.67:0/551952394 shutdown_connections 2024-02-26T10:39:36.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.662+0000 7f22de600640 1 -- 172.21.15.67:0/551952394 wait complete. 2024-02-26T10:39:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.663+0000 7f22de600640 1 Processor -- start 2024-02-26T10:39:36.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 -- start start 2024-02-26T10:39:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c00051c0 0x7f22d81477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 0x7f22d814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d8148200 0x7f22d814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8150590 con 0x7f22d8147d30 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8150410 con 0x7f22c00051c0 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.664+0000 7f22de600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d806b000 con 0x7f22d8148200 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 0x7f22d814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 0x7f22d814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49772/0 (socket says 172.21.15.67:49772) 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 -- 172.21.15.67:0/1560497593 learned_addr learned my addr 172.21.15.67:0/1560497593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22ddc00640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d8148200 0x7f22d814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 -- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d8148200 msgr2=0x7f22d814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d8148200 0x7f22d814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 -- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c00051c0 msgr2=0x7f22d81477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c00051c0 0x7f22d81477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 -- 172.21.15.67:0/1560497593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d81ef160 con 0x7f22d8147d30 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.665+0000 7f22d7e00640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 0x7f22d814e640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22cc010890 tx=0x7f22cc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:36.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.666+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00f040 con 0x7f22d8147d30 2024-02-26T10:39:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.666+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc00eb00 con 0x7f22d8147d30 2024-02-26T10:39:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.666+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc01c400 con 0x7f22d8147d30 2024-02-26T10:39:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.666+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80c5150 con 0x7f22d8147d30 2024-02-26T10:39:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.666+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d80c5710 con 0x7f22d8147d30 2024-02-26T10:39:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.667+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d806bea0 con 0x7f22d8147d30 2024-02-26T10:39:36.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.668+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22cc01c5a0 con 0x7f22d8147d30 2024-02-26T10:39:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.669+0000 7f22d4c00640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f22bc06b500 0x7f22bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.669+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f22cc08a4b0 con 0x7f22d8147d30 2024-02-26T10:39:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.669+0000 7f22dd200640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f22bc06b500 0x7f22bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:36.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.670+0000 7f22dd200640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f22bc06b500 0x7f22bc06d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:36.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.673+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22d806bea0 con 0x7f22d8147d30 2024-02-26T10:39:36.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.866+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f22d8075020 con 0x7f22d8147d30 2024-02-26T10:39:36.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.867+0000 7f22d4c00640 1 -- 172.21.15.67:0/1560497593 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f22d8075020 con 0x7f22d8147d30 2024-02-26T10:39:36.868 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:36.869 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:39:36.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:36.869 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:39:36.869 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:39:36.869 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:36.869 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.869+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f22bc06b500 msgr2=0x7f22bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.869+0000 7f22de600640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f22bc06b500 0x7f22bc06d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c8009040 comp rx=0 tx=0).stop 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.869+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 msgr2=0x7f22d814e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.869+0000 7f22de600640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 0x7f22d814e640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22cc010890 tx=0x7f22cc010d50 comp rx=0 tx=0).stop 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 shutdown_connections 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d8148200 0x7f22d814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22d8147d30 0x7f22d814e640 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f22bc06b500 0x7f22bc06d9c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 --2- 172.21.15.67:0/1560497593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c00051c0 0x7f22d81477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 >> 172.21.15.67:0/1560497593 conn(0x7f22d80c63f0 msgr2=0x7f22d80ac050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:36.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 shutdown_connections 2024-02-26T10:39:36.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:36.870+0000 7f22de600640 1 -- 172.21.15.67:0/1560497593 wait complete. 2024-02-26T10:39:36.889 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T10:39:36.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T10:39:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.100+0000 7ffbae000640 1 Processor -- start 2024-02-26T10:39:37.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.101+0000 7ffbae000640 1 -- start start 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffbae000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffba80c1ae0 0x7ffba80a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffbae000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80c2e20 con 0x7ffba814e9e0 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffbae000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80c1f00 con 0x7ffba80a7c10 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffbae000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba80c2b40 con 0x7ffba80c1ae0 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffba7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffba814e9e0 0x7ffba8151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40174/0 (socket says 172.21.15.67:40174) 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffba7400640 1 -- 172.21.15.67:0/3024638792 learned_addr learned my addr 172.21.15.67:0/3024638792 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.102+0000 7ffba6a00640 1 --2- 172.21.15.67:0/3024638792 >> v2:172.21.15.67:3300/0 conn(0x7ffba80c1ae0 0x7ffba80a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.103+0000 7ffba6000640 1 -- 172.21.15.67:0/3024638792 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932614840 0 0) 0x7ffba80c2e20 con 0x7ffba814e9e0 2024-02-26T10:39:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.103+0000 7ffba6000640 1 -- 172.21.15.67:0/3024638792 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb8c003440 con 0x7ffba814e9e0 2024-02-26T10:39:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.103+0000 7ffba6a00640 1 -- 172.21.15.67:0/3024638792 >> v1:172.21.15.82:6789/0 conn(0x7ffba80a7c10 legacy=0x7ffba806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.103+0000 7ffba6a00640 1 -- 172.21.15.67:0/3024638792 >> v1:172.21.15.67:6789/0 conn(0x7ffba814e9e0 legacy=0x7ffba8151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.103+0000 7ffba6a00640 1 -- 172.21.15.67:0/3024638792 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80c1000 con 0x7ffba80c1ae0 2024-02-26T10:39:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.104+0000 7ffba6a00640 1 --2- 172.21.15.67:0/3024638792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80c1ae0 0x7ffba80a76d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009d90 tx=0x7ffb9802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4739aad7194d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.104+0000 7ffba6000640 1 -- 172.21.15.67:0/3024638792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98039070 con 0x7ffba80c1ae0 2024-02-26T10:39:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.104+0000 7ffba6000640 1 -- 172.21.15.67:0/3024638792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9802cd00 con 0x7ffba80c1ae0 2024-02-26T10:39:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.104+0000 7ffba6000640 1 -- 172.21.15.67:0/3024638792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98034660 con 0x7ffba80c1ae0 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 -- 172.21.15.67:0/3024638792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80c1ae0 msgr2=0x7ffba80a76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 --2- 172.21.15.67:0/3024638792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80c1ae0 0x7ffba80a76d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009d90 tx=0x7ffb9802bf50 comp rx=0 tx=0).stop 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 -- 172.21.15.67:0/3024638792 shutdown_connections 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 --2- 172.21.15.67:0/3024638792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80c1ae0 0x7ffba80a76d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 -- 172.21.15.67:0/3024638792 >> 172.21.15.67:0/3024638792 conn(0x7ffba80c6da0 msgr2=0x7ffba80c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 -- 172.21.15.67:0/3024638792 shutdown_connections 2024-02-26T10:39:37.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.105+0000 7ffbae000640 1 -- 172.21.15.67:0/3024638792 wait complete. 2024-02-26T10:39:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.107+0000 7ffbae000640 1 Processor -- start 2024-02-26T10:39:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.107+0000 7ffbae000640 1 -- start start 2024-02-26T10:39:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffbae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 0x7ffba81ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffbae000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffba80c1ae0 0x7ffba80abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffbae000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba814e9e0 0x7ffba81468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffbae000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c27c0 con 0x7ffba80a7c10 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffbae000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c15c0 con 0x7ffba814e9e0 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffbae000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80c0920 con 0x7ffba80c1ae0 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 0x7ffba81ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 0x7ffba81ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49784/0 (socket says 172.21.15.67:49784) 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 -- 172.21.15.67:0/2999396569 learned_addr learned my addr 172.21.15.67:0/2999396569 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba6a00640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffba80c1ae0 0x7ffba80abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 -- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffba80c1ae0 msgr2=0x7ffba80abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffba80c1ae0 0x7ffba80abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 -- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba814e9e0 msgr2=0x7ffba81468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba814e9e0 0x7ffba81468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.108+0000 7ffba7400640 1 -- 172.21.15.67:0/2999396569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba81ee7c0 con 0x7ffba80a7c10 2024-02-26T10:39:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.109+0000 7ffba7400640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 0x7ffba81ee1e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c410 tx=0x7ffb9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.109+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90017070 con 0x7ffba80a7c10 2024-02-26T10:39:37.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.109+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb90005ce0 con 0x7ffba80a7c10 2024-02-26T10:39:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.109+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba81eeaa0 con 0x7ffba80a7c10 2024-02-26T10:39:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.109+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900072f0 con 0x7ffba80a7c10 2024-02-26T10:39:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.109+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80711d0 con 0x7ffba80a7c10 2024-02-26T10:39:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.110+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba8074a30 con 0x7ffba80a7c10 2024-02-26T10:39:37.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.111+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb9001a070 con 0x7ffba80a7c10 2024-02-26T10:39:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.113+0000 7ffb9f400640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb8406b500 0x7ffb8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.113+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffb90089370 con 0x7ffba80a7c10 2024-02-26T10:39:37.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.114+0000 7ffba6a00640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb8406b500 0x7ffb8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:37.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.115+0000 7ffba6a00640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb8406b500 0x7ffb8406d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb98036000 tx=0x7ffb98030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:37.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.118+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb90058560 con 0x7ffba80a7c10 2024-02-26T10:39:37.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.316+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffba80c25a0 con 0x7ffba80a7c10 2024-02-26T10:39:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.955+0000 7ffb9f400640 1 -- 172.21.15.67:0/2999396569 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffb90057e60 con 0x7ffba80a7c10 2024-02-26T10:39:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-02-26T10:39:37.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.959+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb8406b500 msgr2=0x7ffb8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.959+0000 7ffbae000640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb8406b500 0x7ffb8406d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb98036000 tx=0x7ffb98030000 comp rx=0 tx=0).stop 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.959+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 msgr2=0x7ffba81ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.959+0000 7ffbae000640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 0x7ffba81ee1e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c410 tx=0x7ffb9000c8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 shutdown_connections 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb8406b500 0x7ffb8406d9c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba814e9e0 0x7ffba81468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffba80c1ae0 0x7ffba80abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 --2- 172.21.15.67:0/2999396569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba80a7c10 0x7ffba81ee1e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 >> 172.21.15.67:0/2999396569 conn(0x7ffba80c6da0 msgr2=0x7ffba80a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:37.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 shutdown_connections 2024-02-26T10:39:37.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:37.960+0000 7ffbae000640 1 -- 172.21.15.67:0/2999396569 wait complete. 2024-02-26T10:39:37.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:39:38.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.203+0000 7f8657400640 1 Processor -- start 2024-02-26T10:39:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.204+0000 7f8657400640 1 -- start start 2024-02-26T10:39:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.205+0000 7f8657400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f865006ae00 0x7f86500c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.205+0000 7f8657400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 0x7f8650152380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.205+0000 7f8657400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86500c0f60 con 0x7f86500c1740 2024-02-26T10:39:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.205+0000 7f8657400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86500c1590 con 0x7f865006ae00 2024-02-26T10:39:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.205+0000 7f8657400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86500c5f80 con 0x7f865006b1e0 2024-02-26T10:39:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.206+0000 7f8655400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f86500c1740 0x7f8650155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40184/0 (socket says 172.21.15.67:40184) 2024-02-26T10:39:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.206+0000 7f8655400640 1 -- 172.21.15.67:0/1276340833 learned_addr learned my addr 172.21.15.67:0/1276340833 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.206+0000 7f8654a00640 1 --2- 172.21.15.67:0/1276340833 >> v2:172.21.15.82:3300/0 conn(0x7f865006ae00 0x7f86500c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.206+0000 7f864fe00640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 0x7f8650152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.206+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185690151 0 0) 0x7f86500c0f60 con 0x7f86500c1740 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.206+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8638003520 con 0x7f86500c1740 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1997659832 0 0) 0x7f8638003520 con 0x7f86500c1740 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 msgr2=0x7f8650152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 0x7f8650152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 >> v2:172.21.15.82:3300/0 conn(0x7f865006ae00 msgr2=0x7f86500c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 --2- 172.21.15.67:0/1276340833 >> v2:172.21.15.82:3300/0 conn(0x7f865006ae00 0x7f86500c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500c6260 con 0x7f86500c1740 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f86480019f0 con 0x7f86500c1740 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 >> v1:172.21.15.67:6789/0 conn(0x7f86500c1740 legacy=0x7f8650155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 0x7f86380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f8638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8638004d40 0x7f86380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86500c5f80 con 0x7f8638003f30 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863800b9c0 con 0x7f8638004d40 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863800bb40 con 0x7f865006b1e0 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f864fe00640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f8638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f8654a00640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 0x7f86380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.207+0000 7f8655400640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8638004d40 0x7f86380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.208+0000 7f864fe00640 1 -- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 msgr2=0x7f86380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.208+0000 7f864fe00640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f865006b1e0 0x7f86380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.208+0000 7f864fe00640 1 -- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8638004d40 msgr2=0x7f86380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.208+0000 7f864fe00640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8638004d40 0x7f86380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.208+0000 7f864fe00640 1 -- 172.21.15.67:0/1276340833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500c6260 con 0x7f8638003f30 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.209+0000 7f864fe00640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f8638004800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f863c009ac0 tx=0x7f8638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8368af320deb8cca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.209+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c039070 con 0x7f8638003f30 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.209+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c02c680 con 0x7f8638003f30 2024-02-26T10:39:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.209+0000 7f864f400640 1 -- 172.21.15.67:0/1276340833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c009690 con 0x7f8638003f30 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8657400640 1 -- 172.21.15.67:0/1276340833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 msgr2=0x7f8638004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8657400640 1 --2- 172.21.15.67:0/1276340833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f8638004800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f863c009ac0 tx=0x7f8638005830 comp rx=0 tx=0).stop 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8655400640 1 -- 172.21.15.67:0/1276340833 reap_dead start 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8657400640 1 -- 172.21.15.67:0/1276340833 shutdown_connections 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8657400640 1 -- 172.21.15.67:0/1276340833 >> 172.21.15.67:0/1276340833 conn(0x7f86500c7160 msgr2=0x7f86500a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8657400640 1 -- 172.21.15.67:0/1276340833 shutdown_connections 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.210+0000 7f8657400640 1 -- 172.21.15.67:0/1276340833 wait complete. 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.212+0000 7f8657400640 1 Processor -- start 2024-02-26T10:39:38.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.212+0000 7f8657400640 1 -- start start 2024-02-26T10:39:38.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8657400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f86501dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8657400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8650151f30 0x7f86501ede60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8657400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86500a9610 0x7f86500a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8657400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650155ca0 con 0x7f8638003f30 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8657400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650155b20 con 0x7f86500a9610 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8657400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8650155e20 con 0x7f8650151f30 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8654a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f86501dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8654a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f86501dd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49796/0 (socket says 172.21.15.67:49796) 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.213+0000 7f8654a00640 1 -- 172.21.15.67:0/3281832515 learned_addr learned my addr 172.21.15.67:0/3281832515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8655400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86500a9610 0x7f86500a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f864fe00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8650151f30 0x7f86501ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8654a00640 1 -- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8650151f30 msgr2=0x7f86501ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8654a00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8650151f30 0x7f86501ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8654a00640 1 -- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86500a9610 msgr2=0x7f86500a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8654a00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86500a9610 0x7f86500a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8654a00640 1 -- 172.21.15.67:0/3281832515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650071090 con 0x7f8638003f30 2024-02-26T10:39:38.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f8654a00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f86501dd9f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f86440116d0 tx=0x7f8644011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:38.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.214+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400f9c0 con 0x7f8638003f30 2024-02-26T10:39:38.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.215+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864400fb60 con 0x7f8638003f30 2024-02-26T10:39:38.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.215+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644015460 con 0x7f8638003f30 2024-02-26T10:39:38.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.215+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8650071370 con 0x7f8638003f30 2024-02-26T10:39:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.215+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86501dd4d0 con 0x7f8638003f30 2024-02-26T10:39:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.216+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8650075120 con 0x7f8638003f30 2024-02-26T10:39:38.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.217+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8644015600 con 0x7f8638003f30 2024-02-26T10:39:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.218+0000 7f864cc00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.218+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f864408ab10 con 0x7f8638003f30 2024-02-26T10:39:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.218+0000 7f864fe00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:38.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.220+0000 7f864fe00640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f863006b4d0 0x7f863006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f863c036000 tx=0x7f863c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:38.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.223+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8650075120 con 0x7f8638003f30 2024-02-26T10:39:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.422+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f86501ddf30 con 0x7f8638003f30 2024-02-26T10:39:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.971+0000 7f864cc00640 1 -- 172.21.15.67:0/3281832515 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8644019030 con 0x7f8638003f30 2024-02-26T10:39:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-02-26T10:39:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f863006b4d0 msgr2=0x7f863006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f863006b4d0 0x7f863006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f863c036000 tx=0x7f863c030000 comp rx=0 tx=0).stop 2024-02-26T10:39:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 msgr2=0x7f86501dd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f86501dd9f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f86440116d0 tx=0x7f8644011b90 comp rx=0 tx=0).stop 2024-02-26T10:39:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 shutdown_connections 2024-02-26T10:39:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86500a9610 0x7f86500a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8650151f30 0x7f86501ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 --2- 172.21.15.67:0/3281832515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8638003f30 0x7f86501dd9f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:38.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.975+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 >> 172.21.15.67:0/3281832515 conn(0x7f86500c7160 msgr2=0x7f865006db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:38.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.976+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 shutdown_connections 2024-02-26T10:39:38.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:38.976+0000 7f8657400640 1 -- 172.21.15.67:0/3281832515 wait complete. 2024-02-26T10:39:39.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T10:39:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.202+0000 7fe9bda00640 1 Processor -- start 2024-02-26T10:39:39.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.202+0000 7fe9bda00640 1 -- start start 2024-02-26T10:39:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.204+0000 7fe9bda00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9b80a9430 0x7fe9b80ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.204+0000 7fe9bda00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b8150780 con 0x7fe9b81503a0 2024-02-26T10:39:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.204+0000 7fe9bda00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b80a98c0 con 0x7fe9b806e2d0 2024-02-26T10:39:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.204+0000 7fe9bda00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b80a9ba0 con 0x7fe9b80a9430 2024-02-26T10:39:39.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.204+0000 7fe9b7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe9b81503a0 0x7fe9b81535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40198/0 (socket says 172.21.15.67:40198) 2024-02-26T10:39:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.204+0000 7fe9b7e00640 1 -- 172.21.15.67:0/3283558002 learned_addr learned my addr 172.21.15.67:0/3283558002 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b7400640 1 --2- 172.21.15.67:0/3283558002 >> v2:172.21.15.82:3300/0 conn(0x7fe9b80a9430 0x7fe9b80ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068866511 0 0) 0x7fe9b8150780 con 0x7fe9b81503a0 2024-02-26T10:39:39.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe99c003520 con 0x7fe9b81503a0 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445069244 0 0) 0x7fe99c003520 con 0x7fe9b81503a0 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 >> v2:172.21.15.82:3300/0 conn(0x7fe9b80a9430 msgr2=0x7fe9b80ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 --2- 172.21.15.67:0/3283558002 >> v2:172.21.15.82:3300/0 conn(0x7fe9b80a9430 0x7fe9b80ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 >> v1:172.21.15.82:6789/0 conn(0x7fe9b806e2d0 legacy=0x7fe9b80c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80a9e80 con 0x7fe9b81503a0 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fe9a8001670 con 0x7fe9b81503a0 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 >> v1:172.21.15.67:6789/0 conn(0x7fe9b81503a0 legacy=0x7fe9b81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe99c0039b0 0x7fe99c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe99c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe99c0051c0 0x7fe99c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80a9ba0 con 0x7fe99c004800 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c010470 con 0x7fe99c0051c0 2024-02-26T10:39:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.205+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c010670 con 0x7fe99c0039b0 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7400640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe99c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9bd000640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe99c0051c0 0x7fe99c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe99c0039b0 0x7fe99c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7400640 1 -- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe99c0039b0 msgr2=0x7fe99c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7400640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe99c0039b0 0x7fe99c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7400640 1 -- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe99c0051c0 msgr2=0x7fe99c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7400640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe99c0051c0 0x7fe99c00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9b7400640 1 -- 172.21.15.67:0/3283558002 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c003520 con 0x7fe99c004800 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9bd000640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe99c0051c0 0x7fe99c00fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.206+0000 7fe9bd000640 1 -- 172.21.15.67:0/3283558002 reap_dead start 2024-02-26T10:39:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.207+0000 7fe9b7400640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe99c004c80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ac0 tx=0x7fe99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8a721a826c50f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4039070 con 0x7fe99c004800 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a402c6b0 con 0x7fe99c004800 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9b6a00640 1 -- 172.21.15.67:0/3283558002 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4009690 con 0x7fe99c004800 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 -- 172.21.15.67:0/3283558002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 msgr2=0x7fe99c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe99c004c80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ac0 tx=0x7fe99c005830 comp rx=0 tx=0).stop 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 -- 172.21.15.67:0/3283558002 shutdown_connections 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 --2- 172.21.15.67:0/3283558002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe99c004c80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 -- 172.21.15.67:0/3283558002 >> 172.21.15.67:0/3283558002 conn(0x7fe9b80c6380 msgr2=0x7fe9b80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 -- 172.21.15.67:0/3283558002 shutdown_connections 2024-02-26T10:39:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.208+0000 7fe9bda00640 1 -- 172.21.15.67:0/3283558002 wait complete. 2024-02-26T10:39:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.210+0000 7fe9bda00640 1 Processor -- start 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.210+0000 7fe9bda00640 1 -- start start 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe9b80c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9bda00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b80c5010 0x7fe9b80c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9bda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9b8145b20 0x7fe9b8145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9bda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80c0f80 con 0x7fe99c004800 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9bda00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b806ddb0 con 0x7fe9b80c5010 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9bda00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b806e720 con 0x7fe9b8145b20 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.211+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe9b80c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe9b80c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49818/0 (socket says 172.21.15.67:49818) 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 -- 172.21.15.67:0/1676054277 learned_addr learned my addr 172.21.15.67:0/1676054277 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7400640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b80c5010 0x7fe9b80c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9bd000640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9b8145b20 0x7fe9b8145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 -- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9b8145b20 msgr2=0x7fe9b8145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9b8145b20 0x7fe9b8145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 -- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b80c5010 msgr2=0x7fe9b80c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b80c5010 0x7fe9b80c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 -- 172.21.15.67:0/1676054277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b81e6010 con 0x7fe99c004800 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7400640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b80c5010 0x7fe9b80c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.212+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe9b80c4800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d910 tx=0x7fe9a800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.213+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8019070 con 0x7fe99c004800 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.213+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a80074e0 con 0x7fe99c004800 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.213+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80049e0 con 0x7fe99c004800 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.213+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b81e62f0 con 0x7fe99c004800 2024-02-26T10:39:39.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.213+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b81e68b0 con 0x7fe99c004800 2024-02-26T10:39:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.214+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b80c1650 con 0x7fe99c004800 2024-02-26T10:39:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.215+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9a801c070 con 0x7fe99c004800 2024-02-26T10:39:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.216+0000 7fe9afe00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b4d0 0x7fe99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.216+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe9a8089dd0 con 0x7fe99c004800 2024-02-26T10:39:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.216+0000 7fe9b7400640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b4d0 0x7fe99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.218+0000 7fe9b7400640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b4d0 0x7fe99806d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4036000 tx=0x7fe9a4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.221+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9a8058c10 con 0x7fe99c004800 2024-02-26T10:39:39.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.460+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe9b80c4d40 con 0x7fe99c004800 2024-02-26T10:39:39.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.988+0000 7fe9afe00640 1 -- 172.21.15.67:0/1676054277 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe9a8058510 con 0x7fe99c004800 2024-02-26T10:39:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 4 and data pool 5 2024-02-26T10:39:40.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.999+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b4d0 msgr2=0x7fe99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.999+0000 7fe9bda00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b4d0 0x7fe99806d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4036000 tx=0x7fe9a4030000 comp rx=0 tx=0).stop 2024-02-26T10:39:40.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.999+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 msgr2=0x7fe9b80c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:39.999+0000 7fe9bda00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe9b80c4800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d910 tx=0x7fe9a800ddd0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 shutdown_connections 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9b8145b20 0x7fe9b8145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b4d0 0x7fe99806d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b80c5010 0x7fe9b80c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 --2- 172.21.15.67:0/1676054277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c004800 0x7fe9b80c4800 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 >> 172.21.15.67:0/1676054277 conn(0x7fe9b80c6380 msgr2=0x7fe9b80741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.001+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 shutdown_connections 2024-02-26T10:39:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.002+0000 7fe9bda00640 1 -- 172.21.15.67:0/1676054277 wait complete. 2024-02-26T10:39:40.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:40.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.215+0000 7f5006000640 1 Processor -- start 2024-02-26T10:39:40.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.215+0000 7f5006000640 1 -- start start 2024-02-26T10:39:40.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.217+0000 7f5006000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f500006ac20 0x7f50000a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.217+0000 7f5006000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500006b460 con 0x7f500006db00 2024-02-26T10:39:40.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.217+0000 7f5006000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500006df90 con 0x7f500006b000 2024-02-26T10:39:40.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.217+0000 7f5006000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000a9a80 con 0x7f500006ac20 2024-02-26T10:39:40.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.217+0000 7f5005600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f500006db00 0x7f50000abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49248/0 (socket says 172.21.15.67:49248) 2024-02-26T10:39:40.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.217+0000 7f5005600640 1 -- 172.21.15.67:0/705494418 learned_addr learned my addr 172.21.15.67:0/705494418 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:40.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f5004c00640 1 --2- 172.21.15.67:0/705494418 >> v2:172.21.15.82:3300/0 conn(0x7f500006ac20 0x7f50000a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654630882 0 0) 0x7f500006b460 con 0x7f500006db00 2024-02-26T10:39:40.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe8003520 con 0x7f500006db00 2024-02-26T10:39:40.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481261653 0 0) 0x7f4fe8003520 con 0x7f500006db00 2024-02-26T10:39:40.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 >> v2:172.21.15.82:3300/0 conn(0x7f500006ac20 msgr2=0x7f50000a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 --2- 172.21.15.67:0/705494418 >> v2:172.21.15.82:3300/0 conn(0x7f500006ac20 0x7f50000a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 >> v1:172.21.15.82:6789/0 conn(0x7f500006b000 legacy=0x7f5000153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50000a9d60 con 0x7f500006db00 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f4ff80019f0 con 0x7f500006db00 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 >> v1:172.21.15.67:6789/0 conn(0x7f500006db00 legacy=0x7f50000abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4fe8004800 0x7f4fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe80051c0 0x7f4fe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000a9a80 con 0x7f4fe80051c0 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800c0c0 con 0x7f4fe8003a00 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.218+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800c2c0 con 0x7f4fe8004800 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5005600640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe80051c0 0x7f4fe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5004c00640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f4fffe00640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4fe8004800 0x7f4fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5005600640 1 -- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4fe8004800 msgr2=0x7f4fe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5005600640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4fe8004800 0x7f4fe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5005600640 1 -- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe8003a00 msgr2=0x7f4fe8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5005600640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5005600640 1 -- 172.21.15.67:0/705494418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8003520 con 0x7f4fe80051c0 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f4fffe00640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4fe8004800 0x7f4fe8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.219+0000 7f5004c00640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.220+0000 7f5005600640 1 -- 172.21.15.67:0/705494418 reap_dead start 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.220+0000 7f5005600640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe80051c0 0x7f4fe800b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800a790 tx=0x7f4fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ac38a884ca57ee server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800a3b0 con 0x7f4fe80051c0 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8034670 con 0x7f4fe80051c0 2024-02-26T10:39:40.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f4fff400640 1 -- 172.21.15.67:0/705494418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8034a60 con 0x7f4fe80051c0 2024-02-26T10:39:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 -- 172.21.15.67:0/705494418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe80051c0 msgr2=0x7f4fe800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe80051c0 0x7f4fe800b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800a790 tx=0x7f4fe8005830 comp rx=0 tx=0).stop 2024-02-26T10:39:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 -- 172.21.15.67:0/705494418 shutdown_connections 2024-02-26T10:39:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 --2- 172.21.15.67:0/705494418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe80051c0 0x7f4fe800b9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 -- 172.21.15.67:0/705494418 >> 172.21.15.67:0/705494418 conn(0x7f50000c63f0 msgr2=0x7f50000c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:40.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 -- 172.21.15.67:0/705494418 shutdown_connections 2024-02-26T10:39:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.221+0000 7f5006000640 1 -- 172.21.15.67:0/705494418 wait complete. 2024-02-26T10:39:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.223+0000 7f5006000640 1 Processor -- start 2024-02-26T10:39:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.223+0000 7f5006000640 1 -- start start 2024-02-26T10:39:40.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.224+0000 7f5006000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe80051c0 0x7f50001e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.224+0000 7f5006000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 0x7f5000070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.224+0000 7f5006000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50000c3fe0 0x7f5000149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.224+0000 7f5006000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5000150410 con 0x7f500014f940 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.224+0000 7f5006000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000aba30 con 0x7f4fe80051c0 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.224+0000 7f5006000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5000150590 con 0x7f50000c3fe0 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 0x7f5000070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 0x7f5000070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42064/0 (socket says 172.21.15.67:42064) 2024-02-26T10:39:40.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 -- 172.21.15.67:0/1732933275 learned_addr learned my addr 172.21.15.67:0/1732933275 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f5005600640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50000c3fe0 0x7f5000149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 -- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50000c3fe0 msgr2=0x7f5000149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50000c3fe0 0x7f5000149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f5004c00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe80051c0 0x7f50001e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 -- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe80051c0 msgr2=0x7f50001e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe80051c0 0x7f50001e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 -- 172.21.15.67:0/1732933275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500014e9e0 con 0x7f500014f940 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.225+0000 7f4fffe00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 0x7f5000070c50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00d5a0 tx=0x7f4fec00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.226+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec005280 con 0x7f500014f940 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.226+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec005420 con 0x7f500014f940 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.226+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500014a1d0 con 0x7f500014f940 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.226+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec007260 con 0x7f500014f940 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.226+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500014e700 con 0x7f500014f940 2024-02-26T10:39:40.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.227+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5000074a30 con 0x7f500014f940 2024-02-26T10:39:40.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.228+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fec019020 con 0x7f500014f940 2024-02-26T10:39:40.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.229+0000 7f4ffcc00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fe006b500 0x7f4fe006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.229+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fec089920 con 0x7f500014f940 2024-02-26T10:39:40.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.230+0000 7f5004c00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fe006b500 0x7f4fe006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.231+0000 7f5004c00640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fe006b500 0x7f4fe006d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f500014e350 tx=0x7f4ff4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.234+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fec0587a0 con 0x7f500014f940 2024-02-26T10:39:40.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.437+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f500006b9b0 con 0x7f500014f940 2024-02-26T10:39:40.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.439+0000 7f4ffcc00640 1 -- 172.21.15.67:0/1732933275 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f4fec0580a0 con 0x7f500014f940 2024-02-26T10:39:40.440 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:40.440 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:39.462501+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fe006b500 msgr2=0x7f4fe006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fe006b500 0x7f4fe006d9c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f500014e350 tx=0x7f4ff4008040 comp rx=0 tx=0).stop 2024-02-26T10:39:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 msgr2=0x7f5000070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 0x7f5000070c50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00d5a0 tx=0x7f4fec00da60 comp rx=0 tx=0).stop 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 shutdown_connections 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50000c3fe0 0x7f5000149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500014f940 0x7f5000070c50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fe006b500 0x7f4fe006d9c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 --2- 172.21.15.67:0/1732933275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe80051c0 0x7f50001e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 >> 172.21.15.67:0/1732933275 conn(0x7f50000c63f0 msgr2=0x7f500014c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 shutdown_connections 2024-02-26T10:39:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.442+0000 7f5006000640 1 -- 172.21.15.67:0/1732933275 wait complete. 2024-02-26T10:39:40.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.675+0000 7fb060c00640 1 Processor -- start 2024-02-26T10:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.675+0000 7fb060c00640 1 -- start start 2024-02-26T10:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.677+0000 7fb060c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb05c072fe0 0x7fb05c06e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.677+0000 7fb060c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 0x7fb05c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.677+0000 7fb060c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05c15c8a0 con 0x7fb05c156810 2024-02-26T10:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.677+0000 7fb060c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05c15cb80 con 0x7fb05c072fe0 2024-02-26T10:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.677+0000 7fb060c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c15ce60 con 0x7fb05c159210 2024-02-26T10:39:40.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.678+0000 7fb05a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 0x7fb05c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.678+0000 7fb05a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 0x7fb05c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:38096/0 (socket says 172.21.15.67:38096) 2024-02-26T10:39:40.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.678+0000 7fb05a800640 1 -- 172.21.15.67:0/1902802188 learned_addr learned my addr 172.21.15.67:0/1902802188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:40.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.678+0000 7fb059400640 1 --2- 172.21.15.67:0/1902802188 >> v2:172.21.15.82:3300/0 conn(0x7fb05c072fe0 0x7fb05c06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.679+0000 7fb05a800640 1 -- 172.21.15.67:0/1902802188 >> v2:172.21.15.82:3300/0 conn(0x7fb05c072fe0 msgr2=0x7fb05c06e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:40.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.679+0000 7fb05a800640 1 --2- 172.21.15.67:0/1902802188 >> v2:172.21.15.82:3300/0 conn(0x7fb05c072fe0 0x7fb05c06e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.679+0000 7fb05a800640 1 -- 172.21.15.67:0/1902802188 >> v1:172.21.15.82:6789/0 conn(0x7fb05c156810 legacy=0x7fb05c0c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.679+0000 7fb05a800640 1 -- 172.21.15.67:0/1902802188 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c15d140 con 0x7fb05c159210 2024-02-26T10:39:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.680+0000 7fb05a800640 1 --2- 172.21.15.67:0/1902802188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 0x7fb05c159660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb054009870 tx=0x7fb05402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194da2b00ab1727b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.681+0000 7fb058a00640 1 -- 172.21.15.67:0/1902802188 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054035020 con 0x7fb05c159210 2024-02-26T10:39:40.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.681+0000 7fb058a00640 1 -- 172.21.15.67:0/1902802188 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb054031640 con 0x7fb05c159210 2024-02-26T10:39:40.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.681+0000 7fb058a00640 1 -- 172.21.15.67:0/1902802188 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05402d040 con 0x7fb05c159210 2024-02-26T10:39:40.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.681+0000 7fb060c00640 1 -- 172.21.15.67:0/1902802188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 msgr2=0x7fb05c159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.681+0000 7fb060c00640 1 --2- 172.21.15.67:0/1902802188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 0x7fb05c159660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb054009870 tx=0x7fb05402b8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.682+0000 7fb060c00640 1 -- 172.21.15.67:0/1902802188 shutdown_connections 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.682+0000 7fb060c00640 1 --2- 172.21.15.67:0/1902802188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c159210 0x7fb05c159660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.682+0000 7fb060c00640 1 --2- 172.21.15.67:0/1902802188 >> v2:172.21.15.82:3300/0 conn(0x7fb05c072fe0 0x7fb05c06e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.682+0000 7fb060c00640 1 -- 172.21.15.67:0/1902802188 >> 172.21.15.67:0/1902802188 conn(0x7fb05c0a84a0 msgr2=0x7fb05c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.682+0000 7fb060c00640 1 -- 172.21.15.67:0/1902802188 shutdown_connections 2024-02-26T10:39:40.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.682+0000 7fb060c00640 1 -- 172.21.15.67:0/1902802188 wait complete. 2024-02-26T10:39:40.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.684+0000 7fb060c00640 1 Processor -- start 2024-02-26T10:39:40.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.684+0000 7fb060c00640 1 -- start start 2024-02-26T10:39:40.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.685+0000 7fb060c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c072fe0 0x7fb05c151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.685+0000 7fb060c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 0x7fb05c0a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.685+0000 7fb060c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb05c159210 0x7fb05c14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.685+0000 7fb060c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c06ec40 con 0x7fb05c156810 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.685+0000 7fb060c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c15d660 con 0x7fb05c159210 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.685+0000 7fb060c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c159ba0 con 0x7fb05c072fe0 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 0x7fb05c0a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 0x7fb05c0a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42078/0 (socket says 172.21.15.67:42078) 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 -- 172.21.15.67:0/6602135 learned_addr learned my addr 172.21.15.67:0/6602135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059e00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c072fe0 0x7fb05c151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb05a800640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb05c159210 0x7fb05c14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 -- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c072fe0 msgr2=0x7fb05c151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c072fe0 0x7fb05c151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 -- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb05c159210 msgr2=0x7fb05c14b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb05c159210 0x7fb05c14b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.686+0000 7fb059400640 1 -- 172.21.15.67:0/6602135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c14c490 con 0x7fb05c156810 2024-02-26T10:39:40.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.687+0000 7fb059400640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 0x7fb05c0a9bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb04400e9e0 tx=0x7fb04400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.687+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044017070 con 0x7fb05c156810 2024-02-26T10:39:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.687+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400c9d0 con 0x7fb05c156810 2024-02-26T10:39:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.687+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c0ab400 con 0x7fb05c156810 2024-02-26T10:39:40.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.687+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400cdc0 con 0x7fb05c156810 2024-02-26T10:39:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.687+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c1523b0 con 0x7fb05c156810 2024-02-26T10:39:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.688+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c06efe0 con 0x7fb05c156810 2024-02-26T10:39:40.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.689+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb04401a070 con 0x7fb05c156810 2024-02-26T10:39:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.690+0000 7fb052000640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb03806b500 0x7fb03806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.690+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb04408a160 con 0x7fb05c156810 2024-02-26T10:39:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.691+0000 7fb059e00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb03806b500 0x7fb03806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.692+0000 7fb059e00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb03806b500 0x7fb03806d9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb05c14ceb0 tx=0x7fb04c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.697+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb05c06efe0 con 0x7fb05c156810 2024-02-26T10:39:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.897+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb05c074a30 con 0x7fb05c156810 2024-02-26T10:39:40.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.899+0000 7fb052000640 1 -- 172.21.15.67:0/6602135 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fb044059070 con 0x7fb05c156810 2024-02-26T10:39:40.901 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:40.901 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:39.462501+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:40.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb03806b500 msgr2=0x7fb03806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb03806b500 0x7fb03806d9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb05c14ceb0 tx=0x7fb04c009040 comp rx=0 tx=0).stop 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 msgr2=0x7fb05c0a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 0x7fb05c0a9bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb04400e9e0 tx=0x7fb04400eea0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 shutdown_connections 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb03806b500 0x7fb03806d9c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb05c159210 0x7fb05c14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb05c156810 0x7fb05c0a9bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 --2- 172.21.15.67:0/6602135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb05c072fe0 0x7fb05c151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 >> 172.21.15.67:0/6602135 conn(0x7fb05c0a84a0 msgr2=0x7fb05c0c1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 shutdown_connections 2024-02-26T10:39:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:40.902+0000 7fb060c00640 1 -- 172.21.15.67:0/6602135 wait complete. 2024-02-26T10:39:40.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T10:39:41.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.150+0000 7fc5af600640 1 Processor -- start 2024-02-26T10:39:41.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.150+0000 7fc5af600640 1 -- start start 2024-02-26T10:39:41.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.152+0000 7fc5af600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5a80c0ef0 0x7fc5a8158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.152+0000 7fc5af600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a8072ce0 con 0x7fc5a81591b0 2024-02-26T10:39:41.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.152+0000 7fc5af600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a8155bf0 con 0x7fc5a80c1400 2024-02-26T10:39:41.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.152+0000 7fc5af600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a8073010 con 0x7fc5a80c0ef0 2024-02-26T10:39:41.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.152+0000 7fc5acc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc5a81591b0 0x7fc5a8070ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49262/0 (socket says 172.21.15.67:49262) 2024-02-26T10:39:41.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.152+0000 7fc5acc00640 1 -- 172.21.15.67:0/3528834744 learned_addr learned my addr 172.21.15.67:0/3528834744 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184669409 0 0) 0x7fc5a8072ce0 con 0x7fc5a81591b0 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7e00640 1 --2- 172.21.15.67:0/3528834744 >> v2:172.21.15.82:3300/0 conn(0x7fc5a80c0ef0 0x7fc5a8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc590003520 con 0x7fc5a81591b0 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2725752478 0 0) 0x7fc590003520 con 0x7fc5a81591b0 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 >> v2:172.21.15.82:3300/0 conn(0x7fc5a80c0ef0 msgr2=0x7fc5a8158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 --2- 172.21.15.67:0/3528834744 >> v2:172.21.15.82:3300/0 conn(0x7fc5a80c0ef0 0x7fc5a8158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 >> v1:172.21.15.82:6789/0 conn(0x7fc5a80c1400 legacy=0x7fc5a806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8155ed0 con 0x7fc5a81591b0 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fc59c001690 con 0x7fc5a81591b0 2024-02-26T10:39:41.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 >> v1:172.21.15.67:6789/0 conn(0x7fc5a81591b0 legacy=0x7fc5a8070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5900039b0 0x7fc590003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5900052e0 0x7fc59000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8073010 con 0x7fc5900052e0 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000c1b0 con 0x7fc590004800 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.153+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000c3b0 con 0x7fc5900039b0 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5ad600640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5900052e0 0x7fc59000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5acc00640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5900039b0 0x7fc590003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5a7e00640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5ad600640 1 -- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5900039b0 msgr2=0x7fc590003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5ad600640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5900039b0 0x7fc590003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5ad600640 1 -- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc590004800 msgr2=0x7fc590004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5ad600640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:41.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.154+0000 7fc5ad600640 1 -- 172.21.15.67:0/3528834744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590003520 con 0x7fc5900052e0 2024-02-26T10:39:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.155+0000 7fc5ad600640 1 -- 172.21.15.67:0/3528834744 reap_dead start 2024-02-26T10:39:41.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.155+0000 7fc5ad600640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5900052e0 0x7fc59000b9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001e20 tx=0x7fc590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986cb21baeb34cb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.155+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0001a70 con 0x7fc5900052e0 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0003b70 con 0x7fc5900052e0 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5a7400640 1 -- 172.21.15.67:0/3528834744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00050d0 con 0x7fc5900052e0 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 -- 172.21.15.67:0/3528834744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5900052e0 msgr2=0x7fc59000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5900052e0 0x7fc59000b9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001e20 tx=0x7fc590005830 comp rx=0 tx=0).stop 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 -- 172.21.15.67:0/3528834744 shutdown_connections 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 --2- 172.21.15.67:0/3528834744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5900052e0 0x7fc59000b9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 -- 172.21.15.67:0/3528834744 >> 172.21.15.67:0/3528834744 conn(0x7fc5a80c6da0 msgr2=0x7fc5a80c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 -- 172.21.15.67:0/3528834744 shutdown_connections 2024-02-26T10:39:41.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.156+0000 7fc5af600640 1 -- 172.21.15.67:0/3528834744 wait complete. 2024-02-26T10:39:41.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.158+0000 7fc5af600640 1 Processor -- start 2024-02-26T10:39:41.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.158+0000 7fc5af600640 1 -- start start 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5af600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5900052e0 0x7fc5a80a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5af600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 0x7fc5a80c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5af600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5a81edbe0 0x7fc5a81ee060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5af600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8158580 con 0x7fc5a80c40e0 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5af600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a814f490 con 0x7fc5900052e0 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5af600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8158700 con 0x7fc5a81edbe0 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 0x7fc5a80c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 0x7fc5a80c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42096/0 (socket says 172.21.15.67:42096) 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.159+0000 7fc5a7e00640 1 -- 172.21.15.67:0/910340526 learned_addr learned my addr 172.21.15.67:0/910340526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5acc00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5900052e0 0x7fc5a80a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5ad600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5a81edbe0 0x7fc5a81ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a7e00640 1 -- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5a81edbe0 msgr2=0x7fc5a81ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a7e00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5a81edbe0 0x7fc5a81ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a7e00640 1 -- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5900052e0 msgr2=0x7fc5a80a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a7e00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5900052e0 0x7fc5a80a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a7e00640 1 -- 172.21.15.67:0/910340526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8150c30 con 0x7fc5a80c40e0 2024-02-26T10:39:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a7e00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 0x7fc5a80c4560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc59800f550 tx=0x7fc59800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598013810 con 0x7fc5a80c40e0 2024-02-26T10:39:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59801a070 con 0x7fc5a80c40e0 2024-02-26T10:39:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598013bb0 con 0x7fc5a80c40e0 2024-02-26T10:39:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.160+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80aa990 con 0x7fc5a80c40e0 2024-02-26T10:39:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.161+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80abd40 con 0x7fc5a80c40e0 2024-02-26T10:39:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.162+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a80abb50 con 0x7fc5a80c40e0 2024-02-26T10:39:41.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.163+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc598012070 con 0x7fc5a80c40e0 2024-02-26T10:39:41.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.164+0000 7fc5a4c00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc58c06b500 0x7fc58c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:41.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.164+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc59808a0f0 con 0x7fc5a80c40e0 2024-02-26T10:39:41.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.164+0000 7fc5acc00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc58c06b500 0x7fc58c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:41.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.165+0000 7fc5acc00640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc58c06b500 0x7fc58c06d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc59c001050 tx=0x7fc59c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:41.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.168+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc598059000 con 0x7fc5a80c40e0 2024-02-26T10:39:41.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:41.411+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5a8074a30 con 0x7fc5a80c40e0 2024-02-26T10:39:42.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.002+0000 7fc5a4c00640 1 -- 172.21.15.67:0/910340526 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc598058900 con 0x7fc5a80c40e0 2024-02-26T10:39:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc58c06b500 msgr2=0x7fc58c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc58c06b500 0x7fc58c06d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc59c001050 tx=0x7fc59c009040 comp rx=0 tx=0).stop 2024-02-26T10:39:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 msgr2=0x7fc5a80c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 0x7fc5a80c4560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc59800f550 tx=0x7fc59800fa10 comp rx=0 tx=0).stop 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 shutdown_connections 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5a81edbe0 0x7fc5a81ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5a80c40e0 0x7fc5a80c4560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc58c06b500 0x7fc58c06d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 --2- 172.21.15.67:0/910340526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5900052e0 0x7fc5a80a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 >> 172.21.15.67:0/910340526 conn(0x7fc5a80c6da0 msgr2=0x7fc5a8154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.006+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 shutdown_connections 2024-02-26T10:39:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.007+0000 7fc5af600640 1 -- 172.21.15.67:0/910340526 wait complete. 2024-02-26T10:39:42.029 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:39:42.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-26T10:39:42.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.257+0000 7f8deca00640 1 Processor -- start 2024-02-26T10:39:42.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.257+0000 7f8deca00640 1 -- start start 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.258+0000 7f8deca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8de80c25a0 0x7f8de80c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8deca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8de8151980 0x7f8de8151d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8deca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de80c1be0 con 0x7f8de80c1eb0 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8deca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de80c1750 con 0x7f8de8151980 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8deca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de80c3660 con 0x7f8de80c25a0 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8de8151980 0x7f8de8151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8de8151980 0x7f8de8151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42098/0 (socket says 172.21.15.67:42098) 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de6a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8de80c1eb0 0x7f8de814ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49266/0 (socket says 172.21.15.67:49266) 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 -- 172.21.15.67:0/2430316752 learned_addr learned my addr 172.21.15.67:0/2430316752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:42.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7400640 1 --2- 172.21.15.67:0/2430316752 >> v2:172.21.15.82:3300/0 conn(0x7f8de80c25a0 0x7f8de80c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 -- 172.21.15.67:0/2430316752 >> v2:172.21.15.82:3300/0 conn(0x7f8de80c25a0 msgr2=0x7f8de80c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 --2- 172.21.15.67:0/2430316752 >> v2:172.21.15.82:3300/0 conn(0x7f8de80c25a0 0x7f8de80c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 -- 172.21.15.67:0/2430316752 >> v1:172.21.15.67:6789/0 conn(0x7f8de80c1eb0 legacy=0x7f8de814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de7e00640 1 -- 172.21.15.67:0/2430316752 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80c0e20 con 0x7f8de8151980 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.259+0000 7f8de6000640 1 -- 172.21.15.67:0/2430316752 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472035552 0 0) 0x7f8de80c1be0 con 0x7f8de80c1eb0 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.261+0000 7f8de7e00640 1 --2- 172.21.15.67:0/2430316752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de8151d60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8de80a91f0 tx=0x7f8de002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d41073c3b8279d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.261+0000 7f8de6000640 1 -- 172.21.15.67:0/2430316752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de002ca80 con 0x7f8de8151980 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.261+0000 7f8de6000640 1 -- 172.21.15.67:0/2430316752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de002cc20 con 0x7f8de8151980 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.261+0000 7f8de6000640 1 -- 172.21.15.67:0/2430316752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de00343f0 con 0x7f8de8151980 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 -- 172.21.15.67:0/2430316752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 msgr2=0x7f8de8151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 --2- 172.21.15.67:0/2430316752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de8151d60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8de80a91f0 tx=0x7f8de002bee0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 -- 172.21.15.67:0/2430316752 shutdown_connections 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 --2- 172.21.15.67:0/2430316752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de8151d60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 --2- 172.21.15.67:0/2430316752 >> v2:172.21.15.82:3300/0 conn(0x7f8de80c25a0 0x7f8de80c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 -- 172.21.15.67:0/2430316752 >> 172.21.15.67:0/2430316752 conn(0x7f8de80c6380 msgr2=0x7f8de80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:42.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 -- 172.21.15.67:0/2430316752 shutdown_connections 2024-02-26T10:39:42.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.262+0000 7f8deca00640 1 -- 172.21.15.67:0/2430316752 wait complete. 2024-02-26T10:39:42.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.263+0000 7f8deca00640 1 Processor -- start 2024-02-26T10:39:42.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.264+0000 7f8deca00640 1 -- start start 2024-02-26T10:39:42.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.264+0000 7f8deca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8de80c1eb0 0x7f8de8153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:42.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.264+0000 7f8deca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8de80c25a0 0x7f8de80c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:42.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8deca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de813bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8deca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80c3350 con 0x7f8de8151980 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8deca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80c13e0 con 0x7f8de80c25a0 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8deca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80c2980 con 0x7f8de80c1eb0 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de813bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de813bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42112/0 (socket says 172.21.15.67:42112) 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 -- 172.21.15.67:0/2499951641 learned_addr learned my addr 172.21.15.67:0/2499951641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7400640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8de80c1eb0 0x7f8de8153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de6a00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8de80c25a0 0x7f8de80c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 -- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8de80c1eb0 msgr2=0x7f8de8153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8de80c1eb0 0x7f8de8153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 -- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8de80c25a0 msgr2=0x7f8de80c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8de80c25a0 0x7f8de80c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7e00640 1 -- 172.21.15.67:0/2499951641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8154f50 con 0x7f8de8151980 2024-02-26T10:39:42.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.265+0000 7f8de7400640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8de80c1eb0 0x7f8de8153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:42.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.266+0000 7f8de7e00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de813bc10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009f40 tx=0x7f8de0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:42.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.266+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0043020 con 0x7f8de8151980 2024-02-26T10:39:42.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.266+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de003e450 con 0x7f8de8151980 2024-02-26T10:39:42.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.266+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0009510 con 0x7f8de8151980 2024-02-26T10:39:42.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.266+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de8155cf0 con 0x7f8de8151980 2024-02-26T10:39:42.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.266+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de8155210 con 0x7f8de8151980 2024-02-26T10:39:42.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.268+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de80678e0 con 0x7f8de8151980 2024-02-26T10:39:42.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.268+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8de0046020 con 0x7f8de8151980 2024-02-26T10:39:42.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.269+0000 7f8ddf400640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.269+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8de00ac9c0 con 0x7f8de8151980 2024-02-26T10:39:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.270+0000 7f8de7400640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:42.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.271+0000 7f8de7400640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8dc406b4d0 0x7f8dc406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8006fd0 tx=0x7f8dd8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.274+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8de00aee20 con 0x7f8de8151980 2024-02-26T10:39:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:42.517+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8de8074a30 con 0x7f8de8151980 2024-02-26T10:39:42.824 INFO:tasks.ceph.mds.d.smithi082.stderr:2024-02-26T10:39:42.823+0000 7f19f2e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:39:43.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.016+0000 7f8ddf400640 1 -- 172.21.15.67:0/2499951641 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8de004d030 con 0x7f8de8151980 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8dc406b4d0 msgr2=0x7f8dc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8dc406b4d0 0x7f8dc406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8006fd0 tx=0x7f8dd8008040 comp rx=0 tx=0).stop 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 msgr2=0x7f8de813bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de813bc10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009f40 tx=0x7f8de0004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 shutdown_connections 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8de8151980 0x7f8de813bc10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8de80c25a0 0x7f8de80c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 --2- 172.21.15.67:0/2499951641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8de80c1eb0 0x7f8de8153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.020+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 >> 172.21.15.67:0/2499951641 conn(0x7f8de80c6380 msgr2=0x7f8de80716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.021+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 shutdown_connections 2024-02-26T10:39:43.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.021+0000 7f8deca00640 1 -- 172.21.15.67:0/2499951641 wait complete. 2024-02-26T10:39:43.046 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-26T10:39:43.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:43.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.265+0000 7f7d27600640 1 Processor -- start 2024-02-26T10:39:43.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.265+0000 7f7d27600640 1 -- start start 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d27600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7d20150300 0x7f7d200c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d27600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d200c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d27600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d20156230 0x7f7d200aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d27600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d201506e0 con 0x7f7d20156230 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d27600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d20153a60 con 0x7f7d20150300 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d27600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d20152e50 con 0x7f7d2006e410 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d25600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d20156230 0x7f7d200aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d25600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d20156230 0x7f7d200aaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42124/0 (socket says 172.21.15.67:42124) 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.267+0000 7f7d25600640 1 -- 172.21.15.67:0/429031732 learned_addr learned my addr 172.21.15.67:0/429031732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d1fe00640 1 --2- 172.21.15.67:0/429031732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d200c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d24c00640 1 --2- 172.21.15.67:0/429031732 >> v2:172.21.15.82:3300/0 conn(0x7f7d20150300 0x7f7d200c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d25600640 1 -- 172.21.15.67:0/429031732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 msgr2=0x7f7d200c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d25600640 1 --2- 172.21.15.67:0/429031732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d200c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d25600640 1 -- 172.21.15.67:0/429031732 >> v2:172.21.15.82:3300/0 conn(0x7f7d20150300 msgr2=0x7f7d200c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d25600640 1 --2- 172.21.15.67:0/429031732 >> v2:172.21.15.82:3300/0 conn(0x7f7d20150300 0x7f7d200c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.268+0000 7f7d25600640 1 -- 172.21.15.67:0/429031732 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20153130 con 0x7f7d20156230 2024-02-26T10:39:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.269+0000 7f7d25600640 1 --2- 172.21.15.67:0/429031732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d200aaba0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7d2006bcc0 tx=0x7f7d1802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936f307803f3cd76 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.269+0000 7f7d1f400640 1 -- 172.21.15.67:0/429031732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1802c960 con 0x7f7d20156230 2024-02-26T10:39:43.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d1f400640 1 -- 172.21.15.67:0/429031732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1802cb00 con 0x7f7d20156230 2024-02-26T10:39:43.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d1f400640 1 -- 172.21.15.67:0/429031732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180343f0 con 0x7f7d20156230 2024-02-26T10:39:43.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 -- 172.21.15.67:0/429031732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 msgr2=0x7f7d200aaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 --2- 172.21.15.67:0/429031732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d200aaba0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7d2006bcc0 tx=0x7f7d1802bdc0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 -- 172.21.15.67:0/429031732 shutdown_connections 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 --2- 172.21.15.67:0/429031732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d200aaba0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 --2- 172.21.15.67:0/429031732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d200c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 --2- 172.21.15.67:0/429031732 >> v2:172.21.15.82:3300/0 conn(0x7f7d20150300 0x7f7d200c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 -- 172.21.15.67:0/429031732 >> 172.21.15.67:0/429031732 conn(0x7f7d200a84a0 msgr2=0x7f7d200a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 -- 172.21.15.67:0/429031732 shutdown_connections 2024-02-26T10:39:43.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.270+0000 7f7d27600640 1 -- 172.21.15.67:0/429031732 wait complete. 2024-02-26T10:39:43.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.272+0000 7f7d27600640 1 Processor -- start 2024-02-26T10:39:43.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.272+0000 7f7d27600640 1 -- start start 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d27600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d20149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d27600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d20150300 0x7f7d20151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d27600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d20151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d27600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2006dd80 con 0x7f7d20156230 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d27600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d201536f0 con 0x7f7d20150300 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d27600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d200ab0e0 con 0x7f7d2006e410 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d25600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d20151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d25600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d20151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42132/0 (socket says 172.21.15.67:42132) 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.273+0000 7f7d25600640 1 -- 172.21.15.67:0/3341744875 learned_addr learned my addr 172.21.15.67:0/3341744875 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d24c00640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d20149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d25600640 1 -- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 msgr2=0x7f7d20149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d1fe00640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d20150300 0x7f7d20151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d25600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d20149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d25600640 1 -- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d20150300 msgr2=0x7f7d20151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d25600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d20150300 0x7f7d20151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d25600640 1 -- 172.21.15.67:0/3341744875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2014c480 con 0x7f7d20156230 2024-02-26T10:39:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d25600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d20151d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009f40 tx=0x7f7d18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.274+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18043020 con 0x7f7d20156230 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.275+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1803e450 con 0x7f7d20156230 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.275+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18009510 con 0x7f7d20156230 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.275+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d20150860 con 0x7f7d20156230 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.275+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2014c720 con 0x7f7d20156230 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.276+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d20074a30 con 0x7f7d20156230 2024-02-26T10:39:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.277+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d18046020 con 0x7f7d20156230 2024-02-26T10:39:43.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.277+0000 7f7d1cc00640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7cfc06b500 0x7f7cfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.278+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d180aca00 con 0x7f7d20156230 2024-02-26T10:39:43.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.278+0000 7f7d24c00640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7cfc06b500 0x7f7cfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.279+0000 7f7d24c00640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7cfc06b500 0x7f7cfc06d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.282+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d1804d030 con 0x7f7d20156230 2024-02-26T10:39:43.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.517+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d200c0bf0 con 0x7f7d20156230 2024-02-26T10:39:43.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.518+0000 7f7d1cc00640 1 -- 172.21.15.67:0/3341744875 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f7d1807b8b0 con 0x7f7d20156230 2024-02-26T10:39:43.519 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:43.520 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-26T10:39:43.520 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-02-26T10:39:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7cfc06b500 msgr2=0x7f7cfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7cfc06b500 0x7f7cfc06d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d08006fd0 tx=0x7f7d08008040 comp rx=0 tx=0).stop 2024-02-26T10:39:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 msgr2=0x7f7d20151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d20151d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009f40 tx=0x7f7d18004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 shutdown_connections 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d20156230 0x7f7d20151d40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7cfc06b500 0x7f7cfc06d9c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d20150300 0x7f7d20151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 --2- 172.21.15.67:0/3341744875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d2006e410 0x7f7d20149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.521+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 >> 172.21.15.67:0/3341744875 conn(0x7f7d200a84a0 msgr2=0x7f7d200c2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.522+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 shutdown_connections 2024-02-26T10:39:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.522+0000 7f7d27600640 1 -- 172.21.15.67:0/3341744875 wait complete. 2024-02-26T10:39:43.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:43.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.763+0000 7fe9a5a00640 1 Processor -- start 2024-02-26T10:39:43.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.763+0000 7fe9a5a00640 1 -- start start 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9a00c12e0 0x7fe9a006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9a00c19b0 0x7fe9a01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a00c1de0 con 0x7fe9a006ac20 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a015a090 con 0x7fe9a00c19b0 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a0159ad0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9a00c19b0 0x7fe9a01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9a00c19b0 0x7fe9a01590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42146/0 (socket says 172.21.15.67:42146) 2024-02-26T10:39:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe9a5000640 1 -- 172.21.15.67:0/3222192197 learned_addr learned my addr 172.21.15.67:0/3222192197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:43.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.765+0000 7fe99f400640 1 --2- 172.21.15.67:0/3222192197 >> v2:172.21.15.82:3300/0 conn(0x7fe9a00c12e0 0x7fe9a006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.766+0000 7fe9a5000640 1 -- 172.21.15.67:0/3222192197 >> v2:172.21.15.82:3300/0 conn(0x7fe9a00c12e0 msgr2=0x7fe9a006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:43.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.766+0000 7fe9a5000640 1 --2- 172.21.15.67:0/3222192197 >> v2:172.21.15.82:3300/0 conn(0x7fe9a00c12e0 0x7fe9a006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.766+0000 7fe9a5000640 1 -- 172.21.15.67:0/3222192197 >> v1:172.21.15.82:6789/0 conn(0x7fe9a006ac20 legacy=0x7fe9a006e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.766+0000 7fe9a5000640 1 -- 172.21.15.67:0/3222192197 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0159db0 con 0x7fe9a00c19b0 2024-02-26T10:39:43.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.767+0000 7fe9a5000640 1 --2- 172.21.15.67:0/3222192197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c19b0 0x7fe9a01590f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00726f0 tx=0x7fe99802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f559b12dd43bb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:43.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.767+0000 7fe99ea00640 1 -- 172.21.15.67:0/3222192197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99802c9f0 con 0x7fe9a00c19b0 2024-02-26T10:39:43.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.767+0000 7fe99ea00640 1 -- 172.21.15.67:0/3222192197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99802cb90 con 0x7fe9a00c19b0 2024-02-26T10:39:43.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.767+0000 7fe99ea00640 1 -- 172.21.15.67:0/3222192197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980343f0 con 0x7fe9a00c19b0 2024-02-26T10:39:43.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3222192197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c19b0 msgr2=0x7fe9a01590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3222192197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c19b0 0x7fe9a01590f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00726f0 tx=0x7fe99802be50 comp rx=0 tx=0).stop 2024-02-26T10:39:43.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3222192197 shutdown_connections 2024-02-26T10:39:43.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3222192197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c19b0 0x7fe9a01590f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3222192197 >> v2:172.21.15.82:3300/0 conn(0x7fe9a00c12e0 0x7fe9a006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3222192197 >> 172.21.15.67:0/3222192197 conn(0x7fe9a00c63f0 msgr2=0x7fe9a00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:43.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3222192197 shutdown_connections 2024-02-26T10:39:43.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.768+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3222192197 wait complete. 2024-02-26T10:39:43.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.770+0000 7fe9a5a00640 1 Processor -- start 2024-02-26T10:39:43.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.770+0000 7fe9a5a00640 1 -- start start 2024-02-26T10:39:43.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.771+0000 7fe9a5a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9a006ac20 0x7fe9a0153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.771+0000 7fe9a5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 0x7fe9a01583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.771+0000 7fe9a5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9a00c19b0 0x7fe9a0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.771+0000 7fe9a5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a01534d0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.771+0000 7fe9a5a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a006df10 con 0x7fe9a006ac20 2024-02-26T10:39:43.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.771+0000 7fe9a5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a0159630 con 0x7fe9a00c19b0 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 0x7fe9a01583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 0x7fe9a01583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42152/0 (socket says 172.21.15.67:42152) 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 -- 172.21.15.67:0/3505667758 learned_addr learned my addr 172.21.15.67:0/3505667758 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 -- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9a00c19b0 msgr2=0x7fe9a0154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9a00c19b0 0x7fe9a0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 -- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9a006ac20 msgr2=0x7fe9a0153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99fe00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9a006ac20 0x7fe9a0153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9a006ac20 0x7fe9a0153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.772+0000 7fe99f400640 1 -- 172.21.15.67:0/3505667758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a01f2ce0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.773+0000 7fe99f400640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 0x7fe9a01583c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00e8d0 tx=0x7fe98c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.773+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c017070 con 0x7fe9a00c12e0 2024-02-26T10:39:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.773+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c00c940 con 0x7fe9a00c12e0 2024-02-26T10:39:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.773+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00cd30 con 0x7fe9a00c12e0 2024-02-26T10:39:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.773+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a01f2fc0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.773+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0154bb0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.774+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a0074a30 con 0x7fe9a00c12e0 2024-02-26T10:39:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.775+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe98c01a070 con 0x7fe9a00c12e0 2024-02-26T10:39:43.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.776+0000 7fe997e00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:43.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.776+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe98c08a5f0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.777+0000 7fe99fe00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:43.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.778+0000 7fe99fe00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe98006b4d0 0x7fe98006d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9900015a0 tx=0x7fe990009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:43.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.780+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe98c059460 con 0x7fe9a00c12e0 2024-02-26T10:39:43.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.979+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9a00c09d0 con 0x7fe9a00c12e0 2024-02-26T10:39:43.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.980+0000 7fe997e00640 1 -- 172.21.15.67:0/3505667758 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fe98c058d60 con 0x7fe9a00c12e0 2024-02-26T10:39:43.982 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:43.982 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":34,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:40.989006+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe98006b4d0 msgr2=0x7fe98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe98006b4d0 0x7fe98006d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9900015a0 tx=0x7fe990009040 comp rx=0 tx=0).stop 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 msgr2=0x7fe9a01583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 0x7fe9a01583c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00e8d0 tx=0x7fe98c00ed90 comp rx=0 tx=0).stop 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 shutdown_connections 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9a00c19b0 0x7fe9a0154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a00c12e0 0x7fe9a01583c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 --2- 172.21.15.67:0/3505667758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9a006ac20 0x7fe9a0153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.983+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 >> 172.21.15.67:0/3505667758 conn(0x7fe9a00c63f0 msgr2=0x7fe9a00c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.984+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 shutdown_connections 2024-02-26T10:39:43.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:43.984+0000 7fe9a5a00640 1 -- 172.21.15.67:0/3505667758 wait complete. 2024-02-26T10:39:44.002 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:39:44.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:39:44.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.221+0000 7efd87a00640 1 Processor -- start 2024-02-26T10:39:44.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.221+0000 7efd87a00640 1 -- start start 2024-02-26T10:39:44.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd87a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd80150300 0x7efd800c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd87a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd80152e50 con 0x7efd80156230 2024-02-26T10:39:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd87a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd80153130 con 0x7efd8006e410 2024-02-26T10:39:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd87a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd801506e0 con 0x7efd80150300 2024-02-26T10:39:44.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd85000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd80150300 0x7efd800c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd85a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efd80156230 0x7efd800ab2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49282/0 (socket says 172.21.15.67:49282) 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd85a00640 1 -- 172.21.15.67:0/1328073536 learned_addr learned my addr 172.21.15.67:0/1328073536 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd85000640 1 -- 172.21.15.67:0/1328073536 >> v1:172.21.15.82:6789/0 conn(0x7efd8006e410 legacy=0x7efd800aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.223+0000 7efd85000640 1 -- 172.21.15.67:0/1328073536 >> v1:172.21.15.67:6789/0 conn(0x7efd80156230 legacy=0x7efd800ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.224+0000 7efd85000640 1 -- 172.21.15.67:0/1328073536 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80153410 con 0x7efd80150300 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.224+0000 7efd7fe00640 1 -- 172.21.15.67:0/1328073536 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967090123 0 0) 0x7efd80152e50 con 0x7efd80156230 2024-02-26T10:39:44.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.225+0000 7efd85000640 1 --2- 172.21.15.67:0/1328073536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80150300 0x7efd800c2ff0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd70009d90 tx=0x7efd7002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6728124c52f3e006 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:44.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.225+0000 7efd7fe00640 1 -- 172.21.15.67:0/1328073536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70039070 con 0x7efd80150300 2024-02-26T10:39:44.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.225+0000 7efd7fe00640 1 -- 172.21.15.67:0/1328073536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7002cb40 con 0x7efd80150300 2024-02-26T10:39:44.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.225+0000 7efd7fe00640 1 -- 172.21.15.67:0/1328073536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70034440 con 0x7efd80150300 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.225+0000 7efd87a00640 1 -- 172.21.15.67:0/1328073536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80150300 msgr2=0x7efd800c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.225+0000 7efd87a00640 1 --2- 172.21.15.67:0/1328073536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80150300 0x7efd800c2ff0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd70009d90 tx=0x7efd7002bd90 comp rx=0 tx=0).stop 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.226+0000 7efd87a00640 1 -- 172.21.15.67:0/1328073536 shutdown_connections 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.226+0000 7efd87a00640 1 --2- 172.21.15.67:0/1328073536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80150300 0x7efd800c2ff0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.226+0000 7efd87a00640 1 -- 172.21.15.67:0/1328073536 >> 172.21.15.67:0/1328073536 conn(0x7efd800a84a0 msgr2=0x7efd800a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.226+0000 7efd87a00640 1 -- 172.21.15.67:0/1328073536 shutdown_connections 2024-02-26T10:39:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.226+0000 7efd87a00640 1 -- 172.21.15.67:0/1328073536 wait complete. 2024-02-26T10:39:44.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.227+0000 7efd87a00640 1 Processor -- start 2024-02-26T10:39:44.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 -- start start 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8006e410 0x7efd801ee790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80150300 0x7efd800c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 0x7efd80070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8006dd80 con 0x7efd80156230 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd80153a60 con 0x7efd8006e410 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.228+0000 7efd87a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800c3530 con 0x7efd80150300 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 0x7efd80070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 0x7efd80070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42176/0 (socket says 172.21.15.67:42176) 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 -- 172.21.15.67:0/1195646786 learned_addr learned my addr 172.21.15.67:0/1195646786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85000640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8006e410 0x7efd801ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd77e00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80150300 0x7efd800c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 -- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80150300 msgr2=0x7efd800c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80150300 0x7efd800c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 -- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8006e410 msgr2=0x7efd801ee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8006e410 0x7efd801ee790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85a00640 1 -- 172.21.15.67:0/1195646786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8014ae70 con 0x7efd80156230 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.229+0000 7efd85000640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8006e410 0x7efd801ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.230+0000 7efd85a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 0x7efd80070f10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c410 tx=0x7efd7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.230+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780074e0 con 0x7efd80156230 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.230+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd780047e0 con 0x7efd80156230 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.230+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78004bd0 con 0x7efd80156230 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.230+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8014b150 con 0x7efd80156230 2024-02-26T10:39:44.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.230+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd800c59f0 con 0x7efd80156230 2024-02-26T10:39:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.231+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd80149be0 con 0x7efd80156230 2024-02-26T10:39:44.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.232+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd78011070 con 0x7efd80156230 2024-02-26T10:39:44.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.233+0000 7efd7d600640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd5c06b500 0x7efd5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.233+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd78089d00 con 0x7efd80156230 2024-02-26T10:39:44.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.234+0000 7efd85000640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd5c06b500 0x7efd5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.235+0000 7efd85000640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd5c06b500 0x7efd5c06d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd70036000 tx=0x7efd70030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.238+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd78058c10 con 0x7efd80156230 2024-02-26T10:39:44.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.439+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd80074a30 con 0x7efd5c06b500 2024-02-26T10:39:44.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.459+0000 7efd7d600640 1 -- 172.21.15.67:0/1195646786 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7efd80074a30 con 0x7efd5c06b500 2024-02-26T10:39:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:39:44.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.465+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd5c06b500 msgr2=0x7efd5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.465+0000 7efd87a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd5c06b500 0x7efd5c06d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd70036000 tx=0x7efd70030000 comp rx=0 tx=0).stop 2024-02-26T10:39:44.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.465+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 msgr2=0x7efd80070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.465+0000 7efd87a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 0x7efd80070f10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c410 tx=0x7efd7800c8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 shutdown_connections 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd80156230 0x7efd80070f10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80150300 0x7efd800c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd5c06b500 0x7efd5c06d9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 --2- 172.21.15.67:0/1195646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8006e410 0x7efd801ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 >> 172.21.15.67:0/1195646786 conn(0x7efd800a84a0 msgr2=0x7efd800a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 shutdown_connections 2024-02-26T10:39:44.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.466+0000 7efd87a00640 1 -- 172.21.15.67:0/1195646786 wait complete. 2024-02-26T10:39:44.485 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-02-26T10:39:44.347028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220368,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754352,"statfs":{"total":773094113280,"available":772868456448,"internally_reserved":0,"allocated":4300800,"data_stored":2384844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001792"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314588+0000","last_change":"2024-02-26T10:39:40.026044+0000","last_active":"2024-02-26T10:39:41.314588+0000","last_peered":"2024-02-26T10:39:41.314588+0000","last_clean":"2024-02-26T10:39:41.314588+0000","last_became_active":"2024-02-26T10:39:40.025716+0000","last_became_peered":"2024-02-26T10:39:40.025716+0000","last_unstale":"2024-02-26T10:39:41.314588+0000","last_undegraded":"2024-02-26T10:39:41.314588+0000","last_fullsized":"2024-02-26T10:39:41.314588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119894+0000","last_change":"2024-02-26T10:39:40.000691+0000","last_active":"2024-02-26T10:39:41.119894+0000","last_peered":"2024-02-26T10:39:41.119894+0000","last_clean":"2024-02-26T10:39:41.119894+0000","last_became_active":"2024-02-26T10:39:40.000419+0000","last_became_peered":"2024-02-26T10:39:40.000419+0000","last_unstale":"2024-02-26T10:39:41.119894+0000","last_undegraded":"2024-02-26T10:39:41.119894+0000","last_fullsized":"2024-02-26T10:39:41.119894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314963+0000","last_change":"2024-02-26T10:39:40.017208+0000","last_active":"2024-02-26T10:39:41.314963+0000","last_peered":"2024-02-26T10:39:41.314963+0000","last_clean":"2024-02-26T10:39:41.314963+0000","last_became_active":"2024-02-26T10:39:40.016350+0000","last_became_peered":"2024-02-26T10:39:40.016350+0000","last_unstale":"2024-02-26T10:39:41.314963+0000","last_undegraded":"2024-02-26T10:39:41.314963+0000","last_fullsized":"2024-02-26T10:39:41.314963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.005464+0000","last_change":"2024-02-26T10:39:40.009578+0000","last_active":"2024-02-26T10:39:41.005464+0000","last_peered":"2024-02-26T10:39:41.005464+0000","last_clean":"2024-02-26T10:39:41.005464+0000","last_became_active":"2024-02-26T10:39:40.008770+0000","last_became_peered":"2024-02-26T10:39:40.008770+0000","last_unstale":"2024-02-26T10:39:41.005464+0000","last_undegraded":"2024-02-26T10:39:41.005464+0000","last_fullsized":"2024-02-26T10:39:41.005464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:48:42.311486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008177+0000","last_change":"2024-02-26T10:39:39.996527+0000","last_active":"2024-02-26T10:39:41.008177+0000","last_peered":"2024-02-26T10:39:41.008177+0000","last_clean":"2024-02-26T10:39:41.008177+0000","last_became_active":"2024-02-26T10:39:39.995687+0000","last_became_peered":"2024-02-26T10:39:39.995687+0000","last_unstale":"2024-02-26T10:39:41.008177+0000","last_undegraded":"2024-02-26T10:39:41.008177+0000","last_fullsized":"2024-02-26T10:39:41.008177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:29.964072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119730+0000","last_change":"2024-02-26T10:39:40.025212+0000","last_active":"2024-02-26T10:39:41.119730+0000","last_peered":"2024-02-26T10:39:41.119730+0000","last_clean":"2024-02-26T10:39:41.119730+0000","last_became_active":"2024-02-26T10:39:40.024642+0000","last_became_peered":"2024-02-26T10:39:40.024642+0000","last_unstale":"2024-02-26T10:39:41.119730+0000","last_undegraded":"2024-02-26T10:39:41.119730+0000","last_fullsized":"2024-02-26T10:39:41.119730+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217697+0000","last_change":"2024-02-26T10:39:40.014885+0000","last_active":"2024-02-26T10:39:41.217697+0000","last_peered":"2024-02-26T10:39:41.217697+0000","last_clean":"2024-02-26T10:39:41.217697+0000","last_became_active":"2024-02-26T10:39:40.014675+0000","last_became_peered":"2024-02-26T10:39:40.014675+0000","last_unstale":"2024-02-26T10:39:41.217697+0000","last_undegraded":"2024-02-26T10:39:41.217697+0000","last_fullsized":"2024-02-26T10:39:41.217697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217909+0000","last_change":"2024-02-26T10:39:40.005845+0000","last_active":"2024-02-26T10:39:41.217909+0000","last_peered":"2024-02-26T10:39:41.217909+0000","last_clean":"2024-02-26T10:39:41.217909+0000","last_became_active":"2024-02-26T10:39:40.005497+0000","last_became_peered":"2024-02-26T10:39:40.005497+0000","last_unstale":"2024-02-26T10:39:41.217909+0000","last_undegraded":"2024-02-26T10:39:41.217909+0000","last_fullsized":"2024-02-26T10:39:41.217909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011736+0000","last_change":"2024-02-26T10:39:40.006430+0000","last_active":"2024-02-26T10:39:41.011736+0000","last_peered":"2024-02-26T10:39:41.011736+0000","last_clean":"2024-02-26T10:39:41.011736+0000","last_became_active":"2024-02-26T10:39:40.004846+0000","last_became_peered":"2024-02-26T10:39:40.004846+0000","last_unstale":"2024-02-26T10:39:41.011736+0000","last_undegraded":"2024-02-26T10:39:41.011736+0000","last_fullsized":"2024-02-26T10:39:41.011736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006594+0000","last_change":"2024-02-26T10:39:39.995327+0000","last_active":"2024-02-26T10:39:41.006594+0000","last_peered":"2024-02-26T10:39:41.006594+0000","last_clean":"2024-02-26T10:39:41.006594+0000","last_became_active":"2024-02-26T10:39:39.994245+0000","last_became_peered":"2024-02-26T10:39:39.994245+0000","last_unstale":"2024-02-26T10:39:41.006594+0000","last_undegraded":"2024-02-26T10:39:41.006594+0000","last_fullsized":"2024-02-26T10:39:41.006594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217791+0000","last_change":"2024-02-26T10:39:40.017115+0000","last_active":"2024-02-26T10:39:41.217791+0000","last_peered":"2024-02-26T10:39:41.217791+0000","last_clean":"2024-02-26T10:39:41.217791+0000","last_became_active":"2024-02-26T10:39:40.016876+0000","last_became_peered":"2024-02-26T10:39:40.016876+0000","last_unstale":"2024-02-26T10:39:41.217791+0000","last_undegraded":"2024-02-26T10:39:41.217791+0000","last_fullsized":"2024-02-26T10:39:41.217791+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314981+0000","last_change":"2024-02-26T10:39:40.011833+0000","last_active":"2024-02-26T10:39:41.314981+0000","last_peered":"2024-02-26T10:39:41.314981+0000","last_clean":"2024-02-26T10:39:41.314981+0000","last_became_active":"2024-02-26T10:39:40.011581+0000","last_became_peered":"2024-02-26T10:39:40.011581+0000","last_unstale":"2024-02-26T10:39:41.314981+0000","last_undegraded":"2024-02-26T10:39:41.314981+0000","last_fullsized":"2024-02-26T10:39:41.314981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.009182+0000","last_change":"2024-02-26T10:39:40.000902+0000","last_active":"2024-02-26T10:39:41.009182+0000","last_peered":"2024-02-26T10:39:41.009182+0000","last_clean":"2024-02-26T10:39:41.009182+0000","last_became_active":"2024-02-26T10:39:39.999441+0000","last_became_peered":"2024-02-26T10:39:39.999441+0000","last_unstale":"2024-02-26T10:39:41.009182+0000","last_undegraded":"2024-02-26T10:39:41.009182+0000","last_fullsized":"2024-02-26T10:39:41.009182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:27.937806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120132+0000","last_change":"2024-02-26T10:39:39.999249+0000","last_active":"2024-02-26T10:39:41.120132+0000","last_peered":"2024-02-26T10:39:41.120132+0000","last_clean":"2024-02-26T10:39:41.120132+0000","last_became_active":"2024-02-26T10:39:39.998319+0000","last_became_peered":"2024-02-26T10:39:39.998319+0000","last_unstale":"2024-02-26T10:39:41.120132+0000","last_undegraded":"2024-02-26T10:39:41.120132+0000","last_fullsized":"2024-02-26T10:39:41.120132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:29.312334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315682+0000","last_change":"2024-02-26T10:39:40.024788+0000","last_active":"2024-02-26T10:39:41.315682+0000","last_peered":"2024-02-26T10:39:41.315682+0000","last_clean":"2024-02-26T10:39:41.315682+0000","last_became_active":"2024-02-26T10:39:40.024454+0000","last_became_peered":"2024-02-26T10:39:40.024454+0000","last_unstale":"2024-02-26T10:39:41.315682+0000","last_undegraded":"2024-02-26T10:39:41.315682+0000","last_fullsized":"2024-02-26T10:39:41.315682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.218187+0000","last_change":"2024-02-26T10:39:40.011104+0000","last_active":"2024-02-26T10:39:41.218187+0000","last_peered":"2024-02-26T10:39:41.218187+0000","last_clean":"2024-02-26T10:39:41.218187+0000","last_became_active":"2024-02-26T10:39:40.010774+0000","last_became_peered":"2024-02-26T10:39:40.010774+0000","last_unstale":"2024-02-26T10:39:41.218187+0000","last_undegraded":"2024-02-26T10:39:41.218187+0000","last_fullsized":"2024-02-26T10:39:41.218187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.218219+0000","last_change":"2024-02-26T10:39:40.017107+0000","last_active":"2024-02-26T10:39:41.218219+0000","last_peered":"2024-02-26T10:39:41.218219+0000","last_clean":"2024-02-26T10:39:41.218219+0000","last_became_active":"2024-02-26T10:39:40.016813+0000","last_became_peered":"2024-02-26T10:39:40.016813+0000","last_unstale":"2024-02-26T10:39:41.218219+0000","last_undegraded":"2024-02-26T10:39:41.218219+0000","last_fullsized":"2024-02-26T10:39:41.218219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120231+0000","last_change":"2024-02-26T10:39:39.998011+0000","last_active":"2024-02-26T10:39:41.120231+0000","last_peered":"2024-02-26T10:39:41.120231+0000","last_clean":"2024-02-26T10:39:41.120231+0000","last_became_active":"2024-02-26T10:39:39.997725+0000","last_became_peered":"2024-02-26T10:39:39.997725+0000","last_unstale":"2024-02-26T10:39:41.120231+0000","last_undegraded":"2024-02-26T10:39:41.120231+0000","last_fullsized":"2024-02-26T10:39:41.120231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:02.449804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119875+0000","last_change":"2024-02-26T10:39:39.996782+0000","last_active":"2024-02-26T10:39:41.119875+0000","last_peered":"2024-02-26T10:39:41.119875+0000","last_clean":"2024-02-26T10:39:41.119875+0000","last_became_active":"2024-02-26T10:39:39.996469+0000","last_became_peered":"2024-02-26T10:39:39.996469+0000","last_unstale":"2024-02-26T10:39:41.119875+0000","last_undegraded":"2024-02-26T10:39:41.119875+0000","last_fullsized":"2024-02-26T10:39:41.119875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008413+0000","last_change":"2024-02-26T10:39:40.028123+0000","last_active":"2024-02-26T10:39:41.008413+0000","last_peered":"2024-02-26T10:39:41.008413+0000","last_clean":"2024-02-26T10:39:41.008413+0000","last_became_active":"2024-02-26T10:39:40.027299+0000","last_became_peered":"2024-02-26T10:39:40.027299+0000","last_unstale":"2024-02-26T10:39:41.008413+0000","last_undegraded":"2024-02-26T10:39:41.008413+0000","last_fullsized":"2024-02-26T10:39:41.008413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:54:29.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010612+0000","last_change":"2024-02-26T10:39:40.003035+0000","last_active":"2024-02-26T10:39:41.010612+0000","last_peered":"2024-02-26T10:39:41.010612+0000","last_clean":"2024-02-26T10:39:41.010612+0000","last_became_active":"2024-02-26T10:39:40.002724+0000","last_became_peered":"2024-02-26T10:39:40.002724+0000","last_unstale":"2024-02-26T10:39:41.010612+0000","last_undegraded":"2024-02-26T10:39:41.010612+0000","last_fullsized":"2024-02-26T10:39:41.010612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010619+0000","last_change":"2024-02-26T10:39:39.990872+0000","last_active":"2024-02-26T10:39:41.010619+0000","last_peered":"2024-02-26T10:39:41.010619+0000","last_clean":"2024-02-26T10:39:41.010619+0000","last_became_active":"2024-02-26T10:39:39.990134+0000","last_became_peered":"2024-02-26T10:39:39.990134+0000","last_unstale":"2024-02-26T10:39:41.010619+0000","last_undegraded":"2024-02-26T10:39:41.010619+0000","last_fullsized":"2024-02-26T10:39:41.010619+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.024966+0000","last_change":"2024-02-26T10:39:40.022805+0000","last_active":"2024-02-26T10:39:41.024966+0000","last_peered":"2024-02-26T10:39:41.024966+0000","last_clean":"2024-02-26T10:39:41.024966+0000","last_became_active":"2024-02-26T10:39:40.022510+0000","last_became_peered":"2024-02-26T10:39:40.022510+0000","last_unstale":"2024-02-26T10:39:41.024966+0000","last_undegraded":"2024-02-26T10:39:41.024966+0000","last_fullsized":"2024-02-26T10:39:41.024966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314008+0000","last_change":"2024-02-26T10:39:40.017086+0000","last_active":"2024-02-26T10:39:41.314008+0000","last_peered":"2024-02-26T10:39:41.314008+0000","last_clean":"2024-02-26T10:39:41.314008+0000","last_became_active":"2024-02-26T10:39:40.016779+0000","last_became_peered":"2024-02-26T10:39:40.016779+0000","last_unstale":"2024-02-26T10:39:41.314008+0000","last_undegraded":"2024-02-26T10:39:41.314008+0000","last_fullsized":"2024-02-26T10:39:41.314008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006656+0000","last_change":"2024-02-26T10:39:40.015014+0000","last_active":"2024-02-26T10:39:41.006656+0000","last_peered":"2024-02-26T10:39:41.006656+0000","last_clean":"2024-02-26T10:39:41.006656+0000","last_became_active":"2024-02-26T10:39:40.014611+0000","last_became_peered":"2024-02-26T10:39:40.014611+0000","last_unstale":"2024-02-26T10:39:41.006656+0000","last_undegraded":"2024-02-26T10:39:41.006656+0000","last_fullsized":"2024-02-26T10:39:41.006656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120473+0000","last_change":"2024-02-26T10:39:40.007272+0000","last_active":"2024-02-26T10:39:41.120473+0000","last_peered":"2024-02-26T10:39:41.120473+0000","last_clean":"2024-02-26T10:39:41.120473+0000","last_became_active":"2024-02-26T10:39:40.006923+0000","last_became_peered":"2024-02-26T10:39:40.006923+0000","last_unstale":"2024-02-26T10:39:41.120473+0000","last_undegraded":"2024-02-26T10:39:41.120473+0000","last_fullsized":"2024-02-26T10:39:41.120473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:21.270698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006336+0000","last_change":"2024-02-26T10:39:40.008980+0000","last_active":"2024-02-26T10:39:41.006336+0000","last_peered":"2024-02-26T10:39:41.006336+0000","last_clean":"2024-02-26T10:39:41.006336+0000","last_became_active":"2024-02-26T10:39:40.008531+0000","last_became_peered":"2024-02-26T10:39:40.008531+0000","last_unstale":"2024-02-26T10:39:41.006336+0000","last_undegraded":"2024-02-26T10:39:41.006336+0000","last_fullsized":"2024-02-26T10:39:41.006336+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:11:31.399212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315449+0000","last_change":"2024-02-26T10:39:40.019372+0000","last_active":"2024-02-26T10:39:41.315449+0000","last_peered":"2024-02-26T10:39:41.315449+0000","last_clean":"2024-02-26T10:39:41.315449+0000","last_became_active":"2024-02-26T10:39:40.019025+0000","last_became_peered":"2024-02-26T10:39:40.019025+0000","last_unstale":"2024-02-26T10:39:41.315449+0000","last_undegraded":"2024-02-26T10:39:41.315449+0000","last_fullsized":"2024-02-26T10:39:41.315449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011011+0000","last_change":"2024-02-26T10:39:39.994088+0000","last_active":"2024-02-26T10:39:41.011011+0000","last_peered":"2024-02-26T10:39:41.011011+0000","last_clean":"2024-02-26T10:39:41.011011+0000","last_became_active":"2024-02-26T10:39:39.993717+0000","last_became_peered":"2024-02-26T10:39:39.993717+0000","last_unstale":"2024-02-26T10:39:41.011011+0000","last_undegraded":"2024-02-26T10:39:41.011011+0000","last_fullsized":"2024-02-26T10:39:41.011011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217331+0000","last_change":"2024-02-26T10:39:40.010029+0000","last_active":"2024-02-26T10:39:41.217331+0000","last_peered":"2024-02-26T10:39:41.217331+0000","last_clean":"2024-02-26T10:39:41.217331+0000","last_became_active":"2024-02-26T10:39:40.009745+0000","last_became_peered":"2024-02-26T10:39:40.009745+0000","last_unstale":"2024-02-26T10:39:41.217331+0000","last_undegraded":"2024-02-26T10:39:41.217331+0000","last_fullsized":"2024-02-26T10:39:41.217331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314375+0000","last_change":"2024-02-26T10:39:40.008658+0000","last_active":"2024-02-26T10:39:41.314375+0000","last_peered":"2024-02-26T10:39:41.314375+0000","last_clean":"2024-02-26T10:39:41.314375+0000","last_became_active":"2024-02-26T10:39:40.008354+0000","last_became_peered":"2024-02-26T10:39:40.008354+0000","last_unstale":"2024-02-26T10:39:41.314375+0000","last_undegraded":"2024-02-26T10:39:41.314375+0000","last_fullsized":"2024-02-26T10:39:41.314375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217161+0000","last_change":"2024-02-26T10:39:40.013245+0000","last_active":"2024-02-26T10:39:41.217161+0000","last_peered":"2024-02-26T10:39:41.217161+0000","last_clean":"2024-02-26T10:39:41.217161+0000","last_became_active":"2024-02-26T10:39:40.008999+0000","last_became_peered":"2024-02-26T10:39:40.008999+0000","last_unstale":"2024-02-26T10:39:41.217161+0000","last_undegraded":"2024-02-26T10:39:41.217161+0000","last_fullsized":"2024-02-26T10:39:41.217161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025402+0000","last_change":"2024-02-26T10:39:40.021369+0000","last_active":"2024-02-26T10:39:41.025402+0000","last_peered":"2024-02-26T10:39:41.025402+0000","last_clean":"2024-02-26T10:39:41.025402+0000","last_became_active":"2024-02-26T10:39:40.021061+0000","last_became_peered":"2024-02-26T10:39:40.021061+0000","last_unstale":"2024-02-26T10:39:41.025402+0000","last_undegraded":"2024-02-26T10:39:41.025402+0000","last_fullsized":"2024-02-26T10:39:41.025402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315150+0000","last_change":"2024-02-26T10:39:39.014072+0000","last_active":"2024-02-26T10:39:41.315150+0000","last_peered":"2024-02-26T10:39:41.315150+0000","last_clean":"2024-02-26T10:39:41.315150+0000","last_became_active":"2024-02-26T10:39:39.013810+0000","last_became_peered":"2024-02-26T10:39:39.013810+0000","last_unstale":"2024-02-26T10:39:41.315150+0000","last_undegraded":"2024-02-26T10:39:41.315150+0000","last_fullsized":"2024-02-26T10:39:41.315150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:12:03.804658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314975+0000","last_change":"2024-02-26T10:39:40.011479+0000","last_active":"2024-02-26T10:39:41.314975+0000","last_peered":"2024-02-26T10:39:41.314975+0000","last_clean":"2024-02-26T10:39:41.314975+0000","last_became_active":"2024-02-26T10:39:40.011123+0000","last_became_peered":"2024-02-26T10:39:40.011123+0000","last_unstale":"2024-02-26T10:39:41.314975+0000","last_undegraded":"2024-02-26T10:39:41.314975+0000","last_fullsized":"2024-02-26T10:39:41.314975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025487+0000","last_change":"2024-02-26T10:39:39.013171+0000","last_active":"2024-02-26T10:39:41.025487+0000","last_peered":"2024-02-26T10:39:41.025487+0000","last_clean":"2024-02-26T10:39:41.025487+0000","last_became_active":"2024-02-26T10:39:39.012785+0000","last_became_peered":"2024-02-26T10:39:39.012785+0000","last_unstale":"2024-02-26T10:39:41.025487+0000","last_undegraded":"2024-02-26T10:39:41.025487+0000","last_fullsized":"2024-02-26T10:39:41.025487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:04:56.455641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011659+0000","last_change":"2024-02-26T10:39:39.992637+0000","last_active":"2024-02-26T10:39:41.011659+0000","last_peered":"2024-02-26T10:39:41.011659+0000","last_clean":"2024-02-26T10:39:41.011659+0000","last_became_active":"2024-02-26T10:39:39.991960+0000","last_became_peered":"2024-02-26T10:39:39.991960+0000","last_unstale":"2024-02-26T10:39:41.011659+0000","last_undegraded":"2024-02-26T10:39:41.011659+0000","last_fullsized":"2024-02-26T10:39:41.011659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119568+0000","last_change":"2024-02-26T10:39:39.000798+0000","last_active":"2024-02-26T10:39:41.119568+0000","last_peered":"2024-02-26T10:39:41.119568+0000","last_clean":"2024-02-26T10:39:41.119568+0000","last_became_active":"2024-02-26T10:39:38.998438+0000","last_became_peered":"2024-02-26T10:39:38.998438+0000","last_unstale":"2024-02-26T10:39:41.119568+0000","last_undegraded":"2024-02-26T10:39:41.119568+0000","last_fullsized":"2024-02-26T10:39:41.119568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:29:40.306221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006631+0000","last_change":"2024-02-26T10:39:40.008958+0000","last_active":"2024-02-26T10:39:41.006631+0000","last_peered":"2024-02-26T10:39:41.006631+0000","last_clean":"2024-02-26T10:39:41.006631+0000","last_became_active":"2024-02-26T10:39:40.008566+0000","last_became_peered":"2024-02-26T10:39:40.008566+0000","last_unstale":"2024-02-26T10:39:41.006631+0000","last_undegraded":"2024-02-26T10:39:41.006631+0000","last_fullsized":"2024-02-26T10:39:41.006631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008316+0000","last_change":"2024-02-26T10:39:38.993592+0000","last_active":"2024-02-26T10:39:41.008316+0000","last_peered":"2024-02-26T10:39:41.008316+0000","last_clean":"2024-02-26T10:39:41.008316+0000","last_became_active":"2024-02-26T10:39:38.993042+0000","last_became_peered":"2024-02-26T10:39:38.993042+0000","last_unstale":"2024-02-26T10:39:41.008316+0000","last_undegraded":"2024-02-26T10:39:41.008316+0000","last_fullsized":"2024-02-26T10:39:41.008316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:06:10.297346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217504+0000","last_change":"2024-02-26T10:39:40.017365+0000","last_active":"2024-02-26T10:39:41.217504+0000","last_peered":"2024-02-26T10:39:41.217504+0000","last_clean":"2024-02-26T10:39:41.217504+0000","last_became_active":"2024-02-26T10:39:40.010336+0000","last_became_peered":"2024-02-26T10:39:40.010336+0000","last_unstale":"2024-02-26T10:39:41.217504+0000","last_undegraded":"2024-02-26T10:39:41.217504+0000","last_fullsized":"2024-02-26T10:39:41.217504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010790+0000","last_change":"2024-02-26T10:39:39.005253+0000","last_active":"2024-02-26T10:39:41.010790+0000","last_peered":"2024-02-26T10:39:41.010790+0000","last_clean":"2024-02-26T10:39:41.010790+0000","last_became_active":"2024-02-26T10:39:39.004947+0000","last_became_peered":"2024-02-26T10:39:39.004947+0000","last_unstale":"2024-02-26T10:39:41.010790+0000","last_undegraded":"2024-02-26T10:39:41.010790+0000","last_fullsized":"2024-02-26T10:39:41.010790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:19:13.361650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314788+0000","last_change":"2024-02-26T10:39:40.024744+0000","last_active":"2024-02-26T10:39:41.314788+0000","last_peered":"2024-02-26T10:39:41.314788+0000","last_clean":"2024-02-26T10:39:41.314788+0000","last_became_active":"2024-02-26T10:39:40.023928+0000","last_became_peered":"2024-02-26T10:39:40.023928+0000","last_unstale":"2024-02-26T10:39:41.314788+0000","last_undegraded":"2024-02-26T10:39:41.314788+0000","last_fullsized":"2024-02-26T10:39:41.314788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010358+0000","last_change":"2024-02-26T10:39:39.007073+0000","last_active":"2024-02-26T10:39:41.010358+0000","last_peered":"2024-02-26T10:39:41.010358+0000","last_clean":"2024-02-26T10:39:41.010358+0000","last_became_active":"2024-02-26T10:39:39.004616+0000","last_became_peered":"2024-02-26T10:39:39.004616+0000","last_unstale":"2024-02-26T10:39:41.010358+0000","last_undegraded":"2024-02-26T10:39:41.010358+0000","last_fullsized":"2024-02-26T10:39:41.010358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:07.037054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010170+0000","last_change":"2024-02-26T10:39:40.014474+0000","last_active":"2024-02-26T10:39:41.010170+0000","last_peered":"2024-02-26T10:39:41.010170+0000","last_clean":"2024-02-26T10:39:41.010170+0000","last_became_active":"2024-02-26T10:39:40.014129+0000","last_became_peered":"2024-02-26T10:39:40.014129+0000","last_unstale":"2024-02-26T10:39:41.010170+0000","last_undegraded":"2024-02-26T10:39:41.010170+0000","last_fullsized":"2024-02-26T10:39:41.010170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:55.255246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.118850+0000","last_change":"2024-02-26T10:39:38.998681+0000","last_active":"2024-02-26T10:39:41.118850+0000","last_peered":"2024-02-26T10:39:41.118850+0000","last_clean":"2024-02-26T10:39:41.118850+0000","last_became_active":"2024-02-26T10:39:38.998355+0000","last_became_peered":"2024-02-26T10:39:38.998355+0000","last_unstale":"2024-02-26T10:39:41.118850+0000","last_undegraded":"2024-02-26T10:39:41.118850+0000","last_fullsized":"2024-02-26T10:39:41.118850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:54.191021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004524+0000","last_change":"2024-02-26T10:39:40.025109+0000","last_active":"2024-02-26T10:39:41.004524+0000","last_peered":"2024-02-26T10:39:41.004524+0000","last_clean":"2024-02-26T10:39:41.004524+0000","last_became_active":"2024-02-26T10:39:40.024213+0000","last_became_peered":"2024-02-26T10:39:40.024213+0000","last_unstale":"2024-02-26T10:39:41.004524+0000","last_undegraded":"2024-02-26T10:39:41.004524+0000","last_fullsized":"2024-02-26T10:39:41.004524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:48.049981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010050+0000","last_change":"2024-02-26T10:39:39.003513+0000","last_active":"2024-02-26T10:39:41.010050+0000","last_peered":"2024-02-26T10:39:41.010050+0000","last_clean":"2024-02-26T10:39:41.010050+0000","last_became_active":"2024-02-26T10:39:39.002989+0000","last_became_peered":"2024-02-26T10:39:39.002989+0000","last_unstale":"2024-02-26T10:39:41.010050+0000","last_undegraded":"2024-02-26T10:39:41.010050+0000","last_fullsized":"2024-02-26T10:39:41.010050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:13.624048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.316081+0000","last_change":"2024-02-26T10:39:40.025178+0000","last_active":"2024-02-26T10:39:41.316081+0000","last_peered":"2024-02-26T10:39:41.316081+0000","last_clean":"2024-02-26T10:39:41.316081+0000","last_became_active":"2024-02-26T10:39:40.024856+0000","last_became_peered":"2024-02-26T10:39:40.024856+0000","last_unstale":"2024-02-26T10:39:41.316081+0000","last_undegraded":"2024-02-26T10:39:41.316081+0000","last_fullsized":"2024-02-26T10:39:41.316081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120431+0000","last_change":"2024-02-26T10:39:39.008501+0000","last_active":"2024-02-26T10:39:41.120431+0000","last_peered":"2024-02-26T10:39:41.120431+0000","last_clean":"2024-02-26T10:39:41.120431+0000","last_became_active":"2024-02-26T10:39:39.008223+0000","last_became_peered":"2024-02-26T10:39:39.008223+0000","last_unstale":"2024-02-26T10:39:41.120431+0000","last_undegraded":"2024-02-26T10:39:41.120431+0000","last_fullsized":"2024-02-26T10:39:41.120431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:48.232676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120521+0000","last_change":"2024-02-26T10:39:40.006041+0000","last_active":"2024-02-26T10:39:41.120521+0000","last_peered":"2024-02-26T10:39:41.120521+0000","last_clean":"2024-02-26T10:39:41.120521+0000","last_became_active":"2024-02-26T10:39:40.002138+0000","last_became_peered":"2024-02-26T10:39:40.002138+0000","last_unstale":"2024-02-26T10:39:41.120521+0000","last_undegraded":"2024-02-26T10:39:41.120521+0000","last_fullsized":"2024-02-26T10:39:41.120521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:38.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025749+0000","last_change":"2024-02-26T10:39:39.004487+0000","last_active":"2024-02-26T10:39:41.025749+0000","last_peered":"2024-02-26T10:39:41.025749+0000","last_clean":"2024-02-26T10:39:41.025749+0000","last_became_active":"2024-02-26T10:39:39.004135+0000","last_became_peered":"2024-02-26T10:39:39.004135+0000","last_unstale":"2024-02-26T10:39:41.025749+0000","last_undegraded":"2024-02-26T10:39:41.025749+0000","last_fullsized":"2024-02-26T10:39:41.025749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120039+0000","last_change":"2024-02-26T10:39:40.006537+0000","last_active":"2024-02-26T10:39:41.120039+0000","last_peered":"2024-02-26T10:39:41.120039+0000","last_clean":"2024-02-26T10:39:41.120039+0000","last_became_active":"2024-02-26T10:39:40.006231+0000","last_became_peered":"2024-02-26T10:39:40.006231+0000","last_unstale":"2024-02-26T10:39:41.120039+0000","last_undegraded":"2024-02-26T10:39:41.120039+0000","last_fullsized":"2024-02-26T10:39:41.120039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315112+0000","last_change":"2024-02-26T10:39:39.010001+0000","last_active":"2024-02-26T10:39:41.315112+0000","last_peered":"2024-02-26T10:39:41.315112+0000","last_clean":"2024-02-26T10:39:41.315112+0000","last_became_active":"2024-02-26T10:39:39.009754+0000","last_became_peered":"2024-02-26T10:39:39.009754+0000","last_unstale":"2024-02-26T10:39:41.315112+0000","last_undegraded":"2024-02-26T10:39:41.315112+0000","last_fullsized":"2024-02-26T10:39:41.315112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314824+0000","last_change":"2024-02-26T10:39:40.009580+0000","last_active":"2024-02-26T10:39:41.314824+0000","last_peered":"2024-02-26T10:39:41.314824+0000","last_clean":"2024-02-26T10:39:41.314824+0000","last_became_active":"2024-02-26T10:39:40.009277+0000","last_became_peered":"2024-02-26T10:39:40.009277+0000","last_unstale":"2024-02-26T10:39:41.314824+0000","last_undegraded":"2024-02-26T10:39:41.314824+0000","last_fullsized":"2024-02-26T10:39:41.314824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119722+0000","last_change":"2024-02-26T10:39:38.999382+0000","last_active":"2024-02-26T10:39:41.119722+0000","last_peered":"2024-02-26T10:39:41.119722+0000","last_clean":"2024-02-26T10:39:41.119722+0000","last_became_active":"2024-02-26T10:39:38.999045+0000","last_became_peered":"2024-02-26T10:39:38.999045+0000","last_unstale":"2024-02-26T10:39:41.119722+0000","last_undegraded":"2024-02-26T10:39:41.119722+0000","last_fullsized":"2024-02-26T10:39:41.119722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:36.656258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004891+0000","last_change":"2024-02-26T10:39:40.018996+0000","last_active":"2024-02-26T10:39:41.004891+0000","last_peered":"2024-02-26T10:39:41.004891+0000","last_clean":"2024-02-26T10:39:41.004891+0000","last_became_active":"2024-02-26T10:39:40.018734+0000","last_became_peered":"2024-02-26T10:39:40.018734+0000","last_unstale":"2024-02-26T10:39:41.004891+0000","last_undegraded":"2024-02-26T10:39:41.004891+0000","last_fullsized":"2024-02-26T10:39:41.004891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:26.470295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011084+0000","last_change":"2024-02-26T10:39:39.010735+0000","last_active":"2024-02-26T10:39:41.011084+0000","last_peered":"2024-02-26T10:39:41.011084+0000","last_clean":"2024-02-26T10:39:41.011084+0000","last_became_active":"2024-02-26T10:39:39.010394+0000","last_became_peered":"2024-02-26T10:39:39.010394+0000","last_unstale":"2024-02-26T10:39:41.011084+0000","last_undegraded":"2024-02-26T10:39:41.011084+0000","last_fullsized":"2024-02-26T10:39:41.011084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:41:16.074102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119370+0000","last_change":"2024-02-26T10:39:40.006750+0000","last_active":"2024-02-26T10:39:41.119370+0000","last_peered":"2024-02-26T10:39:41.119370+0000","last_clean":"2024-02-26T10:39:41.119370+0000","last_became_active":"2024-02-26T10:39:40.005913+0000","last_became_peered":"2024-02-26T10:39:40.005913+0000","last_unstale":"2024-02-26T10:39:41.119370+0000","last_undegraded":"2024-02-26T10:39:41.119370+0000","last_fullsized":"2024-02-26T10:39:41.119370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004553+0000","last_change":"2024-02-26T10:39:39.000729+0000","last_active":"2024-02-26T10:39:41.004553+0000","last_peered":"2024-02-26T10:39:41.004553+0000","last_clean":"2024-02-26T10:39:41.004553+0000","last_became_active":"2024-02-26T10:39:38.999811+0000","last_became_peered":"2024-02-26T10:39:38.999811+0000","last_unstale":"2024-02-26T10:39:41.004553+0000","last_undegraded":"2024-02-26T10:39:41.004553+0000","last_fullsized":"2024-02-26T10:39:41.004553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217626+0000","last_change":"2024-02-26T10:39:40.008946+0000","last_active":"2024-02-26T10:39:41.217626+0000","last_peered":"2024-02-26T10:39:41.217626+0000","last_clean":"2024-02-26T10:39:41.217626+0000","last_became_active":"2024-02-26T10:39:40.008676+0000","last_became_peered":"2024-02-26T10:39:40.008676+0000","last_unstale":"2024-02-26T10:39:41.217626+0000","last_undegraded":"2024-02-26T10:39:41.217626+0000","last_fullsized":"2024-02-26T10:39:41.217626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119048+0000","last_change":"2024-02-26T10:39:39.008406+0000","last_active":"2024-02-26T10:39:41.119048+0000","last_peered":"2024-02-26T10:39:41.119048+0000","last_clean":"2024-02-26T10:39:41.119048+0000","last_became_active":"2024-02-26T10:39:39.008086+0000","last_became_peered":"2024-02-26T10:39:39.008086+0000","last_unstale":"2024-02-26T10:39:41.119048+0000","last_undegraded":"2024-02-26T10:39:41.119048+0000","last_fullsized":"2024-02-26T10:39:41.119048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314267+0000","last_change":"2024-02-26T10:39:40.008604+0000","last_active":"2024-02-26T10:39:41.314267+0000","last_peered":"2024-02-26T10:39:41.314267+0000","last_clean":"2024-02-26T10:39:41.314267+0000","last_became_active":"2024-02-26T10:39:40.008244+0000","last_became_peered":"2024-02-26T10:39:40.008244+0000","last_unstale":"2024-02-26T10:39:41.314267+0000","last_undegraded":"2024-02-26T10:39:41.314267+0000","last_fullsized":"2024-02-26T10:39:41.314267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119002+0000","last_change":"2024-02-26T10:39:39.002498+0000","last_active":"2024-02-26T10:39:41.119002+0000","last_peered":"2024-02-26T10:39:41.119002+0000","last_clean":"2024-02-26T10:39:41.119002+0000","last_became_active":"2024-02-26T10:39:39.002185+0000","last_became_peered":"2024-02-26T10:39:39.002185+0000","last_unstale":"2024-02-26T10:39:41.119002+0000","last_undegraded":"2024-02-26T10:39:41.119002+0000","last_fullsized":"2024-02-26T10:39:41.119002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:18.706456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011518+0000","last_change":"2024-02-26T10:39:40.026290+0000","last_active":"2024-02-26T10:39:41.011518+0000","last_peered":"2024-02-26T10:39:41.011518+0000","last_clean":"2024-02-26T10:39:41.011518+0000","last_became_active":"2024-02-26T10:39:40.026004+0000","last_became_peered":"2024-02-26T10:39:40.026004+0000","last_unstale":"2024-02-26T10:39:41.011518+0000","last_undegraded":"2024-02-26T10:39:41.011518+0000","last_fullsized":"2024-02-26T10:39:41.011518+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006717+0000","last_change":"2024-02-26T10:39:39.001118+0000","last_active":"2024-02-26T10:39:41.006717+0000","last_peered":"2024-02-26T10:39:41.006717+0000","last_clean":"2024-02-26T10:39:41.006717+0000","last_became_active":"2024-02-26T10:39:39.000747+0000","last_became_peered":"2024-02-26T10:39:39.000747+0000","last_unstale":"2024-02-26T10:39:41.006717+0000","last_undegraded":"2024-02-26T10:39:41.006717+0000","last_fullsized":"2024-02-26T10:39:41.006717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:54.239568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315538+0000","last_change":"2024-02-26T10:39:40.017294+0000","last_active":"2024-02-26T10:39:41.315538+0000","last_peered":"2024-02-26T10:39:41.315538+0000","last_clean":"2024-02-26T10:39:41.315538+0000","last_became_active":"2024-02-26T10:39:40.016268+0000","last_became_peered":"2024-02-26T10:39:40.016268+0000","last_unstale":"2024-02-26T10:39:41.315538+0000","last_undegraded":"2024-02-26T10:39:41.315538+0000","last_fullsized":"2024-02-26T10:39:41.315538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011287+0000","last_change":"2024-02-26T10:39:39.008983+0000","last_active":"2024-02-26T10:39:41.011287+0000","last_peered":"2024-02-26T10:39:41.011287+0000","last_clean":"2024-02-26T10:39:41.011287+0000","last_became_active":"2024-02-26T10:39:39.008687+0000","last_became_peered":"2024-02-26T10:39:39.008687+0000","last_unstale":"2024-02-26T10:39:41.011287+0000","last_undegraded":"2024-02-26T10:39:41.011287+0000","last_fullsized":"2024-02-26T10:39:41.011287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:08:52.607215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.005393+0000","last_change":"2024-02-26T10:39:40.005719+0000","last_active":"2024-02-26T10:39:41.005393+0000","last_peered":"2024-02-26T10:39:41.005393+0000","last_clean":"2024-02-26T10:39:41.005393+0000","last_became_active":"2024-02-26T10:39:40.005399+0000","last_became_peered":"2024-02-26T10:39:40.005399+0000","last_unstale":"2024-02-26T10:39:41.005393+0000","last_undegraded":"2024-02-26T10:39:41.005393+0000","last_fullsized":"2024-02-26T10:39:41.005393+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:40.993108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011230+0000","last_change":"2024-02-26T10:39:38.997283+0000","last_active":"2024-02-26T10:39:41.011230+0000","last_peered":"2024-02-26T10:39:41.011230+0000","last_clean":"2024-02-26T10:39:41.011230+0000","last_became_active":"2024-02-26T10:39:38.996974+0000","last_became_peered":"2024-02-26T10:39:38.996974+0000","last_unstale":"2024-02-26T10:39:41.011230+0000","last_undegraded":"2024-02-26T10:39:41.011230+0000","last_fullsized":"2024-02-26T10:39:41.011230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:48.227511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008102+0000","last_change":"2024-02-26T10:39:40.014954+0000","last_active":"2024-02-26T10:39:41.008102+0000","last_peered":"2024-02-26T10:39:41.008102+0000","last_clean":"2024-02-26T10:39:41.008102+0000","last_became_active":"2024-02-26T10:39:40.014657+0000","last_became_peered":"2024-02-26T10:39:40.014657+0000","last_unstale":"2024-02-26T10:39:41.008102+0000","last_undegraded":"2024-02-26T10:39:41.008102+0000","last_fullsized":"2024-02-26T10:39:41.008102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:38:13.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011102+0000","last_change":"2024-02-26T10:39:39.005544+0000","last_active":"2024-02-26T10:39:41.011102+0000","last_peered":"2024-02-26T10:39:41.011102+0000","last_clean":"2024-02-26T10:39:41.011102+0000","last_became_active":"2024-02-26T10:39:39.005194+0000","last_became_peered":"2024-02-26T10:39:39.005194+0000","last_unstale":"2024-02-26T10:39:41.011102+0000","last_undegraded":"2024-02-26T10:39:41.011102+0000","last_fullsized":"2024-02-26T10:39:41.011102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:51:47.714809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008058+0000","last_change":"2024-02-26T10:39:39.997097+0000","last_active":"2024-02-26T10:39:41.008058+0000","last_peered":"2024-02-26T10:39:41.008058+0000","last_clean":"2024-02-26T10:39:41.008058+0000","last_became_active":"2024-02-26T10:39:39.996720+0000","last_became_peered":"2024-02-26T10:39:39.996720+0000","last_unstale":"2024-02-26T10:39:41.008058+0000","last_undegraded":"2024-02-26T10:39:41.008058+0000","last_fullsized":"2024-02-26T10:39:41.008058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:12.318898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025244+0000","last_change":"2024-02-26T10:39:38.992320+0000","last_active":"2024-02-26T10:39:41.025244+0000","last_peered":"2024-02-26T10:39:41.025244+0000","last_clean":"2024-02-26T10:39:41.025244+0000","last_became_active":"2024-02-26T10:39:38.991970+0000","last_became_peered":"2024-02-26T10:39:38.991970+0000","last_unstale":"2024-02-26T10:39:41.025244+0000","last_undegraded":"2024-02-26T10:39:41.025244+0000","last_fullsized":"2024-02-26T10:39:41.025244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008634+0000","last_change":"2024-02-26T10:39:40.002307+0000","last_active":"2024-02-26T10:39:41.008634+0000","last_peered":"2024-02-26T10:39:41.008634+0000","last_clean":"2024-02-26T10:39:41.008634+0000","last_became_active":"2024-02-26T10:39:40.001899+0000","last_became_peered":"2024-02-26T10:39:40.001899+0000","last_unstale":"2024-02-26T10:39:41.008634+0000","last_undegraded":"2024-02-26T10:39:41.008634+0000","last_fullsized":"2024-02-26T10:39:41.008634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:23.314983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314280+0000","last_change":"2024-02-26T10:39:39.011100+0000","last_active":"2024-02-26T10:39:41.314280+0000","last_peered":"2024-02-26T10:39:41.314280+0000","last_clean":"2024-02-26T10:39:41.314280+0000","last_became_active":"2024-02-26T10:39:39.010704+0000","last_became_peered":"2024-02-26T10:39:39.010704+0000","last_unstale":"2024-02-26T10:39:41.314280+0000","last_undegraded":"2024-02-26T10:39:41.314280+0000","last_fullsized":"2024-02-26T10:39:41.314280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008444+0000","last_change":"2024-02-26T10:39:40.001953+0000","last_active":"2024-02-26T10:39:41.008444+0000","last_peered":"2024-02-26T10:39:41.008444+0000","last_clean":"2024-02-26T10:39:41.008444+0000","last_became_active":"2024-02-26T10:39:40.001670+0000","last_became_peered":"2024-02-26T10:39:40.001670+0000","last_unstale":"2024-02-26T10:39:41.008444+0000","last_undegraded":"2024-02-26T10:39:41.008444+0000","last_fullsized":"2024-02-26T10:39:41.008444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:07.672401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010235+0000","last_change":"2024-02-26T10:39:39.000283+0000","last_active":"2024-02-26T10:39:41.010235+0000","last_peered":"2024-02-26T10:39:41.010235+0000","last_clean":"2024-02-26T10:39:41.010235+0000","last_became_active":"2024-02-26T10:39:38.999392+0000","last_became_peered":"2024-02-26T10:39:38.999392+0000","last_unstale":"2024-02-26T10:39:41.010235+0000","last_undegraded":"2024-02-26T10:39:41.010235+0000","last_fullsized":"2024-02-26T10:39:41.010235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:42.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217147+0000","last_change":"2024-02-26T10:39:40.013051+0000","last_active":"2024-02-26T10:39:41.217147+0000","last_peered":"2024-02-26T10:39:41.217147+0000","last_clean":"2024-02-26T10:39:41.217147+0000","last_became_active":"2024-02-26T10:39:40.012739+0000","last_became_peered":"2024-02-26T10:39:40.012739+0000","last_unstale":"2024-02-26T10:39:41.217147+0000","last_undegraded":"2024-02-26T10:39:41.217147+0000","last_fullsized":"2024-02-26T10:39:41.217147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314235+0000","last_change":"2024-02-26T10:39:39.009824+0000","last_active":"2024-02-26T10:39:41.314235+0000","last_peered":"2024-02-26T10:39:41.314235+0000","last_clean":"2024-02-26T10:39:41.314235+0000","last_became_active":"2024-02-26T10:39:39.009559+0000","last_became_peered":"2024-02-26T10:39:39.009559+0000","last_unstale":"2024-02-26T10:39:41.314235+0000","last_undegraded":"2024-02-26T10:39:41.314235+0000","last_fullsized":"2024-02-26T10:39:41.314235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008781+0000","last_change":"2024-02-26T10:39:40.022558+0000","last_active":"2024-02-26T10:39:41.008781+0000","last_peered":"2024-02-26T10:39:41.008781+0000","last_clean":"2024-02-26T10:39:41.008781+0000","last_became_active":"2024-02-26T10:39:40.022316+0000","last_became_peered":"2024-02-26T10:39:40.022316+0000","last_unstale":"2024-02-26T10:39:41.008781+0000","last_undegraded":"2024-02-26T10:39:41.008781+0000","last_fullsized":"2024-02-26T10:39:41.008781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:11:06.614146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006822+0000","last_change":"2024-02-26T10:39:39.009948+0000","last_active":"2024-02-26T10:39:41.006822+0000","last_peered":"2024-02-26T10:39:41.006822+0000","last_clean":"2024-02-26T10:39:41.006822+0000","last_became_active":"2024-02-26T10:39:39.009640+0000","last_became_peered":"2024-02-26T10:39:39.009640+0000","last_unstale":"2024-02-26T10:39:41.006822+0000","last_undegraded":"2024-02-26T10:39:41.006822+0000","last_fullsized":"2024-02-26T10:39:41.006822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:47.192375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315599+0000","last_change":"2024-02-26T10:39:40.009603+0000","last_active":"2024-02-26T10:39:41.315599+0000","last_peered":"2024-02-26T10:39:41.315599+0000","last_clean":"2024-02-26T10:39:41.315599+0000","last_became_active":"2024-02-26T10:39:40.009327+0000","last_became_peered":"2024-02-26T10:39:40.009327+0000","last_unstale":"2024-02-26T10:39:41.315599+0000","last_undegraded":"2024-02-26T10:39:41.315599+0000","last_fullsized":"2024-02-26T10:39:41.315599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025692+0000","last_change":"2024-02-26T10:39:38.992861+0000","last_active":"2024-02-26T10:39:41.025692+0000","last_peered":"2024-02-26T10:39:41.025692+0000","last_clean":"2024-02-26T10:39:41.025692+0000","last_became_active":"2024-02-26T10:39:38.992258+0000","last_became_peered":"2024-02-26T10:39:38.992258+0000","last_unstale":"2024-02-26T10:39:41.025692+0000","last_undegraded":"2024-02-26T10:39:41.025692+0000","last_fullsized":"2024-02-26T10:39:41.025692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025153+0000","last_change":"2024-02-26T10:39:40.009958+0000","last_active":"2024-02-26T10:39:41.025153+0000","last_peered":"2024-02-26T10:39:41.025153+0000","last_clean":"2024-02-26T10:39:41.025153+0000","last_became_active":"2024-02-26T10:39:40.009557+0000","last_became_peered":"2024-02-26T10:39:40.009557+0000","last_unstale":"2024-02-26T10:39:41.025153+0000","last_undegraded":"2024-02-26T10:39:41.025153+0000","last_fullsized":"2024-02-26T10:39:41.025153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.005621+0000","last_change":"2024-02-26T10:39:38.996371+0000","last_active":"2024-02-26T10:39:41.005621+0000","last_peered":"2024-02-26T10:39:41.005621+0000","last_clean":"2024-02-26T10:39:41.005621+0000","last_became_active":"2024-02-26T10:39:38.996052+0000","last_became_peered":"2024-02-26T10:39:38.996052+0000","last_unstale":"2024-02-26T10:39:41.005621+0000","last_undegraded":"2024-02-26T10:39:41.005621+0000","last_fullsized":"2024-02-26T10:39:41.005621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315022+0000","last_change":"2024-02-26T10:39:40.008904+0000","last_active":"2024-02-26T10:39:41.315022+0000","last_peered":"2024-02-26T10:39:41.315022+0000","last_clean":"2024-02-26T10:39:41.315022+0000","last_became_active":"2024-02-26T10:39:40.008631+0000","last_became_peered":"2024-02-26T10:39:40.008631+0000","last_unstale":"2024-02-26T10:39:41.315022+0000","last_undegraded":"2024-02-26T10:39:41.315022+0000","last_fullsized":"2024-02-26T10:39:41.315022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025549+0000","last_change":"2024-02-26T10:39:39.003861+0000","last_active":"2024-02-26T10:39:41.025549+0000","last_peered":"2024-02-26T10:39:41.025549+0000","last_clean":"2024-02-26T10:39:41.025549+0000","last_became_active":"2024-02-26T10:39:39.003370+0000","last_became_peered":"2024-02-26T10:39:39.003370+0000","last_unstale":"2024-02-26T10:39:41.025549+0000","last_undegraded":"2024-02-26T10:39:41.025549+0000","last_fullsized":"2024-02-26T10:39:41.025549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025536+0000","last_change":"2024-02-26T10:39:40.011820+0000","last_active":"2024-02-26T10:39:41.025536+0000","last_peered":"2024-02-26T10:39:41.025536+0000","last_clean":"2024-02-26T10:39:41.025536+0000","last_became_active":"2024-02-26T10:39:40.011511+0000","last_became_peered":"2024-02-26T10:39:40.011511+0000","last_unstale":"2024-02-26T10:39:41.025536+0000","last_undegraded":"2024-02-26T10:39:41.025536+0000","last_fullsized":"2024-02-26T10:39:41.025536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004696+0000","last_change":"2024-02-26T10:39:39.013362+0000","last_active":"2024-02-26T10:39:41.004696+0000","last_peered":"2024-02-26T10:39:41.004696+0000","last_clean":"2024-02-26T10:39:41.004696+0000","last_became_active":"2024-02-26T10:39:39.012819+0000","last_became_peered":"2024-02-26T10:39:39.012819+0000","last_unstale":"2024-02-26T10:39:41.004696+0000","last_undegraded":"2024-02-26T10:39:41.004696+0000","last_fullsized":"2024-02-26T10:39:41.004696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.118859+0000","last_change":"2024-02-26T10:39:39.997501+0000","last_active":"2024-02-26T10:39:41.118859+0000","last_peered":"2024-02-26T10:39:41.118859+0000","last_clean":"2024-02-26T10:39:41.118859+0000","last_became_active":"2024-02-26T10:39:39.996084+0000","last_became_peered":"2024-02-26T10:39:39.996084+0000","last_unstale":"2024-02-26T10:39:41.118859+0000","last_undegraded":"2024-02-26T10:39:41.118859+0000","last_fullsized":"2024-02-26T10:39:41.118859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010496+0000","last_change":"2024-02-26T10:39:38.999680+0000","last_active":"2024-02-26T10:39:41.010496+0000","last_peered":"2024-02-26T10:39:41.010496+0000","last_clean":"2024-02-26T10:39:41.010496+0000","last_became_active":"2024-02-26T10:39:38.999058+0000","last_became_peered":"2024-02-26T10:39:38.999058+0000","last_unstale":"2024-02-26T10:39:41.010496+0000","last_undegraded":"2024-02-26T10:39:41.010496+0000","last_fullsized":"2024-02-26T10:39:41.010496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:26.840527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010324+0000","last_change":"2024-02-26T10:39:39.995364+0000","last_active":"2024-02-26T10:39:41.010324+0000","last_peered":"2024-02-26T10:39:41.010324+0000","last_clean":"2024-02-26T10:39:41.010324+0000","last_became_active":"2024-02-26T10:39:39.995077+0000","last_became_peered":"2024-02-26T10:39:39.995077+0000","last_unstale":"2024-02-26T10:39:41.010324+0000","last_undegraded":"2024-02-26T10:39:41.010324+0000","last_fullsized":"2024-02-26T10:39:41.010324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119421+0000","last_change":"2024-02-26T10:39:39.006896+0000","last_active":"2024-02-26T10:39:41.119421+0000","last_peered":"2024-02-26T10:39:41.119421+0000","last_clean":"2024-02-26T10:39:41.119421+0000","last_became_active":"2024-02-26T10:39:39.006604+0000","last_became_peered":"2024-02-26T10:39:39.006604+0000","last_unstale":"2024-02-26T10:39:41.119421+0000","last_undegraded":"2024-02-26T10:39:41.119421+0000","last_fullsized":"2024-02-26T10:39:41.119421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217544+0000","last_change":"2024-02-26T10:39:40.018706+0000","last_active":"2024-02-26T10:39:41.217544+0000","last_peered":"2024-02-26T10:39:41.217544+0000","last_clean":"2024-02-26T10:39:41.217544+0000","last_became_active":"2024-02-26T10:39:40.018417+0000","last_became_peered":"2024-02-26T10:39:40.018417+0000","last_unstale":"2024-02-26T10:39:41.217544+0000","last_undegraded":"2024-02-26T10:39:41.217544+0000","last_fullsized":"2024-02-26T10:39:41.217544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.313945+0000","last_change":"2024-02-26T10:39:39.013277+0000","last_active":"2024-02-26T10:39:41.313945+0000","last_peered":"2024-02-26T10:39:41.313945+0000","last_clean":"2024-02-26T10:39:41.313945+0000","last_became_active":"2024-02-26T10:39:39.012073+0000","last_became_peered":"2024-02-26T10:39:39.012073+0000","last_unstale":"2024-02-26T10:39:41.313945+0000","last_undegraded":"2024-02-26T10:39:41.313945+0000","last_fullsized":"2024-02-26T10:39:41.313945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:44.421668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.218047+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:39:41.218047+0000","last_peered":"2024-02-26T10:39:41.218047+0000","last_clean":"2024-02-26T10:39:41.218047+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:39:41.218047+0000","last_undegraded":"2024-02-26T10:39:41.218047+0000","last_fullsized":"2024-02-26T10:39:41.218047+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":740369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":151545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":149981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":151545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":740279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:39:44.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:39:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.749+0000 7f9c1e600640 1 Processor -- start 2024-02-26T10:39:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.749+0000 7f9c1e600640 1 -- start start 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c1e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9c1814f3d0 0x7f9c1806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c1e600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c180c2870 0x7f9c18155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c1e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1806ed10 con 0x7f9c181556a0 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c1e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1806dd80 con 0x7f9c1814f3d0 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c1e600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c18155c00 con 0x7f9c180c2870 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c17400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9c1814f3d0 0x7f9c1806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c17400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9c1814f3d0 0x7f9c1806e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42184/0 (socket says 172.21.15.67:42184) 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.751+0000 7f9c17400640 1 -- 172.21.15.67:0/602973583 learned_addr learned my addr 172.21.15.67:0/602973583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.752+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/602973583 >> v2:172.21.15.82:3300/0 conn(0x7f9c180c2870 0x7f9c18155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.752+0000 7f9c17400640 1 -- 172.21.15.67:0/602973583 >> v2:172.21.15.82:3300/0 conn(0x7f9c180c2870 msgr2=0x7f9c18155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.752+0000 7f9c17400640 1 --2- 172.21.15.67:0/602973583 >> v2:172.21.15.82:3300/0 conn(0x7f9c180c2870 0x7f9c18155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.752+0000 7f9c17400640 1 -- 172.21.15.67:0/602973583 >> v1:172.21.15.67:6789/0 conn(0x7f9c181556a0 legacy=0x7f9c18071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.752+0000 7f9c17400640 1 -- 172.21.15.67:0/602973583 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c18155ee0 con 0x7f9c1814f3d0 2024-02-26T10:39:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.752+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/602973583 >> v2:172.21.15.82:3300/0 conn(0x7f9c180c2870 0x7f9c18155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.753+0000 7f9c17400640 1 --2- 172.21.15.67:0/602973583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c1814f3d0 0x7f9c1806e7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9c180c1120 tx=0x7f9c0802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b25b585e4cbd14c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.753+0000 7f9c16a00640 1 -- 172.21.15.67:0/602973583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0802e070 con 0x7f9c1814f3d0 2024-02-26T10:39:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.753+0000 7f9c16a00640 1 -- 172.21.15.67:0/602973583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08037070 con 0x7f9c1814f3d0 2024-02-26T10:39:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.753+0000 7f9c16a00640 1 -- 172.21.15.67:0/602973583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08031a00 con 0x7f9c1814f3d0 2024-02-26T10:39:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 -- 172.21.15.67:0/602973583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c1814f3d0 msgr2=0x7f9c1806e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 --2- 172.21.15.67:0/602973583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c1814f3d0 0x7f9c1806e7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9c180c1120 tx=0x7f9c0802b880 comp rx=0 tx=0).stop 2024-02-26T10:39:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 -- 172.21.15.67:0/602973583 shutdown_connections 2024-02-26T10:39:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 --2- 172.21.15.67:0/602973583 >> v2:172.21.15.82:3300/0 conn(0x7f9c180c2870 0x7f9c18155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 --2- 172.21.15.67:0/602973583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c1814f3d0 0x7f9c1806e7d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 -- 172.21.15.67:0/602973583 >> 172.21.15.67:0/602973583 conn(0x7f9c180a84a0 msgr2=0x7f9c180a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 -- 172.21.15.67:0/602973583 shutdown_connections 2024-02-26T10:39:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.754+0000 7f9c1e600640 1 -- 172.21.15.67:0/602973583 wait complete. 2024-02-26T10:39:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.756+0000 7f9c1e600640 1 Processor -- start 2024-02-26T10:39:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.756+0000 7f9c1e600640 1 -- start start 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c1e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c180c2870 0x7f9c18145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c1e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1814f3d0 0x7f9c1813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c1e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 0x7f9c1813d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c1e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180c1810 con 0x7f9c181556a0 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c1e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1806e2b0 con 0x7f9c1814f3d0 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c1e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c18071e10 con 0x7f9c180c2870 2024-02-26T10:39:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 0x7f9c1813d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 0x7f9c1813d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42192/0 (socket says 172.21.15.67:42192) 2024-02-26T10:39:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.757+0000 7f9c17e00640 1 -- 172.21.15.67:0/2413189175 learned_addr learned my addr 172.21.15.67:0/2413189175 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17400640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c180c2870 0x7f9c18145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c0ea00640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1814f3d0 0x7f9c1813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17e00640 1 -- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c180c2870 msgr2=0x7f9c18145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17e00640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c180c2870 0x7f9c18145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17e00640 1 -- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1814f3d0 msgr2=0x7f9c1813cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17e00640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1814f3d0 0x7f9c1813cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17e00640 1 -- 172.21.15.67:0/2413189175 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c181ea870 con 0x7f9c181556a0 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c17e00640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 0x7f9c1813d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c6b0 tx=0x7f9c1000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.758+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10003b70 con 0x7f9c181556a0 2024-02-26T10:39:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.759+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10003d30 con 0x7f9c181556a0 2024-02-26T10:39:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.759+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10005310 con 0x7f9c181556a0 2024-02-26T10:39:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.759+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c181eab30 con 0x7f9c181556a0 2024-02-26T10:39:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.759+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c181eb0f0 con 0x7f9c181556a0 2024-02-26T10:39:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.760+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c18074a30 con 0x7f9c181556a0 2024-02-26T10:39:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.761+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c100054b0 con 0x7f9c181556a0 2024-02-26T10:39:44.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.762+0000 7f9c0fe00640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9bf406b500 0x7f9bf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:44.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.762+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c10089f90 con 0x7f9c181556a0 2024-02-26T10:39:44.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.762+0000 7f9c17400640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9bf406b500 0x7f9bf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.764+0000 7f9c17400640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9bf406b500 0x7f9bf406d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c08036000 tx=0x7f9c08008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:44.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.766+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c10058ea0 con 0x7f9c181556a0 2024-02-26T10:39:44.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.961+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c180c13f0 con 0x7f9bf406b500 2024-02-26T10:39:44.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.981+0000 7f9c0fe00640 1 -- 172.21.15.67:0/2413189175 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7f9c180c13f0 con 0x7f9bf406b500 2024-02-26T10:39:44.985 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:44.986 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:39:44.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9bf406b500 msgr2=0x7f9bf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9bf406b500 0x7f9bf406d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c08036000 tx=0x7f9c08008030 comp rx=0 tx=0).stop 2024-02-26T10:39:44.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 msgr2=0x7f9c1813d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:44.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 0x7f9c1813d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c6b0 tx=0x7f9c1000cb70 comp rx=0 tx=0).stop 2024-02-26T10:39:44.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 shutdown_connections 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c181556a0 0x7f9c1813d300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9bf406b500 0x7f9bf406d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c1814f3d0 0x7f9c1813cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 --2- 172.21.15.67:0/2413189175 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c180c2870 0x7f9c18145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 >> 172.21.15.67:0/2413189175 conn(0x7f9c180a84a0 msgr2=0x7f9c180c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 shutdown_connections 2024-02-26T10:39:44.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:44.986+0000 7f9c1e600640 1 -- 172.21.15.67:0/2413189175 wait complete. 2024-02-26T10:39:45.007 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-02-26T10:39:44.347028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220368,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754352,"statfs":{"total":773094113280,"available":772868456448,"internally_reserved":0,"allocated":4300800,"data_stored":2384844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001792"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314588+0000","last_change":"2024-02-26T10:39:40.026044+0000","last_active":"2024-02-26T10:39:41.314588+0000","last_peered":"2024-02-26T10:39:41.314588+0000","last_clean":"2024-02-26T10:39:41.314588+0000","last_became_active":"2024-02-26T10:39:40.025716+0000","last_became_peered":"2024-02-26T10:39:40.025716+0000","last_unstale":"2024-02-26T10:39:41.314588+0000","last_undegraded":"2024-02-26T10:39:41.314588+0000","last_fullsized":"2024-02-26T10:39:41.314588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119894+0000","last_change":"2024-02-26T10:39:40.000691+0000","last_active":"2024-02-26T10:39:41.119894+0000","last_peered":"2024-02-26T10:39:41.119894+0000","last_clean":"2024-02-26T10:39:41.119894+0000","last_became_active":"2024-02-26T10:39:40.000419+0000","last_became_peered":"2024-02-26T10:39:40.000419+0000","last_unstale":"2024-02-26T10:39:41.119894+0000","last_undegraded":"2024-02-26T10:39:41.119894+0000","last_fullsized":"2024-02-26T10:39:41.119894+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314963+0000","last_change":"2024-02-26T10:39:40.017208+0000","last_active":"2024-02-26T10:39:41.314963+0000","last_peered":"2024-02-26T10:39:41.314963+0000","last_clean":"2024-02-26T10:39:41.314963+0000","last_became_active":"2024-02-26T10:39:40.016350+0000","last_became_peered":"2024-02-26T10:39:40.016350+0000","last_unstale":"2024-02-26T10:39:41.314963+0000","last_undegraded":"2024-02-26T10:39:41.314963+0000","last_fullsized":"2024-02-26T10:39:41.314963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.005464+0000","last_change":"2024-02-26T10:39:40.009578+0000","last_active":"2024-02-26T10:39:41.005464+0000","last_peered":"2024-02-26T10:39:41.005464+0000","last_clean":"2024-02-26T10:39:41.005464+0000","last_became_active":"2024-02-26T10:39:40.008770+0000","last_became_peered":"2024-02-26T10:39:40.008770+0000","last_unstale":"2024-02-26T10:39:41.005464+0000","last_undegraded":"2024-02-26T10:39:41.005464+0000","last_fullsized":"2024-02-26T10:39:41.005464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:48:42.311486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008177+0000","last_change":"2024-02-26T10:39:39.996527+0000","last_active":"2024-02-26T10:39:41.008177+0000","last_peered":"2024-02-26T10:39:41.008177+0000","last_clean":"2024-02-26T10:39:41.008177+0000","last_became_active":"2024-02-26T10:39:39.995687+0000","last_became_peered":"2024-02-26T10:39:39.995687+0000","last_unstale":"2024-02-26T10:39:41.008177+0000","last_undegraded":"2024-02-26T10:39:41.008177+0000","last_fullsized":"2024-02-26T10:39:41.008177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:29.964072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119730+0000","last_change":"2024-02-26T10:39:40.025212+0000","last_active":"2024-02-26T10:39:41.119730+0000","last_peered":"2024-02-26T10:39:41.119730+0000","last_clean":"2024-02-26T10:39:41.119730+0000","last_became_active":"2024-02-26T10:39:40.024642+0000","last_became_peered":"2024-02-26T10:39:40.024642+0000","last_unstale":"2024-02-26T10:39:41.119730+0000","last_undegraded":"2024-02-26T10:39:41.119730+0000","last_fullsized":"2024-02-26T10:39:41.119730+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217697+0000","last_change":"2024-02-26T10:39:40.014885+0000","last_active":"2024-02-26T10:39:41.217697+0000","last_peered":"2024-02-26T10:39:41.217697+0000","last_clean":"2024-02-26T10:39:41.217697+0000","last_became_active":"2024-02-26T10:39:40.014675+0000","last_became_peered":"2024-02-26T10:39:40.014675+0000","last_unstale":"2024-02-26T10:39:41.217697+0000","last_undegraded":"2024-02-26T10:39:41.217697+0000","last_fullsized":"2024-02-26T10:39:41.217697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217909+0000","last_change":"2024-02-26T10:39:40.005845+0000","last_active":"2024-02-26T10:39:41.217909+0000","last_peered":"2024-02-26T10:39:41.217909+0000","last_clean":"2024-02-26T10:39:41.217909+0000","last_became_active":"2024-02-26T10:39:40.005497+0000","last_became_peered":"2024-02-26T10:39:40.005497+0000","last_unstale":"2024-02-26T10:39:41.217909+0000","last_undegraded":"2024-02-26T10:39:41.217909+0000","last_fullsized":"2024-02-26T10:39:41.217909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011736+0000","last_change":"2024-02-26T10:39:40.006430+0000","last_active":"2024-02-26T10:39:41.011736+0000","last_peered":"2024-02-26T10:39:41.011736+0000","last_clean":"2024-02-26T10:39:41.011736+0000","last_became_active":"2024-02-26T10:39:40.004846+0000","last_became_peered":"2024-02-26T10:39:40.004846+0000","last_unstale":"2024-02-26T10:39:41.011736+0000","last_undegraded":"2024-02-26T10:39:41.011736+0000","last_fullsized":"2024-02-26T10:39:41.011736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006594+0000","last_change":"2024-02-26T10:39:39.995327+0000","last_active":"2024-02-26T10:39:41.006594+0000","last_peered":"2024-02-26T10:39:41.006594+0000","last_clean":"2024-02-26T10:39:41.006594+0000","last_became_active":"2024-02-26T10:39:39.994245+0000","last_became_peered":"2024-02-26T10:39:39.994245+0000","last_unstale":"2024-02-26T10:39:41.006594+0000","last_undegraded":"2024-02-26T10:39:41.006594+0000","last_fullsized":"2024-02-26T10:39:41.006594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217791+0000","last_change":"2024-02-26T10:39:40.017115+0000","last_active":"2024-02-26T10:39:41.217791+0000","last_peered":"2024-02-26T10:39:41.217791+0000","last_clean":"2024-02-26T10:39:41.217791+0000","last_became_active":"2024-02-26T10:39:40.016876+0000","last_became_peered":"2024-02-26T10:39:40.016876+0000","last_unstale":"2024-02-26T10:39:41.217791+0000","last_undegraded":"2024-02-26T10:39:41.217791+0000","last_fullsized":"2024-02-26T10:39:41.217791+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314981+0000","last_change":"2024-02-26T10:39:40.011833+0000","last_active":"2024-02-26T10:39:41.314981+0000","last_peered":"2024-02-26T10:39:41.314981+0000","last_clean":"2024-02-26T10:39:41.314981+0000","last_became_active":"2024-02-26T10:39:40.011581+0000","last_became_peered":"2024-02-26T10:39:40.011581+0000","last_unstale":"2024-02-26T10:39:41.314981+0000","last_undegraded":"2024-02-26T10:39:41.314981+0000","last_fullsized":"2024-02-26T10:39:41.314981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.009182+0000","last_change":"2024-02-26T10:39:40.000902+0000","last_active":"2024-02-26T10:39:41.009182+0000","last_peered":"2024-02-26T10:39:41.009182+0000","last_clean":"2024-02-26T10:39:41.009182+0000","last_became_active":"2024-02-26T10:39:39.999441+0000","last_became_peered":"2024-02-26T10:39:39.999441+0000","last_unstale":"2024-02-26T10:39:41.009182+0000","last_undegraded":"2024-02-26T10:39:41.009182+0000","last_fullsized":"2024-02-26T10:39:41.009182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:27.937806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120132+0000","last_change":"2024-02-26T10:39:39.999249+0000","last_active":"2024-02-26T10:39:41.120132+0000","last_peered":"2024-02-26T10:39:41.120132+0000","last_clean":"2024-02-26T10:39:41.120132+0000","last_became_active":"2024-02-26T10:39:39.998319+0000","last_became_peered":"2024-02-26T10:39:39.998319+0000","last_unstale":"2024-02-26T10:39:41.120132+0000","last_undegraded":"2024-02-26T10:39:41.120132+0000","last_fullsized":"2024-02-26T10:39:41.120132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:29.312334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315682+0000","last_change":"2024-02-26T10:39:40.024788+0000","last_active":"2024-02-26T10:39:41.315682+0000","last_peered":"2024-02-26T10:39:41.315682+0000","last_clean":"2024-02-26T10:39:41.315682+0000","last_became_active":"2024-02-26T10:39:40.024454+0000","last_became_peered":"2024-02-26T10:39:40.024454+0000","last_unstale":"2024-02-26T10:39:41.315682+0000","last_undegraded":"2024-02-26T10:39:41.315682+0000","last_fullsized":"2024-02-26T10:39:41.315682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.218187+0000","last_change":"2024-02-26T10:39:40.011104+0000","last_active":"2024-02-26T10:39:41.218187+0000","last_peered":"2024-02-26T10:39:41.218187+0000","last_clean":"2024-02-26T10:39:41.218187+0000","last_became_active":"2024-02-26T10:39:40.010774+0000","last_became_peered":"2024-02-26T10:39:40.010774+0000","last_unstale":"2024-02-26T10:39:41.218187+0000","last_undegraded":"2024-02-26T10:39:41.218187+0000","last_fullsized":"2024-02-26T10:39:41.218187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.218219+0000","last_change":"2024-02-26T10:39:40.017107+0000","last_active":"2024-02-26T10:39:41.218219+0000","last_peered":"2024-02-26T10:39:41.218219+0000","last_clean":"2024-02-26T10:39:41.218219+0000","last_became_active":"2024-02-26T10:39:40.016813+0000","last_became_peered":"2024-02-26T10:39:40.016813+0000","last_unstale":"2024-02-26T10:39:41.218219+0000","last_undegraded":"2024-02-26T10:39:41.218219+0000","last_fullsized":"2024-02-26T10:39:41.218219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120231+0000","last_change":"2024-02-26T10:39:39.998011+0000","last_active":"2024-02-26T10:39:41.120231+0000","last_peered":"2024-02-26T10:39:41.120231+0000","last_clean":"2024-02-26T10:39:41.120231+0000","last_became_active":"2024-02-26T10:39:39.997725+0000","last_became_peered":"2024-02-26T10:39:39.997725+0000","last_unstale":"2024-02-26T10:39:41.120231+0000","last_undegraded":"2024-02-26T10:39:41.120231+0000","last_fullsized":"2024-02-26T10:39:41.120231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:02.449804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119875+0000","last_change":"2024-02-26T10:39:39.996782+0000","last_active":"2024-02-26T10:39:41.119875+0000","last_peered":"2024-02-26T10:39:41.119875+0000","last_clean":"2024-02-26T10:39:41.119875+0000","last_became_active":"2024-02-26T10:39:39.996469+0000","last_became_peered":"2024-02-26T10:39:39.996469+0000","last_unstale":"2024-02-26T10:39:41.119875+0000","last_undegraded":"2024-02-26T10:39:41.119875+0000","last_fullsized":"2024-02-26T10:39:41.119875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008413+0000","last_change":"2024-02-26T10:39:40.028123+0000","last_active":"2024-02-26T10:39:41.008413+0000","last_peered":"2024-02-26T10:39:41.008413+0000","last_clean":"2024-02-26T10:39:41.008413+0000","last_became_active":"2024-02-26T10:39:40.027299+0000","last_became_peered":"2024-02-26T10:39:40.027299+0000","last_unstale":"2024-02-26T10:39:41.008413+0000","last_undegraded":"2024-02-26T10:39:41.008413+0000","last_fullsized":"2024-02-26T10:39:41.008413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:54:29.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010612+0000","last_change":"2024-02-26T10:39:40.003035+0000","last_active":"2024-02-26T10:39:41.010612+0000","last_peered":"2024-02-26T10:39:41.010612+0000","last_clean":"2024-02-26T10:39:41.010612+0000","last_became_active":"2024-02-26T10:39:40.002724+0000","last_became_peered":"2024-02-26T10:39:40.002724+0000","last_unstale":"2024-02-26T10:39:41.010612+0000","last_undegraded":"2024-02-26T10:39:41.010612+0000","last_fullsized":"2024-02-26T10:39:41.010612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010619+0000","last_change":"2024-02-26T10:39:39.990872+0000","last_active":"2024-02-26T10:39:41.010619+0000","last_peered":"2024-02-26T10:39:41.010619+0000","last_clean":"2024-02-26T10:39:41.010619+0000","last_became_active":"2024-02-26T10:39:39.990134+0000","last_became_peered":"2024-02-26T10:39:39.990134+0000","last_unstale":"2024-02-26T10:39:41.010619+0000","last_undegraded":"2024-02-26T10:39:41.010619+0000","last_fullsized":"2024-02-26T10:39:41.010619+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.024966+0000","last_change":"2024-02-26T10:39:40.022805+0000","last_active":"2024-02-26T10:39:41.024966+0000","last_peered":"2024-02-26T10:39:41.024966+0000","last_clean":"2024-02-26T10:39:41.024966+0000","last_became_active":"2024-02-26T10:39:40.022510+0000","last_became_peered":"2024-02-26T10:39:40.022510+0000","last_unstale":"2024-02-26T10:39:41.024966+0000","last_undegraded":"2024-02-26T10:39:41.024966+0000","last_fullsized":"2024-02-26T10:39:41.024966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314008+0000","last_change":"2024-02-26T10:39:40.017086+0000","last_active":"2024-02-26T10:39:41.314008+0000","last_peered":"2024-02-26T10:39:41.314008+0000","last_clean":"2024-02-26T10:39:41.314008+0000","last_became_active":"2024-02-26T10:39:40.016779+0000","last_became_peered":"2024-02-26T10:39:40.016779+0000","last_unstale":"2024-02-26T10:39:41.314008+0000","last_undegraded":"2024-02-26T10:39:41.314008+0000","last_fullsized":"2024-02-26T10:39:41.314008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006656+0000","last_change":"2024-02-26T10:39:40.015014+0000","last_active":"2024-02-26T10:39:41.006656+0000","last_peered":"2024-02-26T10:39:41.006656+0000","last_clean":"2024-02-26T10:39:41.006656+0000","last_became_active":"2024-02-26T10:39:40.014611+0000","last_became_peered":"2024-02-26T10:39:40.014611+0000","last_unstale":"2024-02-26T10:39:41.006656+0000","last_undegraded":"2024-02-26T10:39:41.006656+0000","last_fullsized":"2024-02-26T10:39:41.006656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120473+0000","last_change":"2024-02-26T10:39:40.007272+0000","last_active":"2024-02-26T10:39:41.120473+0000","last_peered":"2024-02-26T10:39:41.120473+0000","last_clean":"2024-02-26T10:39:41.120473+0000","last_became_active":"2024-02-26T10:39:40.006923+0000","last_became_peered":"2024-02-26T10:39:40.006923+0000","last_unstale":"2024-02-26T10:39:41.120473+0000","last_undegraded":"2024-02-26T10:39:41.120473+0000","last_fullsized":"2024-02-26T10:39:41.120473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:21.270698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006336+0000","last_change":"2024-02-26T10:39:40.008980+0000","last_active":"2024-02-26T10:39:41.006336+0000","last_peered":"2024-02-26T10:39:41.006336+0000","last_clean":"2024-02-26T10:39:41.006336+0000","last_became_active":"2024-02-26T10:39:40.008531+0000","last_became_peered":"2024-02-26T10:39:40.008531+0000","last_unstale":"2024-02-26T10:39:41.006336+0000","last_undegraded":"2024-02-26T10:39:41.006336+0000","last_fullsized":"2024-02-26T10:39:41.006336+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:11:31.399212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315449+0000","last_change":"2024-02-26T10:39:40.019372+0000","last_active":"2024-02-26T10:39:41.315449+0000","last_peered":"2024-02-26T10:39:41.315449+0000","last_clean":"2024-02-26T10:39:41.315449+0000","last_became_active":"2024-02-26T10:39:40.019025+0000","last_became_peered":"2024-02-26T10:39:40.019025+0000","last_unstale":"2024-02-26T10:39:41.315449+0000","last_undegraded":"2024-02-26T10:39:41.315449+0000","last_fullsized":"2024-02-26T10:39:41.315449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011011+0000","last_change":"2024-02-26T10:39:39.994088+0000","last_active":"2024-02-26T10:39:41.011011+0000","last_peered":"2024-02-26T10:39:41.011011+0000","last_clean":"2024-02-26T10:39:41.011011+0000","last_became_active":"2024-02-26T10:39:39.993717+0000","last_became_peered":"2024-02-26T10:39:39.993717+0000","last_unstale":"2024-02-26T10:39:41.011011+0000","last_undegraded":"2024-02-26T10:39:41.011011+0000","last_fullsized":"2024-02-26T10:39:41.011011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217331+0000","last_change":"2024-02-26T10:39:40.010029+0000","last_active":"2024-02-26T10:39:41.217331+0000","last_peered":"2024-02-26T10:39:41.217331+0000","last_clean":"2024-02-26T10:39:41.217331+0000","last_became_active":"2024-02-26T10:39:40.009745+0000","last_became_peered":"2024-02-26T10:39:40.009745+0000","last_unstale":"2024-02-26T10:39:41.217331+0000","last_undegraded":"2024-02-26T10:39:41.217331+0000","last_fullsized":"2024-02-26T10:39:41.217331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314375+0000","last_change":"2024-02-26T10:39:40.008658+0000","last_active":"2024-02-26T10:39:41.314375+0000","last_peered":"2024-02-26T10:39:41.314375+0000","last_clean":"2024-02-26T10:39:41.314375+0000","last_became_active":"2024-02-26T10:39:40.008354+0000","last_became_peered":"2024-02-26T10:39:40.008354+0000","last_unstale":"2024-02-26T10:39:41.314375+0000","last_undegraded":"2024-02-26T10:39:41.314375+0000","last_fullsized":"2024-02-26T10:39:41.314375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217161+0000","last_change":"2024-02-26T10:39:40.013245+0000","last_active":"2024-02-26T10:39:41.217161+0000","last_peered":"2024-02-26T10:39:41.217161+0000","last_clean":"2024-02-26T10:39:41.217161+0000","last_became_active":"2024-02-26T10:39:40.008999+0000","last_became_peered":"2024-02-26T10:39:40.008999+0000","last_unstale":"2024-02-26T10:39:41.217161+0000","last_undegraded":"2024-02-26T10:39:41.217161+0000","last_fullsized":"2024-02-26T10:39:41.217161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025402+0000","last_change":"2024-02-26T10:39:40.021369+0000","last_active":"2024-02-26T10:39:41.025402+0000","last_peered":"2024-02-26T10:39:41.025402+0000","last_clean":"2024-02-26T10:39:41.025402+0000","last_became_active":"2024-02-26T10:39:40.021061+0000","last_became_peered":"2024-02-26T10:39:40.021061+0000","last_unstale":"2024-02-26T10:39:41.025402+0000","last_undegraded":"2024-02-26T10:39:41.025402+0000","last_fullsized":"2024-02-26T10:39:41.025402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315150+0000","last_change":"2024-02-26T10:39:39.014072+0000","last_active":"2024-02-26T10:39:41.315150+0000","last_peered":"2024-02-26T10:39:41.315150+0000","last_clean":"2024-02-26T10:39:41.315150+0000","last_became_active":"2024-02-26T10:39:39.013810+0000","last_became_peered":"2024-02-26T10:39:39.013810+0000","last_unstale":"2024-02-26T10:39:41.315150+0000","last_undegraded":"2024-02-26T10:39:41.315150+0000","last_fullsized":"2024-02-26T10:39:41.315150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:12:03.804658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314975+0000","last_change":"2024-02-26T10:39:40.011479+0000","last_active":"2024-02-26T10:39:41.314975+0000","last_peered":"2024-02-26T10:39:41.314975+0000","last_clean":"2024-02-26T10:39:41.314975+0000","last_became_active":"2024-02-26T10:39:40.011123+0000","last_became_peered":"2024-02-26T10:39:40.011123+0000","last_unstale":"2024-02-26T10:39:41.314975+0000","last_undegraded":"2024-02-26T10:39:41.314975+0000","last_fullsized":"2024-02-26T10:39:41.314975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025487+0000","last_change":"2024-02-26T10:39:39.013171+0000","last_active":"2024-02-26T10:39:41.025487+0000","last_peered":"2024-02-26T10:39:41.025487+0000","last_clean":"2024-02-26T10:39:41.025487+0000","last_became_active":"2024-02-26T10:39:39.012785+0000","last_became_peered":"2024-02-26T10:39:39.012785+0000","last_unstale":"2024-02-26T10:39:41.025487+0000","last_undegraded":"2024-02-26T10:39:41.025487+0000","last_fullsized":"2024-02-26T10:39:41.025487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:04:56.455641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011659+0000","last_change":"2024-02-26T10:39:39.992637+0000","last_active":"2024-02-26T10:39:41.011659+0000","last_peered":"2024-02-26T10:39:41.011659+0000","last_clean":"2024-02-26T10:39:41.011659+0000","last_became_active":"2024-02-26T10:39:39.991960+0000","last_became_peered":"2024-02-26T10:39:39.991960+0000","last_unstale":"2024-02-26T10:39:41.011659+0000","last_undegraded":"2024-02-26T10:39:41.011659+0000","last_fullsized":"2024-02-26T10:39:41.011659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119568+0000","last_change":"2024-02-26T10:39:39.000798+0000","last_active":"2024-02-26T10:39:41.119568+0000","last_peered":"2024-02-26T10:39:41.119568+0000","last_clean":"2024-02-26T10:39:41.119568+0000","last_became_active":"2024-02-26T10:39:38.998438+0000","last_became_peered":"2024-02-26T10:39:38.998438+0000","last_unstale":"2024-02-26T10:39:41.119568+0000","last_undegraded":"2024-02-26T10:39:41.119568+0000","last_fullsized":"2024-02-26T10:39:41.119568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:29:40.306221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006631+0000","last_change":"2024-02-26T10:39:40.008958+0000","last_active":"2024-02-26T10:39:41.006631+0000","last_peered":"2024-02-26T10:39:41.006631+0000","last_clean":"2024-02-26T10:39:41.006631+0000","last_became_active":"2024-02-26T10:39:40.008566+0000","last_became_peered":"2024-02-26T10:39:40.008566+0000","last_unstale":"2024-02-26T10:39:41.006631+0000","last_undegraded":"2024-02-26T10:39:41.006631+0000","last_fullsized":"2024-02-26T10:39:41.006631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008316+0000","last_change":"2024-02-26T10:39:38.993592+0000","last_active":"2024-02-26T10:39:41.008316+0000","last_peered":"2024-02-26T10:39:41.008316+0000","last_clean":"2024-02-26T10:39:41.008316+0000","last_became_active":"2024-02-26T10:39:38.993042+0000","last_became_peered":"2024-02-26T10:39:38.993042+0000","last_unstale":"2024-02-26T10:39:41.008316+0000","last_undegraded":"2024-02-26T10:39:41.008316+0000","last_fullsized":"2024-02-26T10:39:41.008316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:06:10.297346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217504+0000","last_change":"2024-02-26T10:39:40.017365+0000","last_active":"2024-02-26T10:39:41.217504+0000","last_peered":"2024-02-26T10:39:41.217504+0000","last_clean":"2024-02-26T10:39:41.217504+0000","last_became_active":"2024-02-26T10:39:40.010336+0000","last_became_peered":"2024-02-26T10:39:40.010336+0000","last_unstale":"2024-02-26T10:39:41.217504+0000","last_undegraded":"2024-02-26T10:39:41.217504+0000","last_fullsized":"2024-02-26T10:39:41.217504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010790+0000","last_change":"2024-02-26T10:39:39.005253+0000","last_active":"2024-02-26T10:39:41.010790+0000","last_peered":"2024-02-26T10:39:41.010790+0000","last_clean":"2024-02-26T10:39:41.010790+0000","last_became_active":"2024-02-26T10:39:39.004947+0000","last_became_peered":"2024-02-26T10:39:39.004947+0000","last_unstale":"2024-02-26T10:39:41.010790+0000","last_undegraded":"2024-02-26T10:39:41.010790+0000","last_fullsized":"2024-02-26T10:39:41.010790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:19:13.361650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314788+0000","last_change":"2024-02-26T10:39:40.024744+0000","last_active":"2024-02-26T10:39:41.314788+0000","last_peered":"2024-02-26T10:39:41.314788+0000","last_clean":"2024-02-26T10:39:41.314788+0000","last_became_active":"2024-02-26T10:39:40.023928+0000","last_became_peered":"2024-02-26T10:39:40.023928+0000","last_unstale":"2024-02-26T10:39:41.314788+0000","last_undegraded":"2024-02-26T10:39:41.314788+0000","last_fullsized":"2024-02-26T10:39:41.314788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010358+0000","last_change":"2024-02-26T10:39:39.007073+0000","last_active":"2024-02-26T10:39:41.010358+0000","last_peered":"2024-02-26T10:39:41.010358+0000","last_clean":"2024-02-26T10:39:41.010358+0000","last_became_active":"2024-02-26T10:39:39.004616+0000","last_became_peered":"2024-02-26T10:39:39.004616+0000","last_unstale":"2024-02-26T10:39:41.010358+0000","last_undegraded":"2024-02-26T10:39:41.010358+0000","last_fullsized":"2024-02-26T10:39:41.010358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:07.037054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010170+0000","last_change":"2024-02-26T10:39:40.014474+0000","last_active":"2024-02-26T10:39:41.010170+0000","last_peered":"2024-02-26T10:39:41.010170+0000","last_clean":"2024-02-26T10:39:41.010170+0000","last_became_active":"2024-02-26T10:39:40.014129+0000","last_became_peered":"2024-02-26T10:39:40.014129+0000","last_unstale":"2024-02-26T10:39:41.010170+0000","last_undegraded":"2024-02-26T10:39:41.010170+0000","last_fullsized":"2024-02-26T10:39:41.010170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:55.255246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.118850+0000","last_change":"2024-02-26T10:39:38.998681+0000","last_active":"2024-02-26T10:39:41.118850+0000","last_peered":"2024-02-26T10:39:41.118850+0000","last_clean":"2024-02-26T10:39:41.118850+0000","last_became_active":"2024-02-26T10:39:38.998355+0000","last_became_peered":"2024-02-26T10:39:38.998355+0000","last_unstale":"2024-02-26T10:39:41.118850+0000","last_undegraded":"2024-02-26T10:39:41.118850+0000","last_fullsized":"2024-02-26T10:39:41.118850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:54.191021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004524+0000","last_change":"2024-02-26T10:39:40.025109+0000","last_active":"2024-02-26T10:39:41.004524+0000","last_peered":"2024-02-26T10:39:41.004524+0000","last_clean":"2024-02-26T10:39:41.004524+0000","last_became_active":"2024-02-26T10:39:40.024213+0000","last_became_peered":"2024-02-26T10:39:40.024213+0000","last_unstale":"2024-02-26T10:39:41.004524+0000","last_undegraded":"2024-02-26T10:39:41.004524+0000","last_fullsized":"2024-02-26T10:39:41.004524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:48.049981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010050+0000","last_change":"2024-02-26T10:39:39.003513+0000","last_active":"2024-02-26T10:39:41.010050+0000","last_peered":"2024-02-26T10:39:41.010050+0000","last_clean":"2024-02-26T10:39:41.010050+0000","last_became_active":"2024-02-26T10:39:39.002989+0000","last_became_peered":"2024-02-26T10:39:39.002989+0000","last_unstale":"2024-02-26T10:39:41.010050+0000","last_undegraded":"2024-02-26T10:39:41.010050+0000","last_fullsized":"2024-02-26T10:39:41.010050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:13.624048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.316081+0000","last_change":"2024-02-26T10:39:40.025178+0000","last_active":"2024-02-26T10:39:41.316081+0000","last_peered":"2024-02-26T10:39:41.316081+0000","last_clean":"2024-02-26T10:39:41.316081+0000","last_became_active":"2024-02-26T10:39:40.024856+0000","last_became_peered":"2024-02-26T10:39:40.024856+0000","last_unstale":"2024-02-26T10:39:41.316081+0000","last_undegraded":"2024-02-26T10:39:41.316081+0000","last_fullsized":"2024-02-26T10:39:41.316081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120431+0000","last_change":"2024-02-26T10:39:39.008501+0000","last_active":"2024-02-26T10:39:41.120431+0000","last_peered":"2024-02-26T10:39:41.120431+0000","last_clean":"2024-02-26T10:39:41.120431+0000","last_became_active":"2024-02-26T10:39:39.008223+0000","last_became_peered":"2024-02-26T10:39:39.008223+0000","last_unstale":"2024-02-26T10:39:41.120431+0000","last_undegraded":"2024-02-26T10:39:41.120431+0000","last_fullsized":"2024-02-26T10:39:41.120431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:48.232676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120521+0000","last_change":"2024-02-26T10:39:40.006041+0000","last_active":"2024-02-26T10:39:41.120521+0000","last_peered":"2024-02-26T10:39:41.120521+0000","last_clean":"2024-02-26T10:39:41.120521+0000","last_became_active":"2024-02-26T10:39:40.002138+0000","last_became_peered":"2024-02-26T10:39:40.002138+0000","last_unstale":"2024-02-26T10:39:41.120521+0000","last_undegraded":"2024-02-26T10:39:41.120521+0000","last_fullsized":"2024-02-26T10:39:41.120521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:38.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025749+0000","last_change":"2024-02-26T10:39:39.004487+0000","last_active":"2024-02-26T10:39:41.025749+0000","last_peered":"2024-02-26T10:39:41.025749+0000","last_clean":"2024-02-26T10:39:41.025749+0000","last_became_active":"2024-02-26T10:39:39.004135+0000","last_became_peered":"2024-02-26T10:39:39.004135+0000","last_unstale":"2024-02-26T10:39:41.025749+0000","last_undegraded":"2024-02-26T10:39:41.025749+0000","last_fullsized":"2024-02-26T10:39:41.025749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.120039+0000","last_change":"2024-02-26T10:39:40.006537+0000","last_active":"2024-02-26T10:39:41.120039+0000","last_peered":"2024-02-26T10:39:41.120039+0000","last_clean":"2024-02-26T10:39:41.120039+0000","last_became_active":"2024-02-26T10:39:40.006231+0000","last_became_peered":"2024-02-26T10:39:40.006231+0000","last_unstale":"2024-02-26T10:39:41.120039+0000","last_undegraded":"2024-02-26T10:39:41.120039+0000","last_fullsized":"2024-02-26T10:39:41.120039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315112+0000","last_change":"2024-02-26T10:39:39.010001+0000","last_active":"2024-02-26T10:39:41.315112+0000","last_peered":"2024-02-26T10:39:41.315112+0000","last_clean":"2024-02-26T10:39:41.315112+0000","last_became_active":"2024-02-26T10:39:39.009754+0000","last_became_peered":"2024-02-26T10:39:39.009754+0000","last_unstale":"2024-02-26T10:39:41.315112+0000","last_undegraded":"2024-02-26T10:39:41.315112+0000","last_fullsized":"2024-02-26T10:39:41.315112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314824+0000","last_change":"2024-02-26T10:39:40.009580+0000","last_active":"2024-02-26T10:39:41.314824+0000","last_peered":"2024-02-26T10:39:41.314824+0000","last_clean":"2024-02-26T10:39:41.314824+0000","last_became_active":"2024-02-26T10:39:40.009277+0000","last_became_peered":"2024-02-26T10:39:40.009277+0000","last_unstale":"2024-02-26T10:39:41.314824+0000","last_undegraded":"2024-02-26T10:39:41.314824+0000","last_fullsized":"2024-02-26T10:39:41.314824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119722+0000","last_change":"2024-02-26T10:39:38.999382+0000","last_active":"2024-02-26T10:39:41.119722+0000","last_peered":"2024-02-26T10:39:41.119722+0000","last_clean":"2024-02-26T10:39:41.119722+0000","last_became_active":"2024-02-26T10:39:38.999045+0000","last_became_peered":"2024-02-26T10:39:38.999045+0000","last_unstale":"2024-02-26T10:39:41.119722+0000","last_undegraded":"2024-02-26T10:39:41.119722+0000","last_fullsized":"2024-02-26T10:39:41.119722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:36.656258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004891+0000","last_change":"2024-02-26T10:39:40.018996+0000","last_active":"2024-02-26T10:39:41.004891+0000","last_peered":"2024-02-26T10:39:41.004891+0000","last_clean":"2024-02-26T10:39:41.004891+0000","last_became_active":"2024-02-26T10:39:40.018734+0000","last_became_peered":"2024-02-26T10:39:40.018734+0000","last_unstale":"2024-02-26T10:39:41.004891+0000","last_undegraded":"2024-02-26T10:39:41.004891+0000","last_fullsized":"2024-02-26T10:39:41.004891+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:26.470295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011084+0000","last_change":"2024-02-26T10:39:39.010735+0000","last_active":"2024-02-26T10:39:41.011084+0000","last_peered":"2024-02-26T10:39:41.011084+0000","last_clean":"2024-02-26T10:39:41.011084+0000","last_became_active":"2024-02-26T10:39:39.010394+0000","last_became_peered":"2024-02-26T10:39:39.010394+0000","last_unstale":"2024-02-26T10:39:41.011084+0000","last_undegraded":"2024-02-26T10:39:41.011084+0000","last_fullsized":"2024-02-26T10:39:41.011084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:41:16.074102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119370+0000","last_change":"2024-02-26T10:39:40.006750+0000","last_active":"2024-02-26T10:39:41.119370+0000","last_peered":"2024-02-26T10:39:41.119370+0000","last_clean":"2024-02-26T10:39:41.119370+0000","last_became_active":"2024-02-26T10:39:40.005913+0000","last_became_peered":"2024-02-26T10:39:40.005913+0000","last_unstale":"2024-02-26T10:39:41.119370+0000","last_undegraded":"2024-02-26T10:39:41.119370+0000","last_fullsized":"2024-02-26T10:39:41.119370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004553+0000","last_change":"2024-02-26T10:39:39.000729+0000","last_active":"2024-02-26T10:39:41.004553+0000","last_peered":"2024-02-26T10:39:41.004553+0000","last_clean":"2024-02-26T10:39:41.004553+0000","last_became_active":"2024-02-26T10:39:38.999811+0000","last_became_peered":"2024-02-26T10:39:38.999811+0000","last_unstale":"2024-02-26T10:39:41.004553+0000","last_undegraded":"2024-02-26T10:39:41.004553+0000","last_fullsized":"2024-02-26T10:39:41.004553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217626+0000","last_change":"2024-02-26T10:39:40.008946+0000","last_active":"2024-02-26T10:39:41.217626+0000","last_peered":"2024-02-26T10:39:41.217626+0000","last_clean":"2024-02-26T10:39:41.217626+0000","last_became_active":"2024-02-26T10:39:40.008676+0000","last_became_peered":"2024-02-26T10:39:40.008676+0000","last_unstale":"2024-02-26T10:39:41.217626+0000","last_undegraded":"2024-02-26T10:39:41.217626+0000","last_fullsized":"2024-02-26T10:39:41.217626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119048+0000","last_change":"2024-02-26T10:39:39.008406+0000","last_active":"2024-02-26T10:39:41.119048+0000","last_peered":"2024-02-26T10:39:41.119048+0000","last_clean":"2024-02-26T10:39:41.119048+0000","last_became_active":"2024-02-26T10:39:39.008086+0000","last_became_peered":"2024-02-26T10:39:39.008086+0000","last_unstale":"2024-02-26T10:39:41.119048+0000","last_undegraded":"2024-02-26T10:39:41.119048+0000","last_fullsized":"2024-02-26T10:39:41.119048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314267+0000","last_change":"2024-02-26T10:39:40.008604+0000","last_active":"2024-02-26T10:39:41.314267+0000","last_peered":"2024-02-26T10:39:41.314267+0000","last_clean":"2024-02-26T10:39:41.314267+0000","last_became_active":"2024-02-26T10:39:40.008244+0000","last_became_peered":"2024-02-26T10:39:40.008244+0000","last_unstale":"2024-02-26T10:39:41.314267+0000","last_undegraded":"2024-02-26T10:39:41.314267+0000","last_fullsized":"2024-02-26T10:39:41.314267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119002+0000","last_change":"2024-02-26T10:39:39.002498+0000","last_active":"2024-02-26T10:39:41.119002+0000","last_peered":"2024-02-26T10:39:41.119002+0000","last_clean":"2024-02-26T10:39:41.119002+0000","last_became_active":"2024-02-26T10:39:39.002185+0000","last_became_peered":"2024-02-26T10:39:39.002185+0000","last_unstale":"2024-02-26T10:39:41.119002+0000","last_undegraded":"2024-02-26T10:39:41.119002+0000","last_fullsized":"2024-02-26T10:39:41.119002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:18.706456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011518+0000","last_change":"2024-02-26T10:39:40.026290+0000","last_active":"2024-02-26T10:39:41.011518+0000","last_peered":"2024-02-26T10:39:41.011518+0000","last_clean":"2024-02-26T10:39:41.011518+0000","last_became_active":"2024-02-26T10:39:40.026004+0000","last_became_peered":"2024-02-26T10:39:40.026004+0000","last_unstale":"2024-02-26T10:39:41.011518+0000","last_undegraded":"2024-02-26T10:39:41.011518+0000","last_fullsized":"2024-02-26T10:39:41.011518+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006717+0000","last_change":"2024-02-26T10:39:39.001118+0000","last_active":"2024-02-26T10:39:41.006717+0000","last_peered":"2024-02-26T10:39:41.006717+0000","last_clean":"2024-02-26T10:39:41.006717+0000","last_became_active":"2024-02-26T10:39:39.000747+0000","last_became_peered":"2024-02-26T10:39:39.000747+0000","last_unstale":"2024-02-26T10:39:41.006717+0000","last_undegraded":"2024-02-26T10:39:41.006717+0000","last_fullsized":"2024-02-26T10:39:41.006717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:54.239568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315538+0000","last_change":"2024-02-26T10:39:40.017294+0000","last_active":"2024-02-26T10:39:41.315538+0000","last_peered":"2024-02-26T10:39:41.315538+0000","last_clean":"2024-02-26T10:39:41.315538+0000","last_became_active":"2024-02-26T10:39:40.016268+0000","last_became_peered":"2024-02-26T10:39:40.016268+0000","last_unstale":"2024-02-26T10:39:41.315538+0000","last_undegraded":"2024-02-26T10:39:41.315538+0000","last_fullsized":"2024-02-26T10:39:41.315538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011287+0000","last_change":"2024-02-26T10:39:39.008983+0000","last_active":"2024-02-26T10:39:41.011287+0000","last_peered":"2024-02-26T10:39:41.011287+0000","last_clean":"2024-02-26T10:39:41.011287+0000","last_became_active":"2024-02-26T10:39:39.008687+0000","last_became_peered":"2024-02-26T10:39:39.008687+0000","last_unstale":"2024-02-26T10:39:41.011287+0000","last_undegraded":"2024-02-26T10:39:41.011287+0000","last_fullsized":"2024-02-26T10:39:41.011287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:08:52.607215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.005393+0000","last_change":"2024-02-26T10:39:40.005719+0000","last_active":"2024-02-26T10:39:41.005393+0000","last_peered":"2024-02-26T10:39:41.005393+0000","last_clean":"2024-02-26T10:39:41.005393+0000","last_became_active":"2024-02-26T10:39:40.005399+0000","last_became_peered":"2024-02-26T10:39:40.005399+0000","last_unstale":"2024-02-26T10:39:41.005393+0000","last_undegraded":"2024-02-26T10:39:41.005393+0000","last_fullsized":"2024-02-26T10:39:41.005393+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:40.993108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011230+0000","last_change":"2024-02-26T10:39:38.997283+0000","last_active":"2024-02-26T10:39:41.011230+0000","last_peered":"2024-02-26T10:39:41.011230+0000","last_clean":"2024-02-26T10:39:41.011230+0000","last_became_active":"2024-02-26T10:39:38.996974+0000","last_became_peered":"2024-02-26T10:39:38.996974+0000","last_unstale":"2024-02-26T10:39:41.011230+0000","last_undegraded":"2024-02-26T10:39:41.011230+0000","last_fullsized":"2024-02-26T10:39:41.011230+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:48.227511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008102+0000","last_change":"2024-02-26T10:39:40.014954+0000","last_active":"2024-02-26T10:39:41.008102+0000","last_peered":"2024-02-26T10:39:41.008102+0000","last_clean":"2024-02-26T10:39:41.008102+0000","last_became_active":"2024-02-26T10:39:40.014657+0000","last_became_peered":"2024-02-26T10:39:40.014657+0000","last_unstale":"2024-02-26T10:39:41.008102+0000","last_undegraded":"2024-02-26T10:39:41.008102+0000","last_fullsized":"2024-02-26T10:39:41.008102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:38:13.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.011102+0000","last_change":"2024-02-26T10:39:39.005544+0000","last_active":"2024-02-26T10:39:41.011102+0000","last_peered":"2024-02-26T10:39:41.011102+0000","last_clean":"2024-02-26T10:39:41.011102+0000","last_became_active":"2024-02-26T10:39:39.005194+0000","last_became_peered":"2024-02-26T10:39:39.005194+0000","last_unstale":"2024-02-26T10:39:41.011102+0000","last_undegraded":"2024-02-26T10:39:41.011102+0000","last_fullsized":"2024-02-26T10:39:41.011102+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:51:47.714809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008058+0000","last_change":"2024-02-26T10:39:39.997097+0000","last_active":"2024-02-26T10:39:41.008058+0000","last_peered":"2024-02-26T10:39:41.008058+0000","last_clean":"2024-02-26T10:39:41.008058+0000","last_became_active":"2024-02-26T10:39:39.996720+0000","last_became_peered":"2024-02-26T10:39:39.996720+0000","last_unstale":"2024-02-26T10:39:41.008058+0000","last_undegraded":"2024-02-26T10:39:41.008058+0000","last_fullsized":"2024-02-26T10:39:41.008058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:12.318898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025244+0000","last_change":"2024-02-26T10:39:38.992320+0000","last_active":"2024-02-26T10:39:41.025244+0000","last_peered":"2024-02-26T10:39:41.025244+0000","last_clean":"2024-02-26T10:39:41.025244+0000","last_became_active":"2024-02-26T10:39:38.991970+0000","last_became_peered":"2024-02-26T10:39:38.991970+0000","last_unstale":"2024-02-26T10:39:41.025244+0000","last_undegraded":"2024-02-26T10:39:41.025244+0000","last_fullsized":"2024-02-26T10:39:41.025244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008634+0000","last_change":"2024-02-26T10:39:40.002307+0000","last_active":"2024-02-26T10:39:41.008634+0000","last_peered":"2024-02-26T10:39:41.008634+0000","last_clean":"2024-02-26T10:39:41.008634+0000","last_became_active":"2024-02-26T10:39:40.001899+0000","last_became_peered":"2024-02-26T10:39:40.001899+0000","last_unstale":"2024-02-26T10:39:41.008634+0000","last_undegraded":"2024-02-26T10:39:41.008634+0000","last_fullsized":"2024-02-26T10:39:41.008634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:23.314983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314280+0000","last_change":"2024-02-26T10:39:39.011100+0000","last_active":"2024-02-26T10:39:41.314280+0000","last_peered":"2024-02-26T10:39:41.314280+0000","last_clean":"2024-02-26T10:39:41.314280+0000","last_became_active":"2024-02-26T10:39:39.010704+0000","last_became_peered":"2024-02-26T10:39:39.010704+0000","last_unstale":"2024-02-26T10:39:41.314280+0000","last_undegraded":"2024-02-26T10:39:41.314280+0000","last_fullsized":"2024-02-26T10:39:41.314280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008444+0000","last_change":"2024-02-26T10:39:40.001953+0000","last_active":"2024-02-26T10:39:41.008444+0000","last_peered":"2024-02-26T10:39:41.008444+0000","last_clean":"2024-02-26T10:39:41.008444+0000","last_became_active":"2024-02-26T10:39:40.001670+0000","last_became_peered":"2024-02-26T10:39:40.001670+0000","last_unstale":"2024-02-26T10:39:41.008444+0000","last_undegraded":"2024-02-26T10:39:41.008444+0000","last_fullsized":"2024-02-26T10:39:41.008444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:07.672401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010235+0000","last_change":"2024-02-26T10:39:39.000283+0000","last_active":"2024-02-26T10:39:41.010235+0000","last_peered":"2024-02-26T10:39:41.010235+0000","last_clean":"2024-02-26T10:39:41.010235+0000","last_became_active":"2024-02-26T10:39:38.999392+0000","last_became_peered":"2024-02-26T10:39:38.999392+0000","last_unstale":"2024-02-26T10:39:41.010235+0000","last_undegraded":"2024-02-26T10:39:41.010235+0000","last_fullsized":"2024-02-26T10:39:41.010235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:42.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217147+0000","last_change":"2024-02-26T10:39:40.013051+0000","last_active":"2024-02-26T10:39:41.217147+0000","last_peered":"2024-02-26T10:39:41.217147+0000","last_clean":"2024-02-26T10:39:41.217147+0000","last_became_active":"2024-02-26T10:39:40.012739+0000","last_became_peered":"2024-02-26T10:39:40.012739+0000","last_unstale":"2024-02-26T10:39:41.217147+0000","last_undegraded":"2024-02-26T10:39:41.217147+0000","last_fullsized":"2024-02-26T10:39:41.217147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.314235+0000","last_change":"2024-02-26T10:39:39.009824+0000","last_active":"2024-02-26T10:39:41.314235+0000","last_peered":"2024-02-26T10:39:41.314235+0000","last_clean":"2024-02-26T10:39:41.314235+0000","last_became_active":"2024-02-26T10:39:39.009559+0000","last_became_peered":"2024-02-26T10:39:39.009559+0000","last_unstale":"2024-02-26T10:39:41.314235+0000","last_undegraded":"2024-02-26T10:39:41.314235+0000","last_fullsized":"2024-02-26T10:39:41.314235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.008781+0000","last_change":"2024-02-26T10:39:40.022558+0000","last_active":"2024-02-26T10:39:41.008781+0000","last_peered":"2024-02-26T10:39:41.008781+0000","last_clean":"2024-02-26T10:39:41.008781+0000","last_became_active":"2024-02-26T10:39:40.022316+0000","last_became_peered":"2024-02-26T10:39:40.022316+0000","last_unstale":"2024-02-26T10:39:41.008781+0000","last_undegraded":"2024-02-26T10:39:41.008781+0000","last_fullsized":"2024-02-26T10:39:41.008781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:11:06.614146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.006822+0000","last_change":"2024-02-26T10:39:39.009948+0000","last_active":"2024-02-26T10:39:41.006822+0000","last_peered":"2024-02-26T10:39:41.006822+0000","last_clean":"2024-02-26T10:39:41.006822+0000","last_became_active":"2024-02-26T10:39:39.009640+0000","last_became_peered":"2024-02-26T10:39:39.009640+0000","last_unstale":"2024-02-26T10:39:41.006822+0000","last_undegraded":"2024-02-26T10:39:41.006822+0000","last_fullsized":"2024-02-26T10:39:41.006822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:47.192375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315599+0000","last_change":"2024-02-26T10:39:40.009603+0000","last_active":"2024-02-26T10:39:41.315599+0000","last_peered":"2024-02-26T10:39:41.315599+0000","last_clean":"2024-02-26T10:39:41.315599+0000","last_became_active":"2024-02-26T10:39:40.009327+0000","last_became_peered":"2024-02-26T10:39:40.009327+0000","last_unstale":"2024-02-26T10:39:41.315599+0000","last_undegraded":"2024-02-26T10:39:41.315599+0000","last_fullsized":"2024-02-26T10:39:41.315599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025692+0000","last_change":"2024-02-26T10:39:38.992861+0000","last_active":"2024-02-26T10:39:41.025692+0000","last_peered":"2024-02-26T10:39:41.025692+0000","last_clean":"2024-02-26T10:39:41.025692+0000","last_became_active":"2024-02-26T10:39:38.992258+0000","last_became_peered":"2024-02-26T10:39:38.992258+0000","last_unstale":"2024-02-26T10:39:41.025692+0000","last_undegraded":"2024-02-26T10:39:41.025692+0000","last_fullsized":"2024-02-26T10:39:41.025692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025153+0000","last_change":"2024-02-26T10:39:40.009958+0000","last_active":"2024-02-26T10:39:41.025153+0000","last_peered":"2024-02-26T10:39:41.025153+0000","last_clean":"2024-02-26T10:39:41.025153+0000","last_became_active":"2024-02-26T10:39:40.009557+0000","last_became_peered":"2024-02-26T10:39:40.009557+0000","last_unstale":"2024-02-26T10:39:41.025153+0000","last_undegraded":"2024-02-26T10:39:41.025153+0000","last_fullsized":"2024-02-26T10:39:41.025153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.005621+0000","last_change":"2024-02-26T10:39:38.996371+0000","last_active":"2024-02-26T10:39:41.005621+0000","last_peered":"2024-02-26T10:39:41.005621+0000","last_clean":"2024-02-26T10:39:41.005621+0000","last_became_active":"2024-02-26T10:39:38.996052+0000","last_became_peered":"2024-02-26T10:39:38.996052+0000","last_unstale":"2024-02-26T10:39:41.005621+0000","last_undegraded":"2024-02-26T10:39:41.005621+0000","last_fullsized":"2024-02-26T10:39:41.005621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.315022+0000","last_change":"2024-02-26T10:39:40.008904+0000","last_active":"2024-02-26T10:39:41.315022+0000","last_peered":"2024-02-26T10:39:41.315022+0000","last_clean":"2024-02-26T10:39:41.315022+0000","last_became_active":"2024-02-26T10:39:40.008631+0000","last_became_peered":"2024-02-26T10:39:40.008631+0000","last_unstale":"2024-02-26T10:39:41.315022+0000","last_undegraded":"2024-02-26T10:39:41.315022+0000","last_fullsized":"2024-02-26T10:39:41.315022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025549+0000","last_change":"2024-02-26T10:39:39.003861+0000","last_active":"2024-02-26T10:39:41.025549+0000","last_peered":"2024-02-26T10:39:41.025549+0000","last_clean":"2024-02-26T10:39:41.025549+0000","last_became_active":"2024-02-26T10:39:39.003370+0000","last_became_peered":"2024-02-26T10:39:39.003370+0000","last_unstale":"2024-02-26T10:39:41.025549+0000","last_undegraded":"2024-02-26T10:39:41.025549+0000","last_fullsized":"2024-02-26T10:39:41.025549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.025536+0000","last_change":"2024-02-26T10:39:40.011820+0000","last_active":"2024-02-26T10:39:41.025536+0000","last_peered":"2024-02-26T10:39:41.025536+0000","last_clean":"2024-02-26T10:39:41.025536+0000","last_became_active":"2024-02-26T10:39:40.011511+0000","last_became_peered":"2024-02-26T10:39:40.011511+0000","last_unstale":"2024-02-26T10:39:41.025536+0000","last_undegraded":"2024-02-26T10:39:41.025536+0000","last_fullsized":"2024-02-26T10:39:41.025536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.004696+0000","last_change":"2024-02-26T10:39:39.013362+0000","last_active":"2024-02-26T10:39:41.004696+0000","last_peered":"2024-02-26T10:39:41.004696+0000","last_clean":"2024-02-26T10:39:41.004696+0000","last_became_active":"2024-02-26T10:39:39.012819+0000","last_became_peered":"2024-02-26T10:39:39.012819+0000","last_unstale":"2024-02-26T10:39:41.004696+0000","last_undegraded":"2024-02-26T10:39:41.004696+0000","last_fullsized":"2024-02-26T10:39:41.004696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.118859+0000","last_change":"2024-02-26T10:39:39.997501+0000","last_active":"2024-02-26T10:39:41.118859+0000","last_peered":"2024-02-26T10:39:41.118859+0000","last_clean":"2024-02-26T10:39:41.118859+0000","last_became_active":"2024-02-26T10:39:39.996084+0000","last_became_peered":"2024-02-26T10:39:39.996084+0000","last_unstale":"2024-02-26T10:39:41.118859+0000","last_undegraded":"2024-02-26T10:39:41.118859+0000","last_fullsized":"2024-02-26T10:39:41.118859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010496+0000","last_change":"2024-02-26T10:39:38.999680+0000","last_active":"2024-02-26T10:39:41.010496+0000","last_peered":"2024-02-26T10:39:41.010496+0000","last_clean":"2024-02-26T10:39:41.010496+0000","last_became_active":"2024-02-26T10:39:38.999058+0000","last_became_peered":"2024-02-26T10:39:38.999058+0000","last_unstale":"2024-02-26T10:39:41.010496+0000","last_undegraded":"2024-02-26T10:39:41.010496+0000","last_fullsized":"2024-02-26T10:39:41.010496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:26.840527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.010324+0000","last_change":"2024-02-26T10:39:39.995364+0000","last_active":"2024-02-26T10:39:41.010324+0000","last_peered":"2024-02-26T10:39:41.010324+0000","last_clean":"2024-02-26T10:39:41.010324+0000","last_became_active":"2024-02-26T10:39:39.995077+0000","last_became_peered":"2024-02-26T10:39:39.995077+0000","last_unstale":"2024-02-26T10:39:41.010324+0000","last_undegraded":"2024-02-26T10:39:41.010324+0000","last_fullsized":"2024-02-26T10:39:41.010324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.119421+0000","last_change":"2024-02-26T10:39:39.006896+0000","last_active":"2024-02-26T10:39:41.119421+0000","last_peered":"2024-02-26T10:39:41.119421+0000","last_clean":"2024-02-26T10:39:41.119421+0000","last_became_active":"2024-02-26T10:39:39.006604+0000","last_became_peered":"2024-02-26T10:39:39.006604+0000","last_unstale":"2024-02-26T10:39:41.119421+0000","last_undegraded":"2024-02-26T10:39:41.119421+0000","last_fullsized":"2024-02-26T10:39:41.119421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.217544+0000","last_change":"2024-02-26T10:39:40.018706+0000","last_active":"2024-02-26T10:39:41.217544+0000","last_peered":"2024-02-26T10:39:41.217544+0000","last_clean":"2024-02-26T10:39:41.217544+0000","last_became_active":"2024-02-26T10:39:40.018417+0000","last_became_peered":"2024-02-26T10:39:40.018417+0000","last_unstale":"2024-02-26T10:39:41.217544+0000","last_undegraded":"2024-02-26T10:39:41.217544+0000","last_fullsized":"2024-02-26T10:39:41.217544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.313945+0000","last_change":"2024-02-26T10:39:39.013277+0000","last_active":"2024-02-26T10:39:41.313945+0000","last_peered":"2024-02-26T10:39:41.313945+0000","last_clean":"2024-02-26T10:39:41.313945+0000","last_became_active":"2024-02-26T10:39:39.012073+0000","last_became_peered":"2024-02-26T10:39:39.012073+0000","last_unstale":"2024-02-26T10:39:41.313945+0000","last_undegraded":"2024-02-26T10:39:41.313945+0000","last_fullsized":"2024-02-26T10:39:41.313945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:44.421668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-02-26T10:39:41.218047+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:39:41.218047+0000","last_peered":"2024-02-26T10:39:41.218047+0000","last_clean":"2024-02-26T10:39:41.218047+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:39:41.218047+0000","last_undegraded":"2024-02-26T10:39:41.218047+0000","last_fullsized":"2024-02-26T10:39:41.218047+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":740369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":151545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":149981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":151545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":740279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:39:45.010 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:39:45.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-26T10:39:45.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.280+0000 7fecfd400640 1 Processor -- start 2024-02-26T10:39:45.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.281+0000 7fecfd400640 1 -- start start 2024-02-26T10:39:45.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.282+0000 7fecfd400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fecf8072dd0 0x7fecf806e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.282+0000 7fecfd400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecf8152ea0 0x7fecf81532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.282+0000 7fecfd400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf806eb60 con 0x7fecf813c000 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.282+0000 7fecfd400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf8073260 con 0x7fecf8072dd0 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.282+0000 7fecfd400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf81564b0 con 0x7fecf8152ea0 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fecf8072dd0 0x7fecf806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fecf8072dd0 0x7fecf806e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42200/0 (socket says 172.21.15.67:42200) 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 -- 172.21.15.67:0/3072781564 learned_addr learned my addr 172.21.15.67:0/3072781564 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecfca00640 1 --2- 172.21.15.67:0/3072781564 >> v2:172.21.15.82:3300/0 conn(0x7fecf8152ea0 0x7fecf81532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 -- 172.21.15.67:0/3072781564 >> v2:172.21.15.82:3300/0 conn(0x7fecf8152ea0 msgr2=0x7fecf81532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 --2- 172.21.15.67:0/3072781564 >> v2:172.21.15.82:3300/0 conn(0x7fecf8152ea0 0x7fecf81532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 -- 172.21.15.67:0/3072781564 >> v1:172.21.15.82:6789/0 conn(0x7fecf813c000 legacy=0x7fecf80c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:45.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.283+0000 7fecf7400640 1 -- 172.21.15.67:0/3072781564 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8156790 con 0x7fecf8072dd0 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.285+0000 7fecf7400640 1 --2- 172.21.15.67:0/3072781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf806e460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecf806c8f0 tx=0x7fece402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad67d8e5c404e1e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.285+0000 7fecf6a00640 1 -- 172.21.15.67:0/3072781564 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece402e070 con 0x7fecf8072dd0 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.285+0000 7fecf6a00640 1 -- 172.21.15.67:0/3072781564 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4037070 con 0x7fecf8072dd0 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.285+0000 7fecf6a00640 1 -- 172.21.15.67:0/3072781564 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4031a00 con 0x7fecf8072dd0 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 -- 172.21.15.67:0/3072781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 msgr2=0x7fecf806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 --2- 172.21.15.67:0/3072781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf806e460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecf806c8f0 tx=0x7fece402b880 comp rx=0 tx=0).stop 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 -- 172.21.15.67:0/3072781564 shutdown_connections 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 --2- 172.21.15.67:0/3072781564 >> v2:172.21.15.82:3300/0 conn(0x7fecf8152ea0 0x7fecf81532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 --2- 172.21.15.67:0/3072781564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf806e460 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 -- 172.21.15.67:0/3072781564 >> 172.21.15.67:0/3072781564 conn(0x7fecf80a84a0 msgr2=0x7fecf80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 -- 172.21.15.67:0/3072781564 shutdown_connections 2024-02-26T10:39:45.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.286+0000 7fecfd400640 1 -- 172.21.15.67:0/3072781564 wait complete. 2024-02-26T10:39:45.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.288+0000 7fecfd400640 1 Processor -- start 2024-02-26T10:39:45.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.288+0000 7fecfd400640 1 -- start start 2024-02-26T10:39:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecfd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf8139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecfd400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf813c000 0x7fecf8154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecfd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf8152ea0 0x7fecf80a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecfd400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf8157270 con 0x7fecf8072dd0 2024-02-26T10:39:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecfd400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf8156d50 con 0x7fecf8152ea0 2024-02-26T10:39:45.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecfd400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf813c3e0 con 0x7fecf813c000 2024-02-26T10:39:45.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf8139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf8139c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42212/0 (socket says 172.21.15.67:42212) 2024-02-26T10:39:45.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.289+0000 7fecf7e00640 1 -- 172.21.15.67:0/1361320479 learned_addr learned my addr 172.21.15.67:0/1361320479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecfca00640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf8152ea0 0x7fecf80a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf813c000 0x7fecf8154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7e00640 1 -- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf813c000 msgr2=0x7fecf8154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7e00640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf813c000 0x7fecf8154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7e00640 1 -- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf8152ea0 msgr2=0x7fecf80a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7e00640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf8152ea0 0x7fecf80a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7e00640 1 -- 172.21.15.67:0/1361320479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8154660 con 0x7fecf8072dd0 2024-02-26T10:39:45.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecfca00640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf8152ea0 0x7fecf80a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.290+0000 7fecf7e00640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf8139c30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fece800b3f0 tx=0x7fece800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.291+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8015020 con 0x7fecf8072dd0 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.291+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece8011830 con 0x7fecf8072dd0 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.291+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8011c20 con 0x7fecf8072dd0 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.291+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf813b780 con 0x7fecf8072dd0 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.291+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf813b4a0 con 0x7fecf8072dd0 2024-02-26T10:39:45.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.292+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf8074a30 con 0x7fecf8072dd0 2024-02-26T10:39:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.293+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fece8018020 con 0x7fecf8072dd0 2024-02-26T10:39:45.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.294+0000 7fecefe00640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fecd806b4d0 0x7fecd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.294+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fece8089d70 con 0x7fecf8072dd0 2024-02-26T10:39:45.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.294+0000 7fecf7400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fecd806b4d0 0x7fecd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.296+0000 7fecf7400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fecd806b4d0 0x7fecd806d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fece4036000 tx=0x7fece4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:45.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.299+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fece801f030 con 0x7fecf8072dd0 2024-02-26T10:39:45.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.567+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fecf80bfd70 con 0x7fecf8072dd0 2024-02-26T10:39:45.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.571+0000 7fecefe00640 1 -- 172.21.15.67:0/1361320479 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fece80172c0 con 0x7fecf8072dd0 2024-02-26T10:39:45.573 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-02-26T10:39:45.573 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw== 2024-02-26T10:39:45.573 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-02-26T10:39:45.573 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:39:45.573 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-02-26T10:39:45.573 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fecd806b4d0 msgr2=0x7fecd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fecd806b4d0 0x7fecd806d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fece4036000 tx=0x7fece4008030 comp rx=0 tx=0).stop 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 msgr2=0x7fecf8139c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf8139c30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fece800b3f0 tx=0x7fece800b8b0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 shutdown_connections 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fecd806b4d0 0x7fecd806d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.574+0000 7fecfd400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf8152ea0 0x7fecf80a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.575+0000 7fecfd400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf813c000 0x7fecf8154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.575+0000 7fecfd400640 1 --2- 172.21.15.67:0/1361320479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf8072dd0 0x7fecf8139c30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.575+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 >> 172.21.15.67:0/1361320479 conn(0x7fecf80a84a0 msgr2=0x7fecf80a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:45.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.575+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 shutdown_connections 2024-02-26T10:39:45.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.575+0000 7fecfd400640 1 -- 172.21.15.67:0/1361320479 wait complete. 2024-02-26T10:39:45.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:45.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.796+0000 7f1505e00640 1 Processor -- start 2024-02-26T10:39:45.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.796+0000 7f1505e00640 1 -- start start 2024-02-26T10:39:45.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.797+0000 7f1505e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1500152420 0x7f15000c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.797+0000 7f1505e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000bfa30 0x7f1500158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.797+0000 7f1505e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15000c15e0 con 0x7f15000c35d0 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.797+0000 7f1505e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150006e660 con 0x7f1500152420 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.797+0000 7f1505e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150006e920 con 0x7f15000bfa30 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1500152420 0x7f15000c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1500152420 0x7f15000c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42222/0 (socket says 172.21.15.67:42222) 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 -- 172.21.15.67:0/2489643710 learned_addr learned my addr 172.21.15.67:0/2489643710 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 -- 172.21.15.67:0/2489643710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000bfa30 msgr2=0x7f1500158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 --2- 172.21.15.67:0/2489643710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000bfa30 0x7f1500158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 -- 172.21.15.67:0/2489643710 >> v1:172.21.15.82:6789/0 conn(0x7f15000c35d0 legacy=0x7f150006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:45.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.798+0000 7f14fea00640 1 -- 172.21.15.67:0/2489643710 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000c1880 con 0x7f1500152420 2024-02-26T10:39:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.799+0000 7f14fea00640 1 --2- 172.21.15.67:0/2489643710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1500152420 0x7f15000c3090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009870 tx=0x7f14f002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac35d217522d65a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f14fe000640 1 -- 172.21.15.67:0/2489643710 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f00317c0 con 0x7f1500152420 2024-02-26T10:39:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f14fe000640 1 -- 172.21.15.67:0/2489643710 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0037070 con 0x7f1500152420 2024-02-26T10:39:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f14fe000640 1 -- 172.21.15.67:0/2489643710 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0031b80 con 0x7f1500152420 2024-02-26T10:39:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 -- 172.21.15.67:0/2489643710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1500152420 msgr2=0x7f15000c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 --2- 172.21.15.67:0/2489643710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1500152420 0x7f15000c3090 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009870 tx=0x7f14f002b860 comp rx=0 tx=0).stop 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 -- 172.21.15.67:0/2489643710 shutdown_connections 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 --2- 172.21.15.67:0/2489643710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000bfa30 0x7f1500158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 --2- 172.21.15.67:0/2489643710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1500152420 0x7f15000c3090 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 -- 172.21.15.67:0/2489643710 >> 172.21.15.67:0/2489643710 conn(0x7f15000a84a0 msgr2=0x7f15000a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 -- 172.21.15.67:0/2489643710 shutdown_connections 2024-02-26T10:39:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.800+0000 7f1505e00640 1 -- 172.21.15.67:0/2489643710 wait complete. 2024-02-26T10:39:45.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.802+0000 7f1505e00640 1 Processor -- start 2024-02-26T10:39:45.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.802+0000 7f1505e00640 1 -- start start 2024-02-26T10:39:45.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f1505e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15000bfa30 0x7f15000c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f1505e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 0x7f15000c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f14f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 0x7f15000c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f1505e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500152420 0x7f1500154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f1505e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500152800 con 0x7f15000c35d0 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f1505e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500158400 con 0x7f15000bfa30 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f1505e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1500154e10 con 0x7f1500152420 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f14f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 0x7f15000c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42228/0 (socket says 172.21.15.67:42228) 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.803+0000 7f14f6000640 1 -- 172.21.15.67:0/3563922922 learned_addr learned my addr 172.21.15.67:0/3563922922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14fea00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15000bfa30 0x7f15000c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f6000640 1 -- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500152420 msgr2=0x7f1500154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f6000640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500152420 0x7f1500154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f6000640 1 -- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15000bfa30 msgr2=0x7f15000c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f6000640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15000bfa30 0x7f15000c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f6000640 1 -- 172.21.15.67:0/3563922922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000c4320 con 0x7f15000c35d0 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f6000640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 0x7f15000c6cd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b440 tx=0x7f14e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8015020 con 0x7f15000c35d0 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.804+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e80118b0 con 0x7f15000c35d0 2024-02-26T10:39:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.805+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011ca0 con 0x7f15000c35d0 2024-02-26T10:39:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.805+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15000c4600 con 0x7f15000c35d0 2024-02-26T10:39:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.805+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15000c7210 con 0x7f15000c35d0 2024-02-26T10:39:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.806+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1500074a30 con 0x7f15000c35d0 2024-02-26T10:39:45.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.807+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14e8018020 con 0x7f15000c35d0 2024-02-26T10:39:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.807+0000 7f14f7e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14dc06b4d0 0x7f14dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:45.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.808+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14e808a2e0 con 0x7f15000c35d0 2024-02-26T10:39:45.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.808+0000 7f14fea00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14dc06b4d0 0x7f14dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:45.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.809+0000 7f14fea00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14dc06b4d0 0x7f14dc06d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f14f0036000 tx=0x7f14f003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:45.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:45.812+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14e801f020 con 0x7f15000c35d0 2024-02-26T10:39:46.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.050+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15000c47d0 con 0x7f15000c35d0 2024-02-26T10:39:46.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.051+0000 7f14f7e00640 1 -- 172.21.15.67:0/3563922922 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f14e800d040 con 0x7f15000c35d0 2024-02-26T10:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-26T10:39:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.053+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14dc06b4d0 msgr2=0x7f14dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.053+0000 7f1505e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14dc06b4d0 0x7f14dc06d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f14f0036000 tx=0x7f14f003d040 comp rx=0 tx=0).stop 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.053+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 msgr2=0x7f15000c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.053+0000 7f1505e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 0x7f15000c6cd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14e800b440 tx=0x7f14e800b900 comp rx=0 tx=0).stop 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 shutdown_connections 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500152420 0x7f1500154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000c35d0 0x7f15000c6cd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14dc06b4d0 0x7f14dc06d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 --2- 172.21.15.67:0/3563922922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15000bfa30 0x7f15000c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 >> 172.21.15.67:0/3563922922 conn(0x7f15000a84a0 msgr2=0x7f1500151030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 shutdown_connections 2024-02-26T10:39:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.054+0000 7f1505e00640 1 -- 172.21.15.67:0/3563922922 wait complete. 2024-02-26T10:39:46.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-26T10:39:39.463009+0000', 'modified': '2024-02-26T10:39:42.519117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4753}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4753': {'gid': 4753, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/155874470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 155874470}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 155874470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T10:39:46.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-26T10:39:46.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-02-26T10:39:46.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.705+0000 7f0a27e00640 1 Processor -- start 2024-02-26T10:39:46.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.705+0000 7f0a27e00640 1 -- start start 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 0x7f0a280a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0a2806b000 0x7f0a2806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2806b460 con 0x7f0a28153610 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a280a9a80 con 0x7f0a2806b000 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280a9d60 con 0x7f0a2806ac20 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0a28153610 0x7f0a280abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49292/0 (socket says 172.21.15.67:49292) 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.707+0000 7f0a27400640 1 -- 172.21.15.67:0/1202295062 learned_addr learned my addr 172.21.15.67:0/1202295062 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 0x7f0a280a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/1202295062 >> v2:172.21.15.82:3300/0 conn(0x7f0a2806b000 0x7f0a2806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487550067 0 0) 0x7f0a2806b460 con 0x7f0a28153610 2024-02-26T10:39:46.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a0c003520 con 0x7f0a28153610 2024-02-26T10:39:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1957556588 0 0) 0x7f0a0c003520 con 0x7f0a28153610 2024-02-26T10:39:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 msgr2=0x7f0a280a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 0x7f0a280a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 >> v2:172.21.15.82:3300/0 conn(0x7f0a2806b000 msgr2=0x7f0a2806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 --2- 172.21.15.67:0/1202295062 >> v2:172.21.15.82:3300/0 conn(0x7f0a2806b000 0x7f0a2806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280ab750 con 0x7f0a28153610 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f0a200019f0 con 0x7f0a28153610 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.708+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 >> v1:172.21.15.67:6789/0 conn(0x7f0a28153610 legacy=0x7f0a280abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26000640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a0c0039b0 0x7f0a0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26000640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 0x7f0a0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26000640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a0c004d40 0x7f0a0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280a9a80 con 0x7f0a0c0039b0 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280a9d60 con 0x7f0a0c004d40 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c00b9c0 con 0x7f0a2806ac20 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a0c0039b0 0x7f0a0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26a00640 1 -- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 msgr2=0x7f0a0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a2806ac20 0x7f0a0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26a00640 1 -- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a0c004d40 msgr2=0x7f0a0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a0c004d40 0x7f0a0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.709+0000 7f0a26a00640 1 -- 172.21.15.67:0/1202295062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c003520 con 0x7f0a0c0039b0 2024-02-26T10:39:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.710+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a0c0039b0 0x7f0a0c003dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009ac0 tx=0x7f0a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcf0f5e06baf0d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.710+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18039070 con 0x7f0a0c0039b0 2024-02-26T10:39:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1802c710 con 0x7f0a0c0039b0 2024-02-26T10:39:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a26000640 1 -- 172.21.15.67:0/1202295062 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18009690 con 0x7f0a0c0039b0 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27e00640 1 -- 172.21.15.67:0/1202295062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a0c0039b0 msgr2=0x7f0a0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27e00640 1 --2- 172.21.15.67:0/1202295062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a0c0039b0 0x7f0a0c003dd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009ac0 tx=0x7f0a0c005830 comp rx=0 tx=0).stop 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27400640 1 -- 172.21.15.67:0/1202295062 reap_dead start 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27e00640 1 -- 172.21.15.67:0/1202295062 shutdown_connections 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27e00640 1 -- 172.21.15.67:0/1202295062 >> 172.21.15.67:0/1202295062 conn(0x7f0a280c63f0 msgr2=0x7f0a280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27e00640 1 -- 172.21.15.67:0/1202295062 shutdown_connections 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.711+0000 7f0a27e00640 1 -- 172.21.15.67:0/1202295062 wait complete. 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.713+0000 7f0a27e00640 1 Processor -- start 2024-02-26T10:39:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.713+0000 7f0a27e00640 1 -- start start 2024-02-26T10:39:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a281ee410 0x7f0a281ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 0x7f0a28071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a27e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a28147990 0x7f0a28147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a27e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28150410 con 0x7f0a28071660 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a27e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28153aa0 con 0x7f0a28147990 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a27e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28150590 con 0x7f0a281ee410 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 0x7f0a28071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 0x7f0a28071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42244/0 (socket says 172.21.15.67:42244) 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.714+0000 7f0a1fe00640 1 -- 172.21.15.67:0/14306659 learned_addr learned my addr 172.21.15.67:0/14306659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1fe00640 1 -- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a281ee410 msgr2=0x7f0a281ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a26a00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a281ee410 0x7f0a281ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a27400640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a28147990 0x7f0a28147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a281ee410 0x7f0a281ee7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1fe00640 1 -- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a28147990 msgr2=0x7f0a28147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a28147990 0x7f0a28147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1fe00640 1 -- 172.21.15.67:0/14306659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280c58c0 con 0x7f0a28071660 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a27400640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a28147990 0x7f0a28147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 0x7f0a28071ae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000e860 tx=0x7f0a1000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1ea00640 1 -- 172.21.15.67:0/14306659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10017070 con 0x7f0a28071660 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.715+0000 7f0a1ea00640 1 -- 172.21.15.67:0/14306659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1000c940 con 0x7f0a28071660 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.716+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a280c5ba0 con 0x7f0a28071660 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.716+0000 7f0a1ea00640 1 -- 172.21.15.67:0/14306659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000cd10 con 0x7f0a28071660 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.716+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2814f8e0 con 0x7f0a28071660 2024-02-26T10:39:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.718+0000 7f0a1ea00640 1 -- 172.21.15.67:0/14306659 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a1001a070 con 0x7f0a28071660 2024-02-26T10:39:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.718+0000 7f0a1ea00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0a0406b500 0x7f0a0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.719+0000 7f0a1ea00640 1 -- 172.21.15.67:0/14306659 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a1008a4b0 con 0x7f0a28071660 2024-02-26T10:39:46.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.719+0000 7f0a26a00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0a0406b500 0x7f0a0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.719+0000 7f0a30993740 1 Processor -- start 2024-02-26T10:39:46.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.720+0000 7f0a30993740 1 -- start start 2024-02-26T10:39:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.720+0000 7f0a26a00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0a0406b500 0x7f0a0406d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a18036000 tx=0x7f0a18030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.720+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b5059681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50592d410 0x55b505967450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a30993740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 0x55b50592c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a30993740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5058905d0 con 0x55b50596a1e0 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b505839900 con 0x55b505967070 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b50588c670 con 0x55b50592d410 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 0x55b50592c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 0x55b50592c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42258/0 (socket says 172.21.15.67:42258) 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 -- 172.21.15.67:0/1176373378 learned_addr learned my addr 172.21.15.67:0/1176373378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b5059681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50592d410 0x55b505967450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 -- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50592d410 msgr2=0x55b505967450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50592d410 0x55b505967450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 -- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 msgr2=0x55b5059681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b5059681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a27400640 1 -- 172.21.15.67:0/1176373378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5055a52d0 con 0x55b50596a1e0 2024-02-26T10:39:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.721+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b5059681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.722+0000 7f0a27400640 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 0x55b50592c190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a2806b960 tx=0x7f0a200021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c495cdfb7bde49b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.722+0000 7f0a17400640 1 -- 172.21.15.67:0/1176373378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000dea0 con 0x55b50596a1e0 2024-02-26T10:39:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.722+0000 7f0a17400640 1 -- 172.21.15.67:0/1176373378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20007500 con 0x55b50596a1e0 2024-02-26T10:39:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.722+0000 7f0a17400640 1 -- 172.21.15.67:0/1176373378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200078f0 con 0x55b50596a1e0 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.722+0000 7f0a30993740 1 -- 172.21.15.67:0/1176373378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 msgr2=0x55b50592c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.722+0000 7f0a30993740 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 0x55b50592c190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a2806b960 tx=0x7f0a200021f0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 -- 172.21.15.67:0/1176373378 shutdown_connections 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b50596a1e0 0x55b50592c190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50592d410 0x55b505967450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 --2- 172.21.15.67:0/1176373378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b5059681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 -- 172.21.15.67:0/1176373378 >> 172.21.15.67:0/1176373378 conn(0x55b505928a50 msgr2=0x55b50594d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 -- 172.21.15.67:0/1176373378 shutdown_connections 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 -- 172.21.15.67:0/1176373378 wait complete. 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.723+0000 7f0a30993740 1 -- start start 2024-02-26T10:39:46.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.724+0000 7f0a30993740 1 Processor -- start 2024-02-26T10:39:46.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b505964a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 0x55b505978ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b505969d50 con 0x55b505967070 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50584cd60 con 0x55b505978860 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b505969810 con 0x55b50596a1e0 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a30993740 10 client.0 _ll_register_callbacks cb 0x55b505888320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42266/0 (socket says 172.21.15.67:42266) 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 -- 172.21.15.67:0/3708436049 learned_addr learned my addr 172.21.15.67:0/3708436049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 -- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 msgr2=0x55b505964a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b505964a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 -- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 msgr2=0x55b505978ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 0x55b505978ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.725+0000 7f0a26a00640 1 -- 172.21.15.67:0/3708436049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b50566f6b0 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a26a00640 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594c6d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a1802c380 tx=0x7f0a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602df55ad1d7e68f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a0be00640 1 -- 172.21.15.67:0/3708436049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18039050 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a0be00640 1 -- 172.21.15.67:0/3708436049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a1802cbe0 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a20007500 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a0be00640 1 -- 172.21.15.67:0/3708436049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a18055050 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a30993740 10 client.5172 fetch_fsmap learned FSMap version 17 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b5059792c0 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.726+0000 7f0a0be00640 1 -- 172.21.15.67:0/3708436049 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20007500 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a0be00640 1 -- 172.21.15.67:0/3708436049 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a18058020 con 0x55b505967070 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a30993740 10 client.5172 fetch_fsmap finished waiting for FSMap version 17 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a30993740 10 client.5172 resolve_mds: resolved name 'd' to daemon mds.d 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a30993740 20 client.5172 populate_metadata read hostname 'smithi067' 2024-02-26T10:39:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a30993740 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b505984f20 0x55b505987310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a30993740 4 client.5172 mds_command: new command op to 4753 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:39:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 --> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b50589db10 con 0x55b505984f20 2024-02-26T10:39:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.727+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b505984f20 0x55b505987310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.728+0000 7f0a1fe00640 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b505984f20 0x55b505987310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faabe3766b1b444 server_cookie=8bc34b64560a9375 in_seq=0 out_seq=0 2024-02-26T10:39:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.728+0000 7f0a0be00640 10 client.5172 ms_handle_connect on v2:172.21.15.82:6835/155874470 2024-02-26T10:39:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.730+0000 7f0a0be00640 1 -- 172.21.15.67:0/3708436049 <== mds.0 v2:172.21.15.82:6835/155874470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b50589db10 con 0x55b505984f20 2024-02-26T10:39:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.730+0000 7f0a0be00640 10 client.5172 handle_command_reply: tid=0 2024-02-26T10:39:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.730+0000 7f0a30993740 1 client.5172 shutdown 2024-02-26T10:39:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 msgr2=0x55b50594c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594c6d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a1802c380 tx=0x7f0a180046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 shutdown_connections 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b505984f20 0x55b505987310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 0x55b505978ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b505964a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a0be00640 0 client.5172 ms_handle_reset on v2:172.21.15.82:6835/155874470 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 --2- 172.21.15.67:0/3708436049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594c6d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 >> 172.21.15.67:0/3708436049 conn(0x55b50592d410 msgr2=0x55b505967d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 shutdown_connections 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 1 -- 172.21.15.67:0/3708436049 wait complete. 2024-02-26T10:39:46.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.731+0000 7f0a30993740 20 client.5172 trim_cache size 0 max 16384 2024-02-26T10:39:46.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.748+0000 7f0a30993740 1 Processor -- start 2024-02-26T10:39:46.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.748+0000 7f0a30993740 1 -- start start 2024-02-26T10:39:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a30993740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b505967d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 0x55b50598b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a30993740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281f7d20 con 0x55b505967070 2024-02-26T10:39:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281f6ec0 con 0x55b505978860 2024-02-26T10:39:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281eb890 con 0x55b50596a1e0 2024-02-26T10:39:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a26a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a26a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42276/0 (socket says 172.21.15.67:42276) 2024-02-26T10:39:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.749+0000 7f0a26a00640 1 -- 172.21.15.67:0/1287370184 learned_addr learned my addr 172.21.15.67:0/1287370184 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a26a00640 1 -- 172.21.15.67:0/1287370184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 msgr2=0x55b505967d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b505967d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a26a00640 1 -- 172.21.15.67:0/1287370184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 msgr2=0x55b50598b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 0x55b50598b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a26a00640 1 -- 172.21.15.67:0/1287370184 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5058556f0 con 0x55b505967070 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a26a00640 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594b970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a1804a000 tx=0x7f0a18048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481616de241786b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a16a00640 1 -- 172.21.15.67:0/1287370184 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1805e070 con 0x55b505967070 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.750+0000 7f0a16a00640 1 -- 172.21.15.67:0/1287370184 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1803a040 con 0x55b505967070 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a16a00640 1 -- 172.21.15.67:0/1287370184 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18039070 con 0x55b505967070 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 -- 172.21.15.67:0/1287370184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 msgr2=0x55b50594b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594b970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a1804a000 tx=0x7f0a18048000 comp rx=0 tx=0).stop 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 -- 172.21.15.67:0/1287370184 shutdown_connections 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505978860 0x55b50598b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b505967d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 --2- 172.21.15.67:0/1287370184 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505967070 0x55b50594b970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 -- 172.21.15.67:0/1287370184 >> 172.21.15.67:0/1287370184 conn(0x55b50592d410 msgr2=0x55b505921c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 -- 172.21.15.67:0/1287370184 shutdown_connections 2024-02-26T10:39:46.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.751+0000 7f0a30993740 1 -- 172.21.15.67:0/1287370184 wait complete. 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.752+0000 7f0a30993740 1 -- start start 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.752+0000 7f0a30993740 1 Processor -- start 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b50597f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b50597e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 0x55b5059cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b50595dc30 con 0x55b505978860 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281f68c0 con 0x55b505967070 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b50595e430 con 0x55b50596a1e0 2024-02-26T10:39:46.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.753+0000 7f0a30993740 10 client.0 _ll_register_callbacks cb 0x7f0a200019f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 0x55b5059cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 0x55b5059cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42282/0 (socket says 172.21.15.67:42282) 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 -- 172.21.15.67:0/2334582527 learned_addr learned my addr 172.21.15.67:0/2334582527 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 -- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 msgr2=0x55b50597e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b50597e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 -- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 msgr2=0x55b50597f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b50597f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 -- 172.21.15.67:0/2334582527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b50594a2c0 con 0x55b505978860 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a27400640 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 0x55b5059cfdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a200070a0 tx=0x7f0a200079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f7b79d6891b6b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a0be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.754+0000 7f0a0be00640 1 -- 172.21.15.67:0/2334582527 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20022030 con 0x55b505978860 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a1803a040 con 0x55b505978860 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a0be00640 1 -- 172.21.15.67:0/2334582527 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a2001e440 con 0x55b505978860 2024-02-26T10:39:46.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a0be00640 1 -- 172.21.15.67:0/2334582527 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2002e050 con 0x55b505978860 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 10 client.5178 fetch_fsmap learned FSMap version 17 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b50594c240 con 0x55b505978860 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a0be00640 1 -- 172.21.15.67:0/2334582527 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1803a040 con 0x55b505978860 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a0be00640 1 -- 172.21.15.67:0/2334582527 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a20031020 con 0x55b505978860 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 10 client.5178 fetch_fsmap finished waiting for FSMap version 17 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 10 client.5178 resolve_mds: resolved name 'd' to daemon mds.d 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 20 client.5178 populate_metadata read hostname 'smithi067' 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b5059d62b0 0x55b5059d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 4 client.5178 mds_command: new command op to 4753 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:39:46.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.755+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 --> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b505897e60 con 0x55b5059d62b0 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.756+0000 7f0a26a00640 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b5059d62b0 0x55b5059d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a26a00640 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b5059d62b0 0x55b5059d6690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afe5f4c984da95 server_cookie=c027d1a1ebcb92c6 in_seq=0 out_seq=0 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a0be00640 10 client.5178 ms_handle_connect on v2:172.21.15.82:6835/155874470 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a0be00640 1 -- 172.21.15.67:0/2334582527 <== mds.0 v2:172.21.15.82:6835/155874470 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b505897e60 con 0x55b5059d62b0 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a0be00640 10 client.5178 handle_command_reply: tid=0 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a30993740 1 client.5178 shutdown 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 msgr2=0x55b5059cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a30993740 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 0x55b5059cfdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a200070a0 tx=0x7f0a200079c0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 shutdown_connections 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.757+0000 7f0a30993740 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:6835/155874470,v1:172.21.15.82:6837/155874470] conn(0x55b5059d62b0 0x55b5059d6690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b505978860 0x55b5059cfdf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b50596a1e0 0x55b50597e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 1 --2- 172.21.15.67:0/2334582527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b505967070 0x55b50597f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a0be00640 0 client.5178 ms_handle_reset on v2:172.21.15.82:6835/155874470 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 >> 172.21.15.67:0/2334582527 conn(0x55b50592d410 msgr2=0x55b505963150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 shutdown_connections 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 1 -- 172.21.15.67:0/2334582527 wait complete. 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a30993740 20 client.5178 trim_cache size 0 max 16384 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:39:46.761 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "f4a3a702-2ce8-46d3-b32d-e4f6872b8b58", 2024-02-26T10:39:46.762 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-02-26T10:39:46.762 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4753, 2024-02-26T10:39:46.762 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-02-26T10:39:46.762 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-02-26T10:39:46.762 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-02-26T10:39:46.762 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 6.7578076820000001, 2024-02-26T10:39:46.763 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2024-02-26T10:39:46.763 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 34, 2024-02-26T10:39:46.763 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 34, 2024-02-26T10:39:46.763 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 15.700740351 2024-02-26T10:39:46.763 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0a0406b500 msgr2=0x7f0a0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a27e00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0a0406b500 0x7f0a0406d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a18036000 tx=0x7f0a18030000 comp rx=0 tx=0).stop 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 msgr2=0x7f0a28071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.758+0000 7f0a27e00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 0x7f0a28071ae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000e860 tx=0x7f0a1000ed20 comp rx=0 tx=0).stop 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 shutdown_connections 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0a0406b500 0x7f0a0406d9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a28147990 0x7f0a28147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a28071660 0x7f0a28071ae0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 --2- 172.21.15.67:0/14306659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a281ee410 0x7f0a281ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 >> 172.21.15.67:0/14306659 conn(0x7f0a280c63f0 msgr2=0x7f0a280abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 shutdown_connections 2024-02-26T10:39:46.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.759+0000 7f0a27e00640 1 -- 172.21.15.67:0/14306659 wait complete. 2024-02-26T10:39:46.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:39:46.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.993+0000 7f5a2fe00640 1 Processor -- start 2024-02-26T10:39:46.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.994+0000 7f5a2fe00640 1 -- start start 2024-02-26T10:39:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.995+0000 7f5a2fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a30152380 0x7f5a300c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.995+0000 7f5a2fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a30152760 con 0x7f5a300c7020 2024-02-26T10:39:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.995+0000 7f5a2fe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a300c7550 con 0x7f5a3006d290 2024-02-26T10:39:46.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.995+0000 7f5a2fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a300c7830 con 0x7f5a30152380 2024-02-26T10:39:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a30152380 0x7f5a300c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5a300c7020 0x7f5a30155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49302/0 (socket says 172.21.15.67:49302) 2024-02-26T10:39:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a30152380 0x7f5a300c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42284/0 (socket says 172.21.15.67:42284) 2024-02-26T10:39:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2e000640 1 -- 172.21.15.67:0/367966273 learned_addr learned my addr 172.21.15.67:0/367966273 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2ea00640 1 -- 172.21.15.67:0/367966273 >> v1:172.21.15.82:6789/0 conn(0x7f5a3006d290 legacy=0x7f5a300c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:46.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2ea00640 1 -- 172.21.15.67:0/367966273 >> v1:172.21.15.67:6789/0 conn(0x7f5a300c7020 legacy=0x7f5a30155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2ea00640 1 -- 172.21.15.67:0/367966273 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3006ca90 con 0x7f5a30152380 2024-02-26T10:39:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.996+0000 7f5a2d600640 1 -- 172.21.15.67:0/367966273 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828700569 0 0) 0x7f5a30152760 con 0x7f5a300c7020 2024-02-26T10:39:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.998+0000 7f5a2ea00640 1 --2- 172.21.15.67:0/367966273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a300c8510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5a200097a0 tx=0x7f5a2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1abe57db25cfb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.998+0000 7f5a2d600640 1 -- 172.21.15.67:0/367966273 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200315a0 con 0x7f5a30152380 2024-02-26T10:39:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.998+0000 7f5a2d600640 1 -- 172.21.15.67:0/367966273 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a20037070 con 0x7f5a30152380 2024-02-26T10:39:46.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.998+0000 7f5a2d600640 1 -- 172.21.15.67:0/367966273 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20031960 con 0x7f5a30152380 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 -- 172.21.15.67:0/367966273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 msgr2=0x7f5a300c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/367966273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a300c8510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5a200097a0 tx=0x7f5a2002b840 comp rx=0 tx=0).stop 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 -- 172.21.15.67:0/367966273 shutdown_connections 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/367966273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a300c8510 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 -- 172.21.15.67:0/367966273 >> 172.21.15.67:0/367966273 conn(0x7f5a300ac1a0 msgr2=0x7f5a300ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 -- 172.21.15.67:0/367966273 shutdown_connections 2024-02-26T10:39:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:46.999+0000 7f5a2fe00640 1 -- 172.21.15.67:0/367966273 wait complete. 2024-02-26T10:39:47.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.001+0000 7f5a2fe00640 1 Processor -- start 2024-02-26T10:39:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.001+0000 7f5a2fe00640 1 -- start start 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a3006d290 0x7f5a3014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a300c7020 0x7f5a301473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a301478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c5820 con 0x7f5a30152380 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c56a0 con 0x7f5a300c7020 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3006d700 con 0x7f5a3006d290 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a301478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a301478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42298/0 (socket says 172.21.15.67:42298) 2024-02-26T10:39:47.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.002+0000 7f5a2f400640 1 -- 172.21.15.67:0/1050213888 learned_addr learned my addr 172.21.15.67:0/1050213888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2ea00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a3006d290 0x7f5a3014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2e000640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a300c7020 0x7f5a301473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2f400640 1 -- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a3006d290 msgr2=0x7f5a3014fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2f400640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a3006d290 0x7f5a3014fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2f400640 1 -- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a300c7020 msgr2=0x7f5a301473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2f400640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a300c7020 0x7f5a301473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2f400640 1 -- 172.21.15.67:0/1050213888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a301eb660 con 0x7f5a30152380 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2e000640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a300c7020 0x7f5a301473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a2f400640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a301478e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800c7b0 tx=0x7f5a2800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.003+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004b60 con 0x7f5a30152380 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.004+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a28004d00 con 0x7f5a30152380 2024-02-26T10:39:47.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.004+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2800aa90 con 0x7f5a30152380 2024-02-26T10:39:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.004+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a301488f0 con 0x7f5a30152380 2024-02-26T10:39:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.004+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301eb350 con 0x7f5a30152380 2024-02-26T10:39:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.005+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30074a30 con 0x7f5a30152380 2024-02-26T10:39:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.006+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a2800ac30 con 0x7f5a30152380 2024-02-26T10:39:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.007+0000 7f5a26a00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a1006b4d0 0x7f5a1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:47.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.007+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a2808a790 con 0x7f5a30152380 2024-02-26T10:39:47.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.008+0000 7f5a2ea00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a1006b4d0 0x7f5a1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.009+0000 7f5a2ea00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a1006b4d0 0x7f5a1006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5a20036000 tx=0x7f5a2003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.012+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a28059600 con 0x7f5a30152380 2024-02-26T10:39:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.257+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a300b0550 con 0x7f5a30152380 2024-02-26T10:39:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.257+0000 7f5a26a00640 1 -- 172.21.15.67:0/1050213888 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5a28058f00 con 0x7f5a30152380 2024-02-26T10:39:47.259 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a1006b4d0 msgr2=0x7f5a1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a1006b4d0 0x7f5a1006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5a20036000 tx=0x7f5a2003d040 comp rx=0 tx=0).stop 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 msgr2=0x7f5a301478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a301478e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800c7b0 tx=0x7f5a2800cc70 comp rx=0 tx=0).stop 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 shutdown_connections 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a30152380 0x7f5a301478e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a1006b4d0 0x7f5a1006d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a300c7020 0x7f5a301473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 --2- 172.21.15.67:0/1050213888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a3006d290 0x7f5a3014fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 >> 172.21.15.67:0/1050213888 conn(0x7f5a300ac1a0 msgr2=0x7f5a30153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 shutdown_connections 2024-02-26T10:39:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:47.260+0000 7f5a2fe00640 1 -- 172.21.15.67:0/1050213888 wait complete. 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:39:47.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:39:47.281 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:39:47.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.281 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:47.329 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42463sec preferred_lft 42463sec 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:47.330 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::288e:93ff:fe9f:f95d/64 scope link 2024-02-26T10:39:47.331 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:47.331 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:39:47.331 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:39:47.331 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::3cfc:46ff:fe71:84e3/64 scope link 2024-02-26T10:39:47.331 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:47.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.331 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:39:47.411 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:39:47.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.416 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:39:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:39:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:39:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:39:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:39:47.466 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:39:47.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.466 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:39:47.466 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:39:47.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:39:47.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:39:47.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:39:47.467 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:39:47.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.649 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:39:47.700 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:39:47.701 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:39:47.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:39:47.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:47.797 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:47.855 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:39:47.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:47.855 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:47.910 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-26T10:39:47.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:47.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:48.058 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:48.059 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:48.094 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:48.094 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:48.094 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:39:48.094 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:39:48.094 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:39:48.096 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T10:39:48.096 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:39:48.096 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:39:48.096 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:39:48.096 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:39:48.100 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:39:48.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:48.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:39:48.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:48.192 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:48.270 INFO:teuthology.orchestra.run.smithi067.stdout:client4712 2024-02-26T10:39:48.270 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:39:48.271 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:48.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:39:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.544+0000 7efef6e00640 1 Processor -- start 2024-02-26T10:39:48.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.545+0000 7efef6e00640 1 -- start start 2024-02-26T10:39:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.546+0000 7efef6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efef0152420 0x7efef00c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.546+0000 7efef6e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef0158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.546+0000 7efef6e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efef00c35d0 0x7efef0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.546+0000 7efef6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef01593a0 con 0x7efef0152420 2024-02-26T10:39:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.546+0000 7efef6e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00c15e0 con 0x7efef00c35d0 2024-02-26T10:39:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.546+0000 7efef6e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef006e660 con 0x7efef00bfa30 2024-02-26T10:39:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efef0152420 0x7efef00c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efef0152420 0x7efef00c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42310/0 (socket says 172.21.15.67:42310) 2024-02-26T10:39:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 -- 172.21.15.67:0/2078026972 learned_addr learned my addr 172.21.15.67:0/2078026972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efef4e00640 1 --2- 172.21.15.67:0/2078026972 >> v2:172.21.15.82:3300/0 conn(0x7efef00c35d0 0x7efef0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeef400640 1 --2- 172.21.15.67:0/2078026972 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 -- 172.21.15.67:0/2078026972 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 msgr2=0x7efef0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 --2- 172.21.15.67:0/2078026972 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef0158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 -- 172.21.15.67:0/2078026972 >> v2:172.21.15.82:3300/0 conn(0x7efef00c35d0 msgr2=0x7efef0158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 --2- 172.21.15.67:0/2078026972 >> v2:172.21.15.82:3300/0 conn(0x7efef00c35d0 0x7efef0158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efeefe00640 1 -- 172.21.15.67:0/2078026972 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef006e920 con 0x7efef0152420 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.547+0000 7efef4e00640 1 --2- 172.21.15.67:0/2078026972 >> v2:172.21.15.82:3300/0 conn(0x7efef00c35d0 0x7efef0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.548+0000 7efeefe00640 1 --2- 172.21.15.67:0/2078026972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef0152420 0x7efef00c3090 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efef006f670 tx=0x7efee002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf031c71ee77f3c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.549+0000 7efeeea00640 1 -- 172.21.15.67:0/2078026972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00317d0 con 0x7efef0152420 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.549+0000 7efeeea00640 1 -- 172.21.15.67:0/2078026972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0037070 con 0x7efef0152420 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.549+0000 7efeeea00640 1 -- 172.21.15.67:0/2078026972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0031b90 con 0x7efef0152420 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 -- 172.21.15.67:0/2078026972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef0152420 msgr2=0x7efef00c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 --2- 172.21.15.67:0/2078026972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef0152420 0x7efef00c3090 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efef006f670 tx=0x7efee002b870 comp rx=0 tx=0).stop 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 -- 172.21.15.67:0/2078026972 shutdown_connections 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 --2- 172.21.15.67:0/2078026972 >> v2:172.21.15.82:3300/0 conn(0x7efef00c35d0 0x7efef0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 --2- 172.21.15.67:0/2078026972 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef0158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 --2- 172.21.15.67:0/2078026972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef0152420 0x7efef00c3090 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 -- 172.21.15.67:0/2078026972 >> 172.21.15.67:0/2078026972 conn(0x7efef00a84a0 msgr2=0x7efef00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 -- 172.21.15.67:0/2078026972 shutdown_connections 2024-02-26T10:39:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.550+0000 7efef6e00640 1 -- 172.21.15.67:0/2078026972 wait complete. 2024-02-26T10:39:48.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.552+0000 7efef6e00640 1 Processor -- start 2024-02-26T10:39:48.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.552+0000 7efef6e00640 1 -- start start 2024-02-26T10:39:48.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.553+0000 7efef6e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.553+0000 7efef6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 0x7efef014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efef6e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef0152420 0x7efef014ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efef6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0158400 con 0x7efef00c35d0 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efef6e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00c1b60 con 0x7efef0152420 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efef6e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0152800 con 0x7efef00bfa30 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 0x7efef014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 0x7efef014e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42318/0 (socket says 172.21.15.67:42318) 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 -- 172.21.15.67:0/374515478 learned_addr learned my addr 172.21.15.67:0/374515478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeefe00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efef4e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef0152420 0x7efef014ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 -- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 msgr2=0x7efef014e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef014e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 -- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef0152420 msgr2=0x7efef014ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef0152420 0x7efef014ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.554+0000 7efeef400640 1 -- 172.21.15.67:0/374515478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef01eb8b0 con 0x7efef00c35d0 2024-02-26T10:39:48.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.555+0000 7efeef400640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 0x7efef014e8a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efedc00c570 tx=0x7efedc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:48.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.555+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc015070 con 0x7efef00c35d0 2024-02-26T10:39:48.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.555+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc004220 con 0x7efef00c35d0 2024-02-26T10:39:48.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.555+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc003610 con 0x7efef00c35d0 2024-02-26T10:39:48.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.555+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01ebb90 con 0x7efef00c35d0 2024-02-26T10:39:48.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.555+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef014cb30 con 0x7efef00c35d0 2024-02-26T10:39:48.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.557+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef01ebd90 con 0x7efef00c35d0 2024-02-26T10:39:48.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.557+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efedc0037b0 con 0x7efef00c35d0 2024-02-26T10:39:48.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.558+0000 7efee7e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efed006b4d0 0x7efed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:48.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.558+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efedc089f50 con 0x7efef00c35d0 2024-02-26T10:39:48.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.559+0000 7efeefe00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efed006b4d0 0x7efed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:48.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.560+0000 7efeefe00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efed006b4d0 0x7efed006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efee0036000 tx=0x7efee003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:48.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.563+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efef01ebd90 con 0x7efef00c35d0 2024-02-26T10:39:48.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.805+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efef0074a30 con 0x7efef00c35d0 2024-02-26T10:39:48.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.806+0000 7efee7e00640 1 -- 172.21.15.67:0/374515478 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7efedc058dc0 con 0x7efef00c35d0 2024-02-26T10:39:48.807 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:39:48.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efed006b4d0 msgr2=0x7efed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efed006b4d0 0x7efed006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efee0036000 tx=0x7efee003d040 comp rx=0 tx=0).stop 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 msgr2=0x7efef014e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 0x7efef014e8a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efedc00c570 tx=0x7efedc00ca30 comp rx=0 tx=0).stop 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 shutdown_connections 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efed006b4d0 0x7efed006d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef0152420 0x7efef014ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efef00c35d0 0x7efef014e8a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 --2- 172.21.15.67:0/374515478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efef00bfa30 0x7efef014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.809+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 >> 172.21.15.67:0/374515478 conn(0x7efef00a84a0 msgr2=0x7efef0150950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.810+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 shutdown_connections 2024-02-26T10:39:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:48.810+0000 7efef6e00640 1 -- 172.21.15.67:0/374515478 wait complete. 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:39:48.830 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:39:48.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:48.831 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:39:48.889 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:39:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:39:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:39:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:39:48.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:48.893 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:39:48.985 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:39:48.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:48.990 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:39:49.043 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:39:49.043 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:39:49.043 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:39:49.044 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:39:49.044 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:39:49.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:49.044 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:39:49.044 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:39:49.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:39:49.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:39:49.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:39:49.044 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:39:49.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:49.220 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:39:49.272 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:39:49.273 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:39:49.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:49.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:39:49.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:49.363 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:49.418 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:39:49.418 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:39:49.418 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:49.467 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-26T10:39:49.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:49.468 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:49.597 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:49.597 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:49.632 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:39:49.632 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:39:49.632 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:39:49.632 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:39:49.632 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:39:49.634 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:39:49.635 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:39:49.635 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:39:49.635 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:39:49.635 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:39:49.640 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:39:49.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:49.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:39:49.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:39:49.716 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:39:49.765 INFO:teuthology.orchestra.run.smithi082.stdout:client4718 2024-02-26T10:39:49.765 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:39:49.765 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:39:49.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-02-26T10:39:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.046+0000 7f6a27e00640 1 Processor -- start 2024-02-26T10:39:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.046+0000 7f6a27e00640 1 -- start start 2024-02-26T10:39:50.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.048+0000 7f6a27e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a280a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.048+0000 7f6a27e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2806b460 con 0x7f6a2806b000 2024-02-26T10:39:50.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.048+0000 7f6a27e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a280a9a80 con 0x7f6a2806db00 2024-02-26T10:39:50.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.048+0000 7f6a27e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2806dfb0 con 0x7f6a2806ac20 2024-02-26T10:39:50.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.049+0000 7f6a1fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6a2806b000 0x7f6a28153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47270/0 (socket says 172.21.15.67:47270) 2024-02-26T10:39:50.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.049+0000 7f6a1fe00640 1 -- 172.21.15.67:0/2195749653 learned_addr learned my addr 172.21.15.67:0/2195749653 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:50.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.049+0000 7f6a26a00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a280a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.049+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429061073 0 0) 0x7f6a2806b460 con 0x7f6a2806b000 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.049+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a0c003520 con 0x7f6a2806b000 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1395151245 0 0) 0x7f6a0c003520 con 0x7f6a2806b000 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 msgr2=0x7f6a280a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a280a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 >> v1:172.21.15.82:6789/0 conn(0x7f6a2806db00 legacy=0x7f6a280abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a280a9d60 con 0x7f6a2806b000 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f6a10001670 con 0x7f6a2806b000 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 >> v1:172.21.15.67:6789/0 conn(0x7f6a2806b000 legacy=0x7f6a28153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a0c0039e0 0x7f6a0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c004d40 0x7f6a0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2806dfb0 con 0x7f6a0c0039e0 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c00baa0 con 0x7f6a0c004d40 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c00bca0 con 0x7f6a2806ac20 2024-02-26T10:39:50.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a26a00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a0c0039e0 0x7f6a0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.050+0000 7f6a27400640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c004d40 0x7f6a0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a1fe00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a26a00640 1 -- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 msgr2=0x7f6a0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a26a00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a26a00640 1 -- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c004d40 msgr2=0x7f6a0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a26a00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c004d40 0x7f6a0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a26a00640 1 -- 172.21.15.67:0/2195749653 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c003520 con 0x7f6a0c0039e0 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a1fe00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a2806ac20 0x7f6a0c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.051+0000 7f6a27400640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c004d40 0x7f6a0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:39:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.052+0000 7f6a26a00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a0c0039e0 0x7f6a0c003e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009a90 tx=0x7f6a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c412b602424893 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.053+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18039070 con 0x7f6a0c0039e0 2024-02-26T10:39:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.053+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1802c680 con 0x7f6a0c0039e0 2024-02-26T10:39:50.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.053+0000 7f6a26000640 1 -- 172.21.15.67:0/2195749653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18009690 con 0x7f6a0c0039e0 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27e00640 1 -- 172.21.15.67:0/2195749653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a0c0039e0 msgr2=0x7f6a0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27e00640 1 --2- 172.21.15.67:0/2195749653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a0c0039e0 0x7f6a0c003e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009a90 tx=0x7f6a0c005830 comp rx=0 tx=0).stop 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27400640 1 -- 172.21.15.67:0/2195749653 reap_dead start 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27e00640 1 -- 172.21.15.67:0/2195749653 shutdown_connections 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27e00640 1 -- 172.21.15.67:0/2195749653 >> 172.21.15.67:0/2195749653 conn(0x7f6a280c63f0 msgr2=0x7f6a280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27e00640 1 -- 172.21.15.67:0/2195749653 shutdown_connections 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.054+0000 7f6a27e00640 1 -- 172.21.15.67:0/2195749653 wait complete. 2024-02-26T10:39:50.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.056+0000 7f6a27e00640 1 Processor -- start 2024-02-26T10:39:50.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.056+0000 7f6a27e00640 1 -- start start 2024-02-26T10:39:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a27e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c0039e0 0x7f6a281eb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 0x7f6a280c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a27e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a280bfa90 0x7f6a281ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a27e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28150410 con 0x7f6a28148410 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a27e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a280aba30 con 0x7f6a0c0039e0 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a27e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28150590 con 0x7f6a280bfa90 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 0x7f6a280c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 0x7f6a280c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54152/0 (socket says 172.21.15.67:54152) 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a1fe00640 1 -- 172.21.15.67:0/1557913815 learned_addr learned my addr 172.21.15.67:0/1557913815 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.057+0000 7f6a26a00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c0039e0 0x7f6a281eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a27400640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a280bfa90 0x7f6a281ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1fe00640 1 -- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a280bfa90 msgr2=0x7f6a281ee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1fe00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a280bfa90 0x7f6a281ee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1fe00640 1 -- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c0039e0 msgr2=0x7f6a281eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1fe00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c0039e0 0x7f6a281eb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1fe00640 1 -- 172.21.15.67:0/1557913815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a280c4860 con 0x7f6a28148410 2024-02-26T10:39:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a27400640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a280bfa90 0x7f6a281ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:50.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1fe00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 0x7f6a280c4320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000d870 tx=0x7f6a1000dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a10019070 con 0x7f6a28148410 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a100074e0 con 0x7f6a28148410 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a100049e0 con 0x7f6a28148410 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.058+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a28071fd0 con 0x7f6a28148410 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.059+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a281ee0c0 con 0x7f6a28148410 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.060+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a280724f0 con 0x7f6a28148410 2024-02-26T10:39:50.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.061+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a10004b80 con 0x7f6a28148410 2024-02-26T10:39:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.062+0000 7f6a1ea00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6a0406b500 0x7f6a0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.062+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a10089b90 con 0x7f6a28148410 2024-02-26T10:39:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.062+0000 7f6a26a00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6a0406b500 0x7f6a0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.063+0000 7f6a26a00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6a0406b500 0x7f6a0406d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a18036000 tx=0x7f6a18030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:50.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.066+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a280724f0 con 0x7f6a28148410 2024-02-26T10:39:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.335+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7f6a28074a30 con 0x7f6a28148410 2024-02-26T10:39:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.336+0000 7f6a1ea00640 1 -- 172.21.15.67:0/1557913815 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v9) v1 ==== 84+0+0 (secure 0 0 0) 0x7f6a10058a00 con 0x7f6a28148410 2024-02-26T10:39:50.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.338+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6a0406b500 msgr2=0x7f6a0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.338+0000 7f6a27e00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6a0406b500 0x7f6a0406d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a18036000 tx=0x7f6a18030000 comp rx=0 tx=0).stop 2024-02-26T10:39:50.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.338+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 msgr2=0x7f6a280c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.338+0000 7f6a27e00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 0x7f6a280c4320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000d870 tx=0x7f6a1000dd30 comp rx=0 tx=0).stop 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 shutdown_connections 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a280bfa90 0x7f6a281ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28148410 0x7f6a280c4320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6a0406b500 0x7f6a0406d9c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 --2- 172.21.15.67:0/1557913815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a0c0039e0 0x7f6a281eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 >> 172.21.15.67:0/1557913815 conn(0x7f6a280c63f0 msgr2=0x7f6a280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 shutdown_connections 2024-02-26T10:39:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.339+0000 7f6a27e00640 1 -- 172.21.15.67:0/1557913815 wait complete. 2024-02-26T10:39:50.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.573+0000 7f48e2a00640 1 Processor -- start 2024-02-26T10:39:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.573+0000 7f48e2a00640 1 -- start start 2024-02-26T10:39:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.575+0000 7f48e2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48dc0c85b0 0x7f48dc14c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.575+0000 7f48e2a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc0c56a0 con 0x7f48dc0c3e70 2024-02-26T10:39:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.575+0000 7f48e2a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc0c5980 con 0x7f48dc0c7d70 2024-02-26T10:39:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.575+0000 7f48e2a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc152ec0 con 0x7f48dc0c85b0 2024-02-26T10:39:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.575+0000 7f48db400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f48dc0c3e70 0x7f48dc153350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47272/0 (socket says 172.21.15.67:47272) 2024-02-26T10:39:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.575+0000 7f48db400640 1 -- 172.21.15.67:0/3383038622 learned_addr learned my addr 172.21.15.67:0/3383038622 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.576+0000 7f48e0a00640 1 --2- 172.21.15.67:0/3383038622 >> v2:172.21.15.82:3300/0 conn(0x7f48dc0c85b0 0x7f48dc14c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.576+0000 7f48daa00640 1 -- 172.21.15.67:0/3383038622 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382554802 0 0) 0x7f48dc0c56a0 con 0x7f48dc0c3e70 2024-02-26T10:39:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.576+0000 7f48daa00640 1 -- 172.21.15.67:0/3383038622 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c0003520 con 0x7f48dc0c3e70 2024-02-26T10:39:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.576+0000 7f48e0a00640 1 -- 172.21.15.67:0/3383038622 >> v1:172.21.15.82:6789/0 conn(0x7f48dc0c7d70 legacy=0x7f48dc0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.576+0000 7f48e0a00640 1 -- 172.21.15.67:0/3383038622 >> v1:172.21.15.67:6789/0 conn(0x7f48dc0c3e70 legacy=0x7f48dc153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.576+0000 7f48e0a00640 1 -- 172.21.15.67:0/3383038622 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0c5c60 con 0x7f48dc0c85b0 2024-02-26T10:39:50.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.578+0000 7f48e0a00640 1 --2- 172.21.15.67:0/3383038622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c85b0 0x7f48dc14c430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48dc153040 tx=0x7f48d402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2607bc38f734b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.578+0000 7f48daa00640 1 -- 172.21.15.67:0/3383038622 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4039070 con 0x7f48dc0c85b0 2024-02-26T10:39:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.578+0000 7f48daa00640 1 -- 172.21.15.67:0/3383038622 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d402cce0 con 0x7f48dc0c85b0 2024-02-26T10:39:50.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.578+0000 7f48daa00640 1 -- 172.21.15.67:0/3383038622 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4034640 con 0x7f48dc0c85b0 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 -- 172.21.15.67:0/3383038622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c85b0 msgr2=0x7f48dc14c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3383038622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c85b0 0x7f48dc14c430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48dc153040 tx=0x7f48d402bf30 comp rx=0 tx=0).stop 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 -- 172.21.15.67:0/3383038622 shutdown_connections 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3383038622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c85b0 0x7f48dc14c430 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 -- 172.21.15.67:0/3383038622 >> 172.21.15.67:0/3383038622 conn(0x7f48dc06b040 msgr2=0x7f48dc06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 -- 172.21.15.67:0/3383038622 shutdown_connections 2024-02-26T10:39:50.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.579+0000 7f48e2a00640 1 -- 172.21.15.67:0/3383038622 wait complete. 2024-02-26T10:39:50.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.581+0000 7f48e2a00640 1 Processor -- start 2024-02-26T10:39:50.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.581+0000 7f48e2a00640 1 -- start start 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48dc0c3e70 0x7f48dc1dd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c7d70 0x7f48dc14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 0x7f48dc0aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc153be0 con 0x7f48dc0c85b0 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc153a60 con 0x7f48dc0c7d70 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0c4d50 con 0x7f48dc0c3e70 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 0x7f48dc0aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 0x7f48dc0aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54158/0 (socket says 172.21.15.67:54158) 2024-02-26T10:39:50.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 -- 172.21.15.67:0/3094350628 learned_addr learned my addr 172.21.15.67:0/3094350628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48db400640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c7d70 0x7f48dc14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48dbe00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48dc0c3e70 0x7f48dc1dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 -- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48dc0c3e70 msgr2=0x7f48dc1dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48dc0c3e70 0x7f48dc1dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 -- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c7d70 msgr2=0x7f48dc14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c7d70 0x7f48dc14e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48e0a00640 1 -- 172.21.15.67:0/3094350628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc1ddd60 con 0x7f48dc0c85b0 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48dbe00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48dc0c3e70 0x7f48dc1dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:50.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.582+0000 7f48db400640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c7d70 0x7f48dc14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.583+0000 7f48e0a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 0x7f48dc0aa170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009e80 tx=0x7f48d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.583+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4042020 con 0x7f48dc0c85b0 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.583+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d403a040 con 0x7f48dc0c85b0 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.583+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4039070 con 0x7f48dc0c85b0 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.583+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc1ee620 con 0x7f48dc0c85b0 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.583+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc1dccd0 con 0x7f48dc0c85b0 2024-02-26T10:39:50.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.584+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc074a30 con 0x7f48dc0c85b0 2024-02-26T10:39:50.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.585+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48d4005d10 con 0x7f48dc0c85b0 2024-02-26T10:39:50.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.586+0000 7f48d3e00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f48bc06b500 0x7f48bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:50.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.586+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d40ac510 con 0x7f48dc0c85b0 2024-02-26T10:39:50.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.587+0000 7f48dbe00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f48bc06b500 0x7f48bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:50.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.588+0000 7f48dbe00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f48bc06b500 0x7f48bc06d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48cc000f60 tx=0x7f48cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:50.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.590+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48d404c070 con 0x7f48dc0c85b0 2024-02-26T10:39:50.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.788+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48dc0adba0 con 0x7f48dc0c85b0 2024-02-26T10:39:50.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.791+0000 7f48d3e00640 1 -- 172.21.15.67:0/3094350628 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f48d403c300 con 0x7f48dc0c85b0 2024-02-26T10:39:50.792 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:50.793 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":34,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:40.989006+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f48bc06b500 msgr2=0x7f48bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f48bc06b500 0x7f48bc06d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48cc000f60 tx=0x7f48cc009040 comp rx=0 tx=0).stop 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 msgr2=0x7f48dc0aa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 0x7f48dc0aa170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009e80 tx=0x7f48d4004a40 comp rx=0 tx=0).stop 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 shutdown_connections 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48dc0c85b0 0x7f48dc0aa170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f48bc06b500 0x7f48bc06d9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48dc0c7d70 0x7f48dc14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 --2- 172.21.15.67:0/3094350628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48dc0c3e70 0x7f48dc1dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 >> 172.21.15.67:0/3094350628 conn(0x7f48dc06b040 msgr2=0x7f48dc0c72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 shutdown_connections 2024-02-26T10:39:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:50.794+0000 7f48e2a00640 1 -- 172.21.15.67:0/3094350628 wait complete. 2024-02-26T10:39:50.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:39:37.371486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:39:38.475801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:39:50.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:39:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.029+0000 7fb7ee200640 1 Processor -- start 2024-02-26T10:39:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.029+0000 7fb7ee200640 1 -- start start 2024-02-26T10:39:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7ee200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e81591b0 0x7fb7e8158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7ee200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7e80c0ef0 0x7fb7e8070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7ee200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e8072ce0 con 0x7fb7e8071410 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7ee200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e8073010 con 0x7fb7e80c0ef0 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7ee200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e814f490 con 0x7fb7e81591b0 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7e80c0ef0 0x7fb7e8070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb7e80c0ef0 0x7fb7e8070ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54172/0 (socket says 172.21.15.67:54172) 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.031+0000 7fb7dfe00640 1 -- 172.21.15.67:0/1634412963 learned_addr learned my addr 172.21.15.67:0/1634412963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.032+0000 7fb7e7400640 1 --2- 172.21.15.67:0/1634412963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e81591b0 0x7fb7e8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.032+0000 7fb7dfe00640 1 -- 172.21.15.67:0/1634412963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e81591b0 msgr2=0x7fb7e8158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.032+0000 7fb7dfe00640 1 --2- 172.21.15.67:0/1634412963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e81591b0 0x7fb7e8158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.032+0000 7fb7dfe00640 1 -- 172.21.15.67:0/1634412963 >> v1:172.21.15.67:6789/0 conn(0x7fb7e8071410 legacy=0x7fb7e806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.032+0000 7fb7e6a00640 1 -- 172.21.15.67:0/1634412963 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337000237 0 0) 0x7fb7e8072ce0 con 0x7fb7e8071410 2024-02-26T10:39:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.032+0000 7fb7dfe00640 1 -- 172.21.15.67:0/1634412963 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8155bf0 con 0x7fb7e80c0ef0 2024-02-26T10:39:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.033+0000 7fb7dfe00640 1 --2- 172.21.15.67:0/1634412963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e8070ed0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8073190 tx=0x7fb7d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d268a28add6f857f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.033+0000 7fb7e6a00640 1 -- 172.21.15.67:0/1634412963 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0035020 con 0x7fb7e80c0ef0 2024-02-26T10:39:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.033+0000 7fb7e6a00640 1 -- 172.21.15.67:0/1634412963 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d0031640 con 0x7fb7e80c0ef0 2024-02-26T10:39:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.033+0000 7fb7e6a00640 1 -- 172.21.15.67:0/1634412963 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d002d040 con 0x7fb7e80c0ef0 2024-02-26T10:39:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 -- 172.21.15.67:0/1634412963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e80c0ef0 msgr2=0x7fb7e8070ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 --2- 172.21.15.67:0/1634412963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e8070ed0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8073190 tx=0x7fb7d002b8f0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 -- 172.21.15.67:0/1634412963 shutdown_connections 2024-02-26T10:39:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 --2- 172.21.15.67:0/1634412963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e8070ed0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 --2- 172.21.15.67:0/1634412963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e81591b0 0x7fb7e8158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 -- 172.21.15.67:0/1634412963 >> 172.21.15.67:0/1634412963 conn(0x7fb7e80c6da0 msgr2=0x7fb7e80c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 -- 172.21.15.67:0/1634412963 shutdown_connections 2024-02-26T10:39:51.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.034+0000 7fb7ee200640 1 -- 172.21.15.67:0/1634412963 wait complete. 2024-02-26T10:39:51.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.036+0000 7fb7ee200640 1 Processor -- start 2024-02-26T10:39:51.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.036+0000 7fb7ee200640 1 -- start start 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7ee200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8071410 0x7fb7e814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7ee200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7ee200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 0x7fb7e814a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7ee200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e806e6e0 con 0x7fb7e81591b0 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7ee200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e806e560 con 0x7fb7e80c0ef0 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7ee200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80a8ba0 con 0x7fb7e8071410 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 0x7fb7e814a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 0x7fb7e814a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54184/0 (socket says 172.21.15.67:54184) 2024-02-26T10:39:51.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.037+0000 7fb7e7e00640 1 -- 172.21.15.67:0/2664034973 learned_addr learned my addr 172.21.15.67:0/2664034973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7dfe00640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7400640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8071410 0x7fb7e814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7e00640 1 -- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8071410 msgr2=0x7fb7e814f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8071410 0x7fb7e814f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7e00640 1 -- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e80c0ef0 msgr2=0x7fb7e814cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e814cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7e00640 1 -- 172.21.15.67:0/2664034973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e81eea60 con 0x7fb7e81591b0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7400640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8071410 0x7fb7e814f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 0x7fb7e814a290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000c760 tx=0x7fb7e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0003da0 con 0x7fb7e81591b0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.038+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0005070 con 0x7fb7e81591b0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.039+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0005460 con 0x7fb7e81591b0 2024-02-26T10:39:51.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.039+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e81eece0 con 0x7fb7e81591b0 2024-02-26T10:39:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.039+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e80a9b00 con 0x7fb7e81591b0 2024-02-26T10:39:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.040+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e8074a30 con 0x7fb7e81591b0 2024-02-26T10:39:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.041+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7e0005600 con 0x7fb7e81591b0 2024-02-26T10:39:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.042+0000 7fb7df400640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b500 0x7fb7c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.042+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7e008a0d0 con 0x7fb7e81591b0 2024-02-26T10:39:51.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.042+0000 7fb7e7400640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b500 0x7fb7c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.043+0000 7fb7e7400640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b500 0x7fb7c406d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006fd0 tx=0x7fb7d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.046+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7e81eea60 con 0x7fb7e81591b0 2024-02-26T10:39:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.246+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7e806eb10 con 0x7fb7e81591b0 2024-02-26T10:39:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.247+0000 7fb7df400640 1 -- 172.21.15.67:0/2664034973 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7e0058f40 con 0x7fb7e81591b0 2024-02-26T10:39:51.247 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:39:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.249+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b500 msgr2=0x7fb7c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.249+0000 7fb7ee200640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b500 0x7fb7c406d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006fd0 tx=0x7fb7d8008040 comp rx=0 tx=0).stop 2024-02-26T10:39:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.249+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 msgr2=0x7fb7e814a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.249+0000 7fb7ee200640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 0x7fb7e814a290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000c760 tx=0x7fb7e000cc20 comp rx=0 tx=0).stop 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 shutdown_connections 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e81591b0 0x7fb7e814a290 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b500 0x7fb7c406d9c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e80c0ef0 0x7fb7e814cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 --2- 172.21.15.67:0/2664034973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8071410 0x7fb7e814f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 >> 172.21.15.67:0/2664034973 conn(0x7fb7e80c6da0 msgr2=0x7fb7e814eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 shutdown_connections 2024-02-26T10:39:51.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.250+0000 7fb7ee200640 1 -- 172.21.15.67:0/2664034973 wait complete. 2024-02-26T10:39:51.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:39:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.498+0000 7f0d55200640 1 Processor -- start 2024-02-26T10:39:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.499+0000 7f0d55200640 1 -- start start 2024-02-26T10:39:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.500+0000 7f0d55200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d501503a0 0x7f0d500ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.500+0000 7f0d55200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d5006e2d0 0x7f0d500c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.500+0000 7f0d55200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d50150780 con 0x7f0d500a9430 2024-02-26T10:39:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.500+0000 7f0d55200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500a98c0 con 0x7f0d501503a0 2024-02-26T10:39:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.500+0000 7f0d55200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500a9ba0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d501503a0 0x7f0d500ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d501503a0 0x7f0d500ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54192/0 (socket says 172.21.15.67:54192) 2024-02-26T10:39:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 -- 172.21.15.67:0/3111399248 learned_addr learned my addr 172.21.15.67:0/3111399248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 -- 172.21.15.67:0/3111399248 >> v2:172.21.15.82:3300/0 conn(0x7f0d5006e2d0 msgr2=0x7f0d500c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 --2- 172.21.15.67:0/3111399248 >> v2:172.21.15.82:3300/0 conn(0x7f0d5006e2d0 0x7f0d500c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 -- 172.21.15.67:0/3111399248 >> v1:172.21.15.82:6789/0 conn(0x7f0d500a9430 legacy=0x7f0d501535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.501+0000 7f0d4f400640 1 -- 172.21.15.67:0/3111399248 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500a9e80 con 0x7f0d501503a0 2024-02-26T10:39:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.503+0000 7f0d4f400640 1 --2- 172.21.15.67:0/3111399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d501503a0 0x7f0d500ab480 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0d500c3f40 tx=0x7f0d4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f3cb21676ea57a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.503+0000 7f0d4ea00640 1 -- 172.21.15.67:0/3111399248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40035020 con 0x7f0d501503a0 2024-02-26T10:39:51.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.503+0000 7f0d4ea00640 1 -- 172.21.15.67:0/3111399248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d40031870 con 0x7f0d501503a0 2024-02-26T10:39:51.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.503+0000 7f0d4ea00640 1 -- 172.21.15.67:0/3111399248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40031c40 con 0x7f0d501503a0 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 -- 172.21.15.67:0/3111399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d501503a0 msgr2=0x7f0d500ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 --2- 172.21.15.67:0/3111399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d501503a0 0x7f0d500ab480 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0d500c3f40 tx=0x7f0d4002b910 comp rx=0 tx=0).stop 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 -- 172.21.15.67:0/3111399248 shutdown_connections 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 --2- 172.21.15.67:0/3111399248 >> v2:172.21.15.82:3300/0 conn(0x7f0d5006e2d0 0x7f0d500c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 --2- 172.21.15.67:0/3111399248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d501503a0 0x7f0d500ab480 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 -- 172.21.15.67:0/3111399248 >> 172.21.15.67:0/3111399248 conn(0x7f0d500c6380 msgr2=0x7f0d500c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 -- 172.21.15.67:0/3111399248 shutdown_connections 2024-02-26T10:39:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.504+0000 7f0d55200640 1 -- 172.21.15.67:0/3111399248 wait complete. 2024-02-26T10:39:51.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.506+0000 7f0d55200640 1 Processor -- start 2024-02-26T10:39:51.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.506+0000 7f0d55200640 1 -- start start 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d55200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 0x7f0d5014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d55200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d501503a0 0x7f0d50149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d55200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d5014e120 0x7f0d50072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d55200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500c0f80 con 0x7f0d5006e2d0 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d55200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5006ddb0 con 0x7f0d5014e120 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d55200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5006e740 con 0x7f0d501503a0 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 0x7f0d5014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 0x7f0d5014f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54206/0 (socket says 172.21.15.67:54206) 2024-02-26T10:39:51.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.507+0000 7f0d4f400640 1 -- 172.21.15.67:0/1468770556 learned_addr learned my addr 172.21.15.67:0/1468770556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4fe00640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d5014e120 0x7f0d50072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d46a00640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d501503a0 0x7f0d50149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4f400640 1 -- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d501503a0 msgr2=0x7f0d50149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4f400640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d501503a0 0x7f0d50149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4f400640 1 -- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d5014e120 msgr2=0x7f0d50072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4f400640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d5014e120 0x7f0d50072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4f400640 1 -- 172.21.15.67:0/1468770556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d501ee9d0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4fe00640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d5014e120 0x7f0d50072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.508+0000 7f0d4f400640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 0x7f0d5014f850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000c00 tx=0x7f0d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.509+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4003ad60 con 0x7f0d5006e2d0 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.509+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d400443f0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.509+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d500c60b0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.509+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d400447e0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.509+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5014d9c0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.510+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d500678e0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.511+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d40038020 con 0x7f0d5006e2d0 2024-02-26T10:39:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.512+0000 7f0d47e00640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.512+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d400ac3e0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.512+0000 7f0d46a00640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.513+0000 7f0d46a00640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0d5014a510 tx=0x7f0d38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.517+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d4003f070 con 0x7f0d5006e2d0 2024-02-26T10:39:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.717+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d50072070 con 0x7f0d5006e2d0 2024-02-26T10:39:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.718+0000 7f0d47e00640 1 -- 172.21.15.67:0/1468770556 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d4007b2d0 con 0x7f0d5006e2d0 2024-02-26T10:39:51.719 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d2c06b4d0 msgr2=0x7f0d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0d5014a510 tx=0x7f0d38009040 comp rx=0 tx=0).stop 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 msgr2=0x7f0d5014f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 0x7f0d5014f850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000c00 tx=0x7f0d400046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 shutdown_connections 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d5014e120 0x7f0d50072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d501503a0 0x7f0d50149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 --2- 172.21.15.67:0/1468770556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d5006e2d0 0x7f0d5014f850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 >> 172.21.15.67:0/1468770556 conn(0x7f0d500c6380 msgr2=0x7f0d500741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:51.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 shutdown_connections 2024-02-26T10:39:51.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.721+0000 7f0d55200640 1 -- 172.21.15.67:0/1468770556 wait complete. 2024-02-26T10:39:51.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:39:51.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.968+0000 7fac4b200640 1 Processor -- start 2024-02-26T10:39:51.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.968+0000 7fac4b200640 1 -- start start 2024-02-26T10:39:51.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.970+0000 7fac4b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fac4c1523b0 0x7fac4c0c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.970+0000 7fac4b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c0c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.970+0000 7fac4b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac4c071fd0 0x7fac4c072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.970+0000 7fac4b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c152790 con 0x7fac4c1523b0 2024-02-26T10:39:51.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.970+0000 7fac4b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c06da30 con 0x7fac4c071fd0 2024-02-26T10:39:51.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.970+0000 7fac4b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c06dc80 con 0x7fac4c154cf0 2024-02-26T10:39:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fac4c1523b0 0x7fac4c0c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fac4c1523b0 0x7fac4c0c2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54208/0 (socket says 172.21.15.67:54208) 2024-02-26T10:39:51.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 -- 172.21.15.67:0/3487962122 learned_addr learned my addr 172.21.15.67:0/3487962122 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 -- 172.21.15.67:0/3487962122 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 msgr2=0x7fac4c0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49400640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c0c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac4a800640 1 --2- 172.21.15.67:0/3487962122 >> v2:172.21.15.82:3300/0 conn(0x7fac4c071fd0 0x7fac4c072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c0c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 -- 172.21.15.67:0/3487962122 >> v2:172.21.15.82:3300/0 conn(0x7fac4c071fd0 msgr2=0x7fac4c072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 --2- 172.21.15.67:0/3487962122 >> v2:172.21.15.82:3300/0 conn(0x7fac4c071fd0 0x7fac4c072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49e00640 1 -- 172.21.15.67:0/3487962122 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c06df60 con 0x7fac4c1523b0 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac4a800640 1 --2- 172.21.15.67:0/3487962122 >> v2:172.21.15.82:3300/0 conn(0x7fac4c071fd0 0x7fac4c072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:51.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.971+0000 7fac49400640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c0c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:51.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.973+0000 7fac49e00640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c1523b0 0x7fac4c0c2bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0c7320 tx=0x7fac3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f609bf781bf2b11 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.973+0000 7fac48a00640 1 -- 172.21.15.67:0/3487962122 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c02e070 con 0x7fac4c1523b0 2024-02-26T10:39:51.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.973+0000 7fac48a00640 1 -- 172.21.15.67:0/3487962122 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c037070 con 0x7fac4c1523b0 2024-02-26T10:39:51.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.973+0000 7fac48a00640 1 -- 172.21.15.67:0/3487962122 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c031a00 con 0x7fac4c1523b0 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 -- 172.21.15.67:0/3487962122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c1523b0 msgr2=0x7fac4c0c2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c1523b0 0x7fac4c0c2bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0c7320 tx=0x7fac3c02b880 comp rx=0 tx=0).stop 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 -- 172.21.15.67:0/3487962122 shutdown_connections 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 --2- 172.21.15.67:0/3487962122 >> v2:172.21.15.82:3300/0 conn(0x7fac4c071fd0 0x7fac4c072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c0c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 --2- 172.21.15.67:0/3487962122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c1523b0 0x7fac4c0c2bf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 -- 172.21.15.67:0/3487962122 >> 172.21.15.67:0/3487962122 conn(0x7fac4c0c6380 msgr2=0x7fac4c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:51.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 -- 172.21.15.67:0/3487962122 shutdown_connections 2024-02-26T10:39:51.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.974+0000 7fac4b200640 1 -- 172.21.15.67:0/3487962122 wait complete. 2024-02-26T10:39:51.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.976+0000 7fac4b200640 1 Processor -- start 2024-02-26T10:39:51.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.976+0000 7fac4b200640 1 -- start start 2024-02-26T10:39:51.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac4b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 0x7fac4c0a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac4b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac4c1523b0 0x7fac4c157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac4b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac4b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c06e800 con 0x7fac4c071fd0 2024-02-26T10:39:51.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac4b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c06e520 con 0x7fac4c1523b0 2024-02-26T10:39:51.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac4b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0c08e0 con 0x7fac4c154cf0 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac49e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 0x7fac4c0a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac49e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 0x7fac4c0a9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54218/0 (socket says 172.21.15.67:54218) 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.977+0000 7fac49e00640 1 -- 172.21.15.67:0/1425503860 learned_addr learned my addr 172.21.15.67:0/1425503860 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49400640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac4c1523b0 0x7fac4c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49e00640 1 -- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 msgr2=0x7fac4c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49e00640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49e00640 1 -- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac4c1523b0 msgr2=0x7fac4c157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49e00640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac4c1523b0 0x7fac4c157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:51.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49e00640 1 -- 172.21.15.67:0/1425503860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c1565d0 con 0x7fac4c071fd0 2024-02-26T10:39:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac49e00640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 0x7fac4c0a9420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fac3c000c00 tx=0x7fac3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c032880 con 0x7fac4c071fd0 2024-02-26T10:39:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c032a20 con 0x7fac4c071fd0 2024-02-26T10:39:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c032e10 con 0x7fac4c071fd0 2024-02-26T10:39:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.978+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c1568b0 con 0x7fac4c071fd0 2024-02-26T10:39:51.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.979+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0a9960 con 0x7fac4c071fd0 2024-02-26T10:39:51.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.980+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c0678e0 con 0x7fac4c071fd0 2024-02-26T10:39:51.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.981+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac3c0453f0 con 0x7fac4c071fd0 2024-02-26T10:39:51.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.981+0000 7fac42000640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fac2c06b4d0 0x7fac2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:51.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.982+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac3c0ac2b0 con 0x7fac4c071fd0 2024-02-26T10:39:51.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.982+0000 7fac49400640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fac2c06b4d0 0x7fac2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:51.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.983+0000 7fac49400640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fac2c06b4d0 0x7fac2c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0a9e40 tx=0x7fac38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:51.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:51.986+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac3c040070 con 0x7fac4c071fd0 2024-02-26T10:39:52.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.184+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac4c0bf840 con 0x7fac4c071fd0 2024-02-26T10:39:52.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.185+0000 7fac42000640 1 -- 172.21.15.67:0/1425503860 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac3c07b130 con 0x7fac4c071fd0 2024-02-26T10:39:52.186 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fac2c06b4d0 msgr2=0x7fac2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fac2c06b4d0 0x7fac2c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0a9e40 tx=0x7fac38008040 comp rx=0 tx=0).stop 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 msgr2=0x7fac4c0a9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 0x7fac4c0a9420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fac3c000c00 tx=0x7fac3c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 shutdown_connections 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac4c154cf0 0x7fac4c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fac2c06b4d0 0x7fac2c06d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac4c1523b0 0x7fac4c157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 --2- 172.21.15.67:0/1425503860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac4c071fd0 0x7fac4c0a9420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 >> 172.21.15.67:0/1425503860 conn(0x7fac4c0c6380 msgr2=0x7fac4c072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 shutdown_connections 2024-02-26T10:39:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.188+0000 7fac4b200640 1 -- 172.21.15.67:0/1425503860 wait complete. 2024-02-26T10:39:52.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:39:52.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.426+0000 7f5075200640 1 Processor -- start 2024-02-26T10:39:52.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.426+0000 7f5075200640 1 -- start start 2024-02-26T10:39:52.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.428+0000 7f5075200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.428+0000 7f5075200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f507006b000 0x7f507006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.428+0000 7f5075200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507006b460 con 0x7f5070153610 2024-02-26T10:39:52.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.428+0000 7f5075200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50700a9a80 con 0x7f507006b000 2024-02-26T10:39:52.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.428+0000 7f5075200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700a9d60 con 0x7f507006ac20 2024-02-26T10:39:52.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f507006b000 0x7f507006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:51734/0 (socket says 172.21.15.67:51734) 2024-02-26T10:39:52.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506f400640 1 -- 172.21.15.67:0/315490835 learned_addr learned my addr 172.21.15.67:0/315490835 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:52.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506ea00640 1 -- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 msgr2=0x7f50700a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:52.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506ea00640 1 --2- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506ea00640 1 -- 172.21.15.67:0/315490835 >> v1:172.21.15.82:6789/0 conn(0x7f5070153610 legacy=0x7f50700abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.429+0000 7f506ea00640 1 -- 172.21.15.67:0/315490835 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700ab750 con 0x7f507006b000 2024-02-26T10:39:52.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.431+0000 7f506ea00640 1 --2- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507006b000 0x7f507006db00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f505c0097a0 tx=0x7f505c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67ed33ea6c51599 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.431+0000 7f506e000640 1 -- 172.21.15.67:0/315490835 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c035020 con 0x7f507006b000 2024-02-26T10:39:52.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.431+0000 7f506e000640 1 -- 172.21.15.67:0/315490835 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c031830 con 0x7f507006b000 2024-02-26T10:39:52.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.431+0000 7f506e000640 1 -- 172.21.15.67:0/315490835 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c031c00 con 0x7f507006b000 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 -- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507006b000 msgr2=0x7f507006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 --2- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507006b000 0x7f507006db00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f505c0097a0 tx=0x7f505c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 -- 172.21.15.67:0/315490835 shutdown_connections 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 --2- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f507006b000 0x7f507006db00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 --2- 172.21.15.67:0/315490835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 -- 172.21.15.67:0/315490835 >> 172.21.15.67:0/315490835 conn(0x7f50700c63f0 msgr2=0x7f50700c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 -- 172.21.15.67:0/315490835 shutdown_connections 2024-02-26T10:39:52.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.432+0000 7f5075200640 1 -- 172.21.15.67:0/315490835 wait complete. 2024-02-26T10:39:52.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.434+0000 7f5075200640 1 Processor -- start 2024-02-26T10:39:52.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.434+0000 7f5075200640 1 -- start start 2024-02-26T10:39:52.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f5075200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f5075200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 0x7f507013daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f5075200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5070153610 0x7f507013dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f5075200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070150410 con 0x7f507006b000 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f5075200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070153aa0 con 0x7f5070153610 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f5075200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5070150590 con 0x7f507006ac20 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f506ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 0x7f507013daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f506ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 0x7f507013daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54222/0 (socket says 172.21.15.67:54222) 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f506ea00640 1 -- 172.21.15.67:0/3116601271 learned_addr learned my addr 172.21.15.67:0/3116601271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f506f400640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.435+0000 7f506fe00640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5070153610 0x7f507013dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f506ea00640 1 -- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 msgr2=0x7f50700bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f506ea00640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f506ea00640 1 -- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5070153610 msgr2=0x7f507013dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f506ea00640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5070153610 0x7f507013dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f506ea00640 1 -- 172.21.15.67:0/3116601271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070072c40 con 0x7f507006b000 2024-02-26T10:39:52.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f506ea00640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 0x7f507013daa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f505c02bee0 tx=0x7f505c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.436+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c03b980 con 0x7f507006b000 2024-02-26T10:39:52.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.437+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c03bb40 con 0x7f507006b000 2024-02-26T10:39:52.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.437+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c046450 con 0x7f507006b000 2024-02-26T10:39:52.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.437+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50701eb4f0 con 0x7f507006b000 2024-02-26T10:39:52.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.437+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50700bffd0 con 0x7f507006b000 2024-02-26T10:39:52.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.438+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50701eb320 con 0x7f507006b000 2024-02-26T10:39:52.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.439+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f505c038020 con 0x7f507006b000 2024-02-26T10:39:52.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.440+0000 7f5067400640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505006b4d0 0x7f505006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.440+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f505c0ac0f0 con 0x7f507006b000 2024-02-26T10:39:52.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.440+0000 7f506f400640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505006b4d0 0x7f505006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.442+0000 7f506f400640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505006b4d0 0x7f505006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5060009b60 tx=0x7f5060008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.444+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f505c07aee0 con 0x7f507006b000 2024-02-26T10:39:52.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.645+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f507006bea0 con 0x7f507006b000 2024-02-26T10:39:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.646+0000 7f5067400640 1 -- 172.21.15.67:0/3116601271 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+207 (secure 0 0 0) 0x7f507006bea0 con 0x7f507006b000 2024-02-26T10:39:52.646 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:52.646 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:39:52.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 4, 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "cephfs_metadata" 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 5, 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "cephfs_data" 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:39:52.647 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505006b4d0 msgr2=0x7f505006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505006b4d0 0x7f505006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5060009b60 tx=0x7f5060008040 comp rx=0 tx=0).stop 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 msgr2=0x7f507013daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 0x7f507013daa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f505c02bee0 tx=0x7f505c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 shutdown_connections 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505006b4d0 0x7f505006d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5070153610 0x7f507013dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507006b000 0x7f507013daa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 --2- 172.21.15.67:0/3116601271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f507006ac20 0x7f50700bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 >> 172.21.15.67:0/3116601271 conn(0x7f50700c63f0 msgr2=0x7f5070150970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:52.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 shutdown_connections 2024-02-26T10:39:52.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.648+0000 7f5075200640 1 -- 172.21.15.67:0/3116601271 wait complete. 2024-02-26T10:39:52.665 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-02-26T10:39:52.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-02-26T10:39:52.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.871+0000 7fd145600640 1 Processor -- start 2024-02-26T10:39:52.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.871+0000 7fd145600640 1 -- start start 2024-02-26T10:39:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.872+0000 7fd145600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1400c1eb0 0x7fd14014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.872+0000 7fd145600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400c1be0 con 0x7fd1400c25a0 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd145600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400c3660 con 0x7fd14014f260 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd145600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1400c1750 con 0x7fd1400c1eb0 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1400c1eb0 0x7fd14014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1400c1eb0 0x7fd14014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54228/0 (socket says 172.21.15.67:54228) 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd1400c25a0 0x7fd1400c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47290/0 (socket says 172.21.15.67:47290) 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13f400640 1 -- 172.21.15.67:0/3330897088 learned_addr learned my addr 172.21.15.67:0/3330897088 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13f400640 1 -- 172.21.15.67:0/3330897088 >> v1:172.21.15.82:6789/0 conn(0x7fd14014f260 legacy=0x7fd140151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13f400640 1 -- 172.21.15.67:0/3330897088 >> v1:172.21.15.67:6789/0 conn(0x7fd1400c25a0 legacy=0x7fd1400c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.873+0000 7fd13f400640 1 -- 172.21.15.67:0/3330897088 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400c0e20 con 0x7fd1400c1eb0 2024-02-26T10:39:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.875+0000 7fd13f400640 1 --2- 172.21.15.67:0/3330897088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c1eb0 0x7fd14014ed20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00bb10 tx=0x7fd12c02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31356a110eb811b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.875+0000 7fd13ea00640 1 -- 172.21.15.67:0/3330897088 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c03b070 con 0x7fd1400c1eb0 2024-02-26T10:39:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.875+0000 7fd13ea00640 1 -- 172.21.15.67:0/3330897088 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c036440 con 0x7fd1400c1eb0 2024-02-26T10:39:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.875+0000 7fd13ea00640 1 -- 172.21.15.67:0/3330897088 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c036810 con 0x7fd1400c1eb0 2024-02-26T10:39:52.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.875+0000 7fd145600640 1 -- 172.21.15.67:0/3330897088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c1eb0 msgr2=0x7fd14014ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.875+0000 7fd145600640 1 --2- 172.21.15.67:0/3330897088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c1eb0 0x7fd14014ed20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00bb10 tx=0x7fd12c02e0d0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.876+0000 7fd145600640 1 -- 172.21.15.67:0/3330897088 shutdown_connections 2024-02-26T10:39:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.876+0000 7fd145600640 1 --2- 172.21.15.67:0/3330897088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c1eb0 0x7fd14014ed20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.876+0000 7fd145600640 1 -- 172.21.15.67:0/3330897088 >> 172.21.15.67:0/3330897088 conn(0x7fd1400c6380 msgr2=0x7fd1400c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.876+0000 7fd145600640 1 -- 172.21.15.67:0/3330897088 shutdown_connections 2024-02-26T10:39:52.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.876+0000 7fd145600640 1 -- 172.21.15.67:0/3330897088 wait complete. 2024-02-26T10:39:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.877+0000 7fd145600640 1 Processor -- start 2024-02-26T10:39:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.877+0000 7fd145600640 1 -- start start 2024-02-26T10:39:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.878+0000 7fd145600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1400c1eb0 0x7fd14013f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.878+0000 7fd145600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 0x7fd140149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.878+0000 7fd145600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14014f260 0x7fd1401e6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.878+0000 7fd145600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400c13e0 con 0x7fd1400c25a0 2024-02-26T10:39:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.878+0000 7fd145600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14014f640 con 0x7fd14014f260 2024-02-26T10:39:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.878+0000 7fd145600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400c2980 con 0x7fd1400c1eb0 2024-02-26T10:39:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 0x7fd140149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 0x7fd140149480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54238/0 (socket says 172.21.15.67:54238) 2024-02-26T10:39:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 -- 172.21.15.67:0/2239091259 learned_addr learned my addr 172.21.15.67:0/2239091259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd144c00640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14014f260 0x7fd1401e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 -- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1400c1eb0 msgr2=0x7fd14013f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1400c1eb0 0x7fd14013f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 -- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14014f260 msgr2=0x7fd1401e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14014f260 0x7fd1401e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 -- 172.21.15.67:0/2239091259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14014ae60 con 0x7fd1400c25a0 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd144c00640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14014f260 0x7fd1401e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.879+0000 7fd13f400640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 0x7fd140149480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b2c0 tx=0x7fd12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.880+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c03b040 con 0x7fd1400c25a0 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.880+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c046070 con 0x7fd1400c25a0 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.880+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c034bd0 con 0x7fd1400c25a0 2024-02-26T10:39:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.880+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1401e9c70 con 0x7fd1400c25a0 2024-02-26T10:39:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.880+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1401ea230 con 0x7fd1400c25a0 2024-02-26T10:39:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.881+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1400678e0 con 0x7fd1400c25a0 2024-02-26T10:39:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.882+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd12c041440 con 0x7fd1400c25a0 2024-02-26T10:39:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.883+0000 7fd137e00640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd12006b520 0x7fd12006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.883+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd12c0ac820 con 0x7fd1400c25a0 2024-02-26T10:39:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.884+0000 7fd13fe00640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd12006b520 0x7fd12006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.885+0000 7fd13fe00640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd12006b520 0x7fd12006d9e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd130000f60 tx=0x7fd130007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:52.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:52.888+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd12c04f020 con 0x7fd1400c25a0 2024-02-26T10:39:53.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:53.091+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd140074a30 con 0x7fd1400c25a0 2024-02-26T10:39:54.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.086+0000 7fd137e00640 1 -- 172.21.15.67:0/2239091259 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7fd12c07b9f0 con 0x7fd1400c25a0 2024-02-26T10:39:54.087 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_metadata' created 2024-02-26T10:39:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.090+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd12006b520 msgr2=0x7fd12006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.090+0000 7fd145600640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd12006b520 0x7fd12006d9e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd130000f60 tx=0x7fd130007a00 comp rx=0 tx=0).stop 2024-02-26T10:39:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.090+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 msgr2=0x7fd140149480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.090+0000 7fd145600640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 0x7fd140149480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b2c0 tx=0x7fd12c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 shutdown_connections 2024-02-26T10:39:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd12006b520 0x7fd12006d9e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14014f260 0x7fd1401e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1400c25a0 0x7fd140149480 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 --2- 172.21.15.67:0/2239091259 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1400c1eb0 0x7fd14013f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 >> 172.21.15.67:0/2239091259 conn(0x7fd1400c6380 msgr2=0x7fd1401463e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 shutdown_connections 2024-02-26T10:39:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.091+0000 7fd145600640 1 -- 172.21.15.67:0/2239091259 wait complete. 2024-02-26T10:39:54.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:39:54.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.336+0000 7f9cb2e00640 1 Processor -- start 2024-02-26T10:39:54.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.336+0000 7f9cb2e00640 1 -- start start 2024-02-26T10:39:54.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.338+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac14cf30 0x7f9cac1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:54.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.338+0000 7f9cb2e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9cac06ea10 0x7f9cac06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:54.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.338+0000 7f9cb2e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac1542d0 con 0x7f9cac06b160 2024-02-26T10:39:54.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.338+0000 7f9cb2e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac154570 con 0x7f9cac06ea10 2024-02-26T10:39:54.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.338+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac154850 con 0x7f9cac14cf30 2024-02-26T10:39:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.339+0000 7f9cb0e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9cac06b160 0x7f9cac155050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:33686/0 (socket says 172.21.15.67:33686) 2024-02-26T10:39:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.339+0000 7f9cb0e00640 1 -- 172.21.15.67:0/2152974445 learned_addr learned my addr 172.21.15.67:0/2152974445 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.339+0000 7f9cab400640 1 --2- 172.21.15.67:0/2152974445 >> v2:172.21.15.82:3300/0 conn(0x7f9cac06ea10 0x7f9cac06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.339+0000 7f9cabe00640 1 --2- 172.21.15.67:0/2152974445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac14cf30 0x7f9cac1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9caaa00640 1 -- 172.21.15.67:0/2152974445 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381443740 0 0) 0x7f9cac1542d0 con 0x7f9cac06b160 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9caaa00640 1 -- 172.21.15.67:0/2152974445 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c90003520 con 0x7f9cac06b160 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9cabe00640 1 -- 172.21.15.67:0/2152974445 >> v2:172.21.15.82:3300/0 conn(0x7f9cac06ea10 msgr2=0x7f9cac06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9cabe00640 1 --2- 172.21.15.67:0/2152974445 >> v2:172.21.15.82:3300/0 conn(0x7f9cac06ea10 0x7f9cac06ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9cabe00640 1 -- 172.21.15.67:0/2152974445 >> v1:172.21.15.82:6789/0 conn(0x7f9cac06b160 legacy=0x7f9cac155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9cabe00640 1 -- 172.21.15.67:0/2152974445 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac154b30 con 0x7f9cac14cf30 2024-02-26T10:39:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.340+0000 7f9cab400640 1 --2- 172.21.15.67:0/2152974445 >> v2:172.21.15.82:3300/0 conn(0x7f9cac06ea10 0x7f9cac06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:39:54.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.342+0000 7f9cabe00640 1 --2- 172.21.15.67:0/2152974445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac14cf30 0x7f9cac1538e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0097a0 tx=0x7f9c9c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47a453d9cc9322e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.342+0000 7f9caaa00640 1 -- 172.21.15.67:0/2152974445 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0315a0 con 0x7f9cac14cf30 2024-02-26T10:39:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.342+0000 7f9caaa00640 1 -- 172.21.15.67:0/2152974445 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c037070 con 0x7f9cac14cf30 2024-02-26T10:39:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.342+0000 7f9caaa00640 1 -- 172.21.15.67:0/2152974445 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c031960 con 0x7f9cac14cf30 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 -- 172.21.15.67:0/2152974445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac14cf30 msgr2=0x7f9cac1538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/2152974445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac14cf30 0x7f9cac1538e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0097a0 tx=0x7f9c9c02b840 comp rx=0 tx=0).stop 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 -- 172.21.15.67:0/2152974445 shutdown_connections 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/2152974445 >> v2:172.21.15.82:3300/0 conn(0x7f9cac06ea10 0x7f9cac06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/2152974445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac14cf30 0x7f9cac1538e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 -- 172.21.15.67:0/2152974445 >> 172.21.15.67:0/2152974445 conn(0x7f9cac0c7160 msgr2=0x7f9cac0a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 -- 172.21.15.67:0/2152974445 shutdown_connections 2024-02-26T10:39:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.343+0000 7f9cb2e00640 1 -- 172.21.15.67:0/2152974445 wait complete. 2024-02-26T10:39:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.345+0000 7f9cb2e00640 1 Processor -- start 2024-02-26T10:39:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.345+0000 7f9cb2e00640 1 -- start start 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.346+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cac06b160 0x7f9cac149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.346+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06ea10 0x7f9cac0724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.346+0000 7f9cb2e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac14cf30 0x7f9cac1ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.346+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac06e790 con 0x7f9cac06b160 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.346+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac06e610 con 0x7f9cac14cf30 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.346+0000 7f9cb2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac155760 con 0x7f9cac06ea10 2024-02-26T10:39:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.347+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cac06b160 0x7f9cac149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.347+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cac06b160 0x7f9cac149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54254/0 (socket says 172.21.15.67:54254) 2024-02-26T10:39:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.347+0000 7f9cabe00640 1 -- 172.21.15.67:0/3512949586 learned_addr learned my addr 172.21.15.67:0/3512949586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.347+0000 7f9cb0e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac14cf30 0x7f9cac1ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.347+0000 7f9cab400640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06ea10 0x7f9cac0724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.348+0000 7f9cab400640 1 -- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac14cf30 msgr2=0x7f9cac1ee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.348+0000 7f9cab400640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac14cf30 0x7f9cac1ee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.348+0000 7f9cab400640 1 -- 172.21.15.67:0/3512949586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cac06b160 msgr2=0x7f9cac149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.348+0000 7f9cab400640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cac06b160 0x7f9cac149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.348+0000 7f9cab400640 1 -- 172.21.15.67:0/3512949586 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac149aa0 con 0x7f9cac06ea10 2024-02-26T10:39:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.348+0000 7f9cab400640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06ea10 0x7f9cac0724e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800e930 tx=0x7f9c9800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.349+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98017070 con 0x7f9cac06ea10 2024-02-26T10:39:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.349+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9800ca00 con 0x7f9cac06ea10 2024-02-26T10:39:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.349+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800cdd0 con 0x7f9cac06ea10 2024-02-26T10:39:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.349+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac0c44c0 con 0x7f9cac06ea10 2024-02-26T10:39:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.349+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac0ab290 con 0x7f9cac06ea10 2024-02-26T10:39:54.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.350+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac074a30 con 0x7f9cac06ea10 2024-02-26T10:39:54.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.353+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c9801a070 con 0x7f9cac06ea10 2024-02-26T10:39:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.353+0000 7f9ca3e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.354+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6080+0+0 (secure 0 0 0) 0x7f9c9808a810 con 0x7f9cac06ea10 2024-02-26T10:39:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.354+0000 7f9cabe00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:54.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.355+0000 7f9cabe00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02b880 tx=0x7f9c9c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:54.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.369+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c98058fe0 con 0x7f9cac06ea10 2024-02-26T10:39:54.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:54.569+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9cac0a5fe0 con 0x7f9cac06ea10 2024-02-26T10:39:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.114+0000 7f9ca3e00640 1 -- 172.21.15.67:0/3512949586 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v36) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9c9805d080 con 0x7f9cac06ea10 2024-02-26T10:39:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_data' created 2024-02-26T10:39:55.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.125+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9c8c06b4d0 msgr2=0x7f9c8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:55.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.125+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02b880 tx=0x7f9c9c03d040 comp rx=0 tx=0).stop 2024-02-26T10:39:55.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.125+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06ea10 msgr2=0x7f9cac0724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:55.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.125+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06ea10 0x7f9cac0724e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800e930 tx=0x7f9c9800edf0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.126+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 shutdown_connections 2024-02-26T10:39:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.126+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9c8c06b4d0 0x7f9c8c06d990 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c02b880 tx=0x7f9c9c03d040 comp rx=0 tx=0).stop 2024-02-26T10:39:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.126+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac14cf30 0x7f9cac1ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.126+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06ea10 0x7f9cac0724e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.126+0000 7f9cb2e00640 1 --2- 172.21.15.67:0/3512949586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cac06b160 0x7f9cac149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.126+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 >> 172.21.15.67:0/3512949586 conn(0x7f9cac0c7160 msgr2=0x7f9cac156410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:55.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.127+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 shutdown_connections 2024-02-26T10:39:55.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.127+0000 7f9cb2e00640 1 -- 172.21.15.67:0/3512949586 wait complete. 2024-02-26T10:39:55.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-02-26T10:39:55.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.343+0000 7f80f5000640 1 Processor -- start 2024-02-26T10:39:55.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.343+0000 7f80f5000640 1 -- start start 2024-02-26T10:39:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.344+0000 7f80f5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.344+0000 7f80f5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80f0153920 0x7f80f015d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.344+0000 7f80f5000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f00a6eb0 con 0x7f80f01503d0 2024-02-26T10:39:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.344+0000 7f80f5000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f00a7190 con 0x7f80f0153920 2024-02-26T10:39:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.344+0000 7f80f5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00a7470 con 0x7f80f00c2950 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80f0153920 0x7f80f015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80f0153920 0x7f80f015d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54266/0 (socket says 172.21.15.67:54266) 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 -- 172.21.15.67:0/801898349 learned_addr learned my addr 172.21.15.67:0/801898349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ed400640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 -- 172.21.15.67:0/801898349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 msgr2=0x7f80f01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 -- 172.21.15.67:0/801898349 >> v1:172.21.15.82:6789/0 conn(0x7f80f01503d0 legacy=0x7f80f006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:55.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ee800640 1 -- 172.21.15.67:0/801898349 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00a7750 con 0x7f80f0153920 2024-02-26T10:39:55.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.345+0000 7f80ed400640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:55.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.347+0000 7f80ee800640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f0153920 0x7f80f015d650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009e50 tx=0x7f80e802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8ade565bec3e09 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:55.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.347+0000 7f80eca00640 1 -- 172.21.15.67:0/801898349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e802c950 con 0x7f80f0153920 2024-02-26T10:39:55.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.347+0000 7f80eca00640 1 -- 172.21.15.67:0/801898349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e802caf0 con 0x7f80f0153920 2024-02-26T10:39:55.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.347+0000 7f80eca00640 1 -- 172.21.15.67:0/801898349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80343f0 con 0x7f80f0153920 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.347+0000 7f80f5000640 1 -- 172.21.15.67:0/801898349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f0153920 msgr2=0x7f80f015d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.347+0000 7f80f5000640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f0153920 0x7f80f015d650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009e50 tx=0x7f80e802bdb0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.348+0000 7f80f5000640 1 -- 172.21.15.67:0/801898349 shutdown_connections 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.348+0000 7f80f5000640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f0153920 0x7f80f015d650 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.348+0000 7f80f5000640 1 --2- 172.21.15.67:0/801898349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.348+0000 7f80f5000640 1 -- 172.21.15.67:0/801898349 >> 172.21.15.67:0/801898349 conn(0x7f80f00c6da0 msgr2=0x7f80f00c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.348+0000 7f80f5000640 1 -- 172.21.15.67:0/801898349 shutdown_connections 2024-02-26T10:39:55.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.348+0000 7f80f5000640 1 -- 172.21.15.67:0/801898349 wait complete. 2024-02-26T10:39:55.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.349+0000 7f80f5000640 1 Processor -- start 2024-02-26T10:39:55.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.349+0000 7f80f5000640 1 -- start start 2024-02-26T10:39:55.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.350+0000 7f80f5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01eb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:55.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.350+0000 7f80f5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 0x7f80f01ec060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:55.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.350+0000 7f80f5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80f0153920 0x7f80f00bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:55.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.350+0000 7f80f5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01507b0 con 0x7f80f01503d0 2024-02-26T10:39:55.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.350+0000 7f80f5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f006ef00 con 0x7f80f0153920 2024-02-26T10:39:55.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.350+0000 7f80f5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f006e560 con 0x7f80f00c2950 2024-02-26T10:39:55.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 0x7f80f01ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:55.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 0x7f80f01ec060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54270/0 (socket says 172.21.15.67:54270) 2024-02-26T10:39:55.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 -- 172.21.15.67:0/3833979207 learned_addr learned my addr 172.21.15.67:0/3833979207 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ede00640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ee800640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80f0153920 0x7f80f00bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 -- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 msgr2=0x7f80f01eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01eb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 -- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80f0153920 msgr2=0x7f80f00bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80f0153920 0x7f80f00bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 -- 172.21.15.67:0/3833979207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f014a1c0 con 0x7f80f01503d0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.351+0000 7f80ed400640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 0x7f80f01ec060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c820 tx=0x7f80d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.352+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8015070 con 0x7f80f01503d0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.352+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8007a30 con 0x7f80f01503d0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.352+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f0072990 con 0x7f80f01503d0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.352+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8007de0 con 0x7f80f01503d0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.352+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f0071070 con 0x7f80f01503d0 2024-02-26T10:39:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.353+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f00c2740 con 0x7f80f01503d0 2024-02-26T10:39:55.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.354+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80d8018070 con 0x7f80f01503d0 2024-02-26T10:39:55.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.356+0000 7f80e6000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f80cc06b4d0 0x7f80cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:55.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.356+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f80d808a310 con 0x7f80f01503d0 2024-02-26T10:39:55.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.356+0000 7f80ede00640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f80cc06b4d0 0x7f80cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:55.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.357+0000 7f80ede00640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f80cc06b4d0 0x7f80cc06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006fd0 tx=0x7f80e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.361+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80f00c2740 con 0x7f80f01503d0 2024-02-26T10:39:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:55.610+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f80f0072eb0 con 0x7f80f01503d0 2024-02-26T10:39:56.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.105+0000 7f80e6000640 1 -- 172.21.15.67:0/3833979207 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 6 and data pool 7 v18) v1 ==== 175+0+0 (secure 0 0 0) 0x7f80d8058f30 con 0x7f80f01503d0 2024-02-26T10:39:56.106 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 6 and data pool 7 2024-02-26T10:39:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f80cc06b4d0 msgr2=0x7f80cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f80cc06b4d0 0x7f80cc06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e0006fd0 tx=0x7f80e0008040 comp rx=0 tx=0).stop 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 msgr2=0x7f80f01ec060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 0x7f80f01ec060 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c820 tx=0x7f80d800cce0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 shutdown_connections 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f80cc06b4d0 0x7f80cc06d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80f0153920 0x7f80f00bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80f01503d0 0x7f80f01ec060 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 --2- 172.21.15.67:0/3833979207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f80f00c2950 0x7f80f01eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.110+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 >> 172.21.15.67:0/3833979207 conn(0x7f80f00c6da0 msgr2=0x7f80f00c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.111+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 shutdown_connections 2024-02-26T10:39:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.111+0000 7f80f5000640 1 -- 172.21.15.67:0/3833979207 wait complete. 2024-02-26T10:39:56.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:56.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.345+0000 7fa060800640 1 Processor -- start 2024-02-26T10:39:56.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.345+0000 7fa060800640 1 -- start start 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.346+0000 7fa060800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa05814f3d0 0x7fa05806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.346+0000 7fa060800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0580c2870 0x7fa058155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa060800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0581556a0 0x7fa058071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa060800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05806dd80 con 0x7fa05814f3d0 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa060800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05806ed10 con 0x7fa0580c2870 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa060800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa058155c00 con 0x7fa0581556a0 2024-02-26T10:39:56.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa05814f3d0 0x7fa05806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa05814f3d0 0x7fa05806e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54272/0 (socket says 172.21.15.67:54272) 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 -- 172.21.15.67:0/593090774 learned_addr learned my addr 172.21.15.67:0/593090774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05e800640 1 --2- 172.21.15.67:0/593090774 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0581556a0 0x7fa058071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05d400640 1 --2- 172.21.15.67:0/593090774 >> v2:172.21.15.82:3300/0 conn(0x7fa0580c2870 0x7fa058155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 -- 172.21.15.67:0/593090774 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0581556a0 msgr2=0x7fa058071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 --2- 172.21.15.67:0/593090774 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0581556a0 0x7fa058071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 -- 172.21.15.67:0/593090774 >> v2:172.21.15.82:3300/0 conn(0x7fa0580c2870 msgr2=0x7fa058155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 --2- 172.21.15.67:0/593090774 >> v2:172.21.15.82:3300/0 conn(0x7fa0580c2870 0x7fa058155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05de00640 1 -- 172.21.15.67:0/593090774 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058155ee0 con 0x7fa05814f3d0 2024-02-26T10:39:56.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.347+0000 7fa05d400640 1 --2- 172.21.15.67:0/593090774 >> v2:172.21.15.82:3300/0 conn(0x7fa0580c2870 0x7fa058155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.349+0000 7fa05de00640 1 --2- 172.21.15.67:0/593090774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05814f3d0 0x7fa05806e7d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa0580c1120 tx=0x7fa04c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d31fcef64edc6c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.349+0000 7fa05ca00640 1 -- 172.21.15.67:0/593090774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c02e070 con 0x7fa05814f3d0 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.349+0000 7fa05ca00640 1 -- 172.21.15.67:0/593090774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c037070 con 0x7fa05814f3d0 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.349+0000 7fa05ca00640 1 -- 172.21.15.67:0/593090774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c031a00 con 0x7fa05814f3d0 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 -- 172.21.15.67:0/593090774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05814f3d0 msgr2=0x7fa05806e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 --2- 172.21.15.67:0/593090774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05814f3d0 0x7fa05806e7d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa0580c1120 tx=0x7fa04c02b880 comp rx=0 tx=0).stop 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 -- 172.21.15.67:0/593090774 shutdown_connections 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 --2- 172.21.15.67:0/593090774 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0581556a0 0x7fa058071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 --2- 172.21.15.67:0/593090774 >> v2:172.21.15.82:3300/0 conn(0x7fa0580c2870 0x7fa058155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 --2- 172.21.15.67:0/593090774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05814f3d0 0x7fa05806e7d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 -- 172.21.15.67:0/593090774 >> 172.21.15.67:0/593090774 conn(0x7fa0580a84a0 msgr2=0x7fa0580a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 -- 172.21.15.67:0/593090774 shutdown_connections 2024-02-26T10:39:56.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.350+0000 7fa060800640 1 -- 172.21.15.67:0/593090774 wait complete. 2024-02-26T10:39:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.352+0000 7fa060800640 1 Processor -- start 2024-02-26T10:39:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.352+0000 7fa060800640 1 -- start start 2024-02-26T10:39:56.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa060800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 0x7fa0581e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa060800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa05814f3d0 0x7fa0580c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa060800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0581556a0 0x7fa0581ec720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa060800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580c1810 con 0x7fa0580c2870 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa060800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05806e2b0 con 0x7fa0581556a0 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa060800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580c2cd0 con 0x7fa05814f3d0 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa05de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 0x7fa0581e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa05de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 0x7fa0581e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54274/0 (socket says 172.21.15.67:54274) 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa05de00640 1 -- 172.21.15.67:0/1242799761 learned_addr learned my addr 172.21.15.67:0/1242799761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.353+0000 7fa05d400640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa05814f3d0 0x7fa0580c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05e800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0581556a0 0x7fa0581ec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05de00640 1 -- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa05814f3d0 msgr2=0x7fa0580c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05de00640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa05814f3d0 0x7fa0580c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05de00640 1 -- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0581556a0 msgr2=0x7fa0581ec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05de00640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0581556a0 0x7fa0581ec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05de00640 1 -- 172.21.15.67:0/1242799761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058146400 con 0x7fa0580c2870 2024-02-26T10:39:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05e800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0581556a0 0x7fa0581ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:39:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa05de00640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 0x7fa0581e88a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000c00 tx=0x7fa04c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c032670 con 0x7fa0580c2870 2024-02-26T10:39:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c032850 con 0x7fa0580c2870 2024-02-26T10:39:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c032c40 con 0x7fa0580c2870 2024-02-26T10:39:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.354+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0581e8de0 con 0x7fa0580c2870 2024-02-26T10:39:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.355+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa058146120 con 0x7fa0580c2870 2024-02-26T10:39:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.356+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa058074a30 con 0x7fa0580c2870 2024-02-26T10:39:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.357+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa04c04c060 con 0x7fa0580c2870 2024-02-26T10:39:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.358+0000 7fa052000640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa03c06b4d0 0x7fa03c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.358+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa04c0abf20 con 0x7fa0580c2870 2024-02-26T10:39:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.359+0000 7fa05d400640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa03c06b4d0 0x7fa03c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.360+0000 7fa05d400640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa03c06b4d0 0x7fa03c06d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa0580c0a70 tx=0x7fa048008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.362+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa04c040070 con 0x7fa0580c2870 2024-02-26T10:39:56.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.560+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0580c13f0 con 0x7fa0580c2870 2024-02-26T10:39:56.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.563+0000 7fa052000640 1 -- 172.21.15.67:0/1242799761 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+20336 (secure 0 0 0) 0x7fa04c07aad0 con 0x7fa0580c2870 2024-02-26T10:39:56.564 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:56.564 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:55.612556+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa03c06b4d0 msgr2=0x7fa03c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa03c06b4d0 0x7fa03c06d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa0580c0a70 tx=0x7fa048008040 comp rx=0 tx=0).stop 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 msgr2=0x7fa0581e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 0x7fa0581e88a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa04c000c00 tx=0x7fa04c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 shutdown_connections 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa03c06b4d0 0x7fa03c06d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0581556a0 0x7fa0581ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa05814f3d0 0x7fa0580c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 --2- 172.21.15.67:0/1242799761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0580c2870 0x7fa0581e88a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 >> 172.21.15.67:0/1242799761 conn(0x7fa0580a84a0 msgr2=0x7fa058152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:56.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 shutdown_connections 2024-02-26T10:39:56.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.566+0000 7fa060800640 1 -- 172.21.15.67:0/1242799761 wait complete. 2024-02-26T10:39:56.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:56.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.782+0000 7f8739600640 1 Processor -- start 2024-02-26T10:39:56.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.782+0000 7f8739600640 1 -- start start 2024-02-26T10:39:56.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.783+0000 7f8739600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 0x7f873406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.784+0000 7f8739600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873406ed10 con 0x7f8734071700 2024-02-26T10:39:56.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.784+0000 7f8739600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8734155c00 con 0x7f87340c2870 2024-02-26T10:39:56.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.784+0000 7f8739600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873406dd80 con 0x7f873414f3d0 2024-02-26T10:39:56.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.784+0000 7f8733400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8734071700 0x7f873415c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47304/0 (socket says 172.21.15.67:47304) 2024-02-26T10:39:56.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.784+0000 7f8733400640 1 -- 172.21.15.67:0/1855365557 learned_addr learned my addr 172.21.15.67:0/1855365557 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.784+0000 7f8732a00640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 0x7f873406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357651230 0 0) 0x7f873406ed10 con 0x7f8734071700 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8718005730 con 0x7f8734071700 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455140366 0 0) 0x7f8734155c00 con 0x7f87340c2870 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f873406ed10 con 0x7f87340c2870 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710004868 0 0) 0x7f8718005730 con 0x7f8734071700 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 msgr2=0x7f873406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 0x7f873406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 >> v1:172.21.15.82:6789/0 conn(0x7f87340c2870 legacy=0x7f8734155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734155ee0 con 0x7f8734071700 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f872c001670 con 0x7f8734071700 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 >> v1:172.21.15.67:6789/0 conn(0x7f8734071700 legacy=0x7f873415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8718005d80 0x7f8718006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 0x7f8718006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718006f50 0x7f87180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873406dd80 con 0x7f8718005d80 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871800dd60 con 0x7f8718006f50 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871800df30 con 0x7f873414f3d0 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.785+0000 7f8732a00640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8718005d80 0x7f8718006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8732000640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 0x7f8718006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8733400640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718006f50 0x7f87180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8732a00640 1 -- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 msgr2=0x7f8718006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8732a00640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f873414f3d0 0x7f8718006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8732a00640 1 -- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718006f50 msgr2=0x7f87180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:56.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8732a00640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718006f50 0x7f87180073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.786+0000 7f8732a00640 1 -- 172.21.15.67:0/1855365557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8718005730 con 0x7f8718005d80 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.787+0000 7f8732a00640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8718005d80 0x7f8718006180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f872400e170 tx=0x7f8718007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dd859a616a6e22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.787+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724039070 con 0x7f8718005d80 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.787+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872402cbf0 con 0x7f8718005d80 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.787+0000 7f8731600640 1 -- 172.21.15.67:0/1855365557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724009c10 con 0x7f8718005d80 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8739600640 1 -- 172.21.15.67:0/1855365557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8718005d80 msgr2=0x7f8718006180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8739600640 1 --2- 172.21.15.67:0/1855365557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8718005d80 0x7f8718006180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f872400e170 tx=0x7f8718007a40 comp rx=0 tx=0).stop 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8733400640 1 -- 172.21.15.67:0/1855365557 reap_dead start 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8739600640 1 -- 172.21.15.67:0/1855365557 shutdown_connections 2024-02-26T10:39:56.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8739600640 1 -- 172.21.15.67:0/1855365557 >> 172.21.15.67:0/1855365557 conn(0x7f87340a84a0 msgr2=0x7f87340a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:56.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8739600640 1 -- 172.21.15.67:0/1855365557 shutdown_connections 2024-02-26T10:39:56.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.788+0000 7f8739600640 1 -- 172.21.15.67:0/1855365557 wait complete. 2024-02-26T10:39:56.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.790+0000 7f8739600640 1 Processor -- start 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.790+0000 7f8739600640 1 -- start start 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8739600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718005d80 0x7f87340c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8739600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87341ee9d0 0x7f87340bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8739600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 0x7f873413edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8739600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87340c1810 con 0x7f87341eb310 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8739600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873406e2b0 con 0x7f8718005d80 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8739600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8734155870 con 0x7f87341ee9d0 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8733400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 0x7f873413edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8733400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 0x7f873413edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54278/0 (socket says 172.21.15.67:54278) 2024-02-26T10:39:56.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8733400640 1 -- 172.21.15.67:0/1998231534 learned_addr learned my addr 172.21.15.67:0/1998231534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8732000640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87341ee9d0 0x7f87340bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.791+0000 7f8732a00640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718005d80 0x7f87340c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8733400640 1 -- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87341ee9d0 msgr2=0x7f87340bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8733400640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87341ee9d0 0x7f87340bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8733400640 1 -- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718005d80 msgr2=0x7f87340c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8733400640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718005d80 0x7f87340c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8733400640 1 -- 172.21.15.67:0/1998231534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340c4260 con 0x7f87341eb310 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8733400640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 0x7f873413edc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f87340c41c0 tx=0x7f872c00d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0074e0 con 0x7f87341eb310 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c0047e0 con 0x7f87341eb310 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8734151fa0 con 0x7f87341eb310 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c004bd0 con 0x7f87341eb310 2024-02-26T10:39:56.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.792+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8734152560 con 0x7f87341eb310 2024-02-26T10:39:56.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.793+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8734074a30 con 0x7f87341eb310 2024-02-26T10:39:56.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.794+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f872c01b020 con 0x7f87341eb310 2024-02-26T10:39:56.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.795+0000 7f872aa00640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f871006b500 0x7f871006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:56.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.795+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f872c089fe0 con 0x7f87341eb310 2024-02-26T10:39:56.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.796+0000 7f8732a00640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f871006b500 0x7f871006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:56.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.797+0000 7f8732a00640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f871006b500 0x7f871006d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8724036000 tx=0x7f872402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:56.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.800+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f872c00f2f0 con 0x7f87341eb310 2024-02-26T10:39:56.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.994+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873414a520 con 0x7f87341eb310 2024-02-26T10:39:56.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.996+0000 7f872aa00640 1 -- 172.21.15.67:0/1998231534 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+20336 (secure 0 0 0) 0x7f872c058b30 con 0x7f87341eb310 2024-02-26T10:39:56.998 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:56.998 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:55.612556+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:39:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f871006b500 msgr2=0x7f871006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f871006b500 0x7f871006d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8724036000 tx=0x7f872402e000 comp rx=0 tx=0).stop 2024-02-26T10:39:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 msgr2=0x7f873413edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 0x7f873413edc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f87340c41c0 tx=0x7f872c00d950 comp rx=0 tx=0).stop 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 shutdown_connections 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87341eb310 0x7f873413edc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87341ee9d0 0x7f87340bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f871006b500 0x7f871006d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 --2- 172.21.15.67:0/1998231534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8718005d80 0x7f87340c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:56.999+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 >> 172.21.15.67:0/1998231534 conn(0x7f87340a84a0 msgr2=0x7f873406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.000+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 shutdown_connections 2024-02-26T10:39:57.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.000+0000 7f8739600640 1 -- 172.21.15.67:0/1998231534 wait complete. 2024-02-26T10:39:57.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-02-26T10:39:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.226+0000 7f1bc5a00640 1 Processor -- start 2024-02-26T10:39:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.226+0000 7f1bc5a00640 1 -- start start 2024-02-26T10:39:57.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.227+0000 7f1bc5a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1bc01503d0 0x7f1bc006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.227+0000 7f1bc5a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00a6eb0 con 0x7f1bc0153af0 2024-02-26T10:39:57.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.227+0000 7f1bc5a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00a7190 con 0x7f1bc006e390 2024-02-26T10:39:57.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.227+0000 7f1bc5a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc00a7470 con 0x7f1bc01503d0 2024-02-26T10:39:57.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.228+0000 7f1bbf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1bc0153af0 0x7f1bc015d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47306/0 (socket says 172.21.15.67:47306) 2024-02-26T10:39:57.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.228+0000 7f1bbf400640 1 -- 172.21.15.67:0/4198579931 learned_addr learned my addr 172.21.15.67:0/4198579931 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.228+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164087114 0 0) 0x7f1bc00a6eb0 con 0x7f1bc0153af0 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.228+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba4003520 con 0x7f1bc0153af0 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.228+0000 7f1bbea00640 1 --2- 172.21.15.67:0/4198579931 >> v2:172.21.15.82:3300/0 conn(0x7f1bc01503d0 0x7f1bc006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3720871180 0 0) 0x7f1ba4003520 con 0x7f1bc0153af0 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 >> v2:172.21.15.82:3300/0 conn(0x7f1bc01503d0 msgr2=0x7f1bc006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 --2- 172.21.15.67:0/4198579931 >> v2:172.21.15.82:3300/0 conn(0x7f1bc01503d0 0x7f1bc006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 >> v1:172.21.15.82:6789/0 conn(0x7f1bc006e390 legacy=0x7f1bc01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00a7750 con 0x7f1bc0153af0 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1bb80019f0 con 0x7f1bc0153af0 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 >> v1:172.21.15.67:6789/0 conn(0x7f1bc0153af0 legacy=0x7f1bc015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ba40039b0 0x7f1ba4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ba4004800 0x7f1ba4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba40053c0 0x7f1ba400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00a7470 con 0x7f1ba4004800 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba400c250 con 0x7f1ba40053c0 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba400c450 con 0x7f1ba40039b0 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbe000640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ba4004800 0x7f1ba4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.229+0000 7f1bbea00640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ba40039b0 0x7f1ba4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbe000640 1 -- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ba40039b0 msgr2=0x7f1ba4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbe000640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ba40039b0 0x7f1ba4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbf400640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba40053c0 0x7f1ba400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbe000640 1 -- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba40053c0 msgr2=0x7f1ba400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbe000640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba40053c0 0x7f1ba400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbe000640 1 -- 172.21.15.67:0/4198579931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba4003520 con 0x7f1ba4004800 2024-02-26T10:39:57.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.230+0000 7f1bbf400640 1 -- 172.21.15.67:0/4198579931 reap_dead start 2024-02-26T10:39:57.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.231+0000 7f1bbe000640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ba4004800 0x7f1ba4004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1bac001750 tx=0x7f1ba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c17eff55eb76287 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:57.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.231+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac001370 con 0x7f1ba4004800 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.231+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac0052d0 con 0x7f1ba4004800 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.231+0000 7f1bbd600640 1 -- 172.21.15.67:0/4198579931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac003830 con 0x7f1ba4004800 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.231+0000 7f1bc5a00640 1 -- 172.21.15.67:0/4198579931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ba4004800 msgr2=0x7f1ba4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.231+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ba4004800 0x7f1ba4004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1bac001750 tx=0x7f1ba4005830 comp rx=0 tx=0).stop 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.232+0000 7f1bc5a00640 1 -- 172.21.15.67:0/4198579931 shutdown_connections 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.232+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/4198579931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ba4004800 0x7f1ba4004c80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.232+0000 7f1bc5a00640 1 -- 172.21.15.67:0/4198579931 >> 172.21.15.67:0/4198579931 conn(0x7f1bc00c6da0 msgr2=0x7f1bc00c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.232+0000 7f1bc5a00640 1 -- 172.21.15.67:0/4198579931 shutdown_connections 2024-02-26T10:39:57.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.232+0000 7f1bc5a00640 1 -- 172.21.15.67:0/4198579931 wait complete. 2024-02-26T10:39:57.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.233+0000 7f1bc5a00640 1 Processor -- start 2024-02-26T10:39:57.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 -- start start 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba4004800 0x7f1bc00abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bc0072b70 0x7f1bc01eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 0x7f1bc0148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01507b0 con 0x7f1bc01eea00 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc006ef00 con 0x7f1ba4004800 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.234+0000 7f1bc5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00c0ef0 con 0x7f1bc0072b70 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 0x7f1bc0148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 0x7f1bc0148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54292/0 (socket says 172.21.15.67:54292) 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 -- 172.21.15.67:0/457836718 learned_addr learned my addr 172.21.15.67:0/457836718 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 -- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bc0072b70 msgr2=0x7f1bc01eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bc0072b70 0x7f1bc01eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 -- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba4004800 msgr2=0x7f1bc00abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba4004800 0x7f1bc00abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:57.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 -- 172.21.15.67:0/457836718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0071720 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.235+0000 7f1bbf400640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 0x7f1bc0148850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000f80 tx=0x7f1bb800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.236+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800b8d0 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.236+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb800ba70 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.236+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc0071a00 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.236+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800be60 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.236+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc01ee200 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.237+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc0074a30 con 0x7f1bc01eea00 2024-02-26T10:39:57.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.238+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bb801b060 con 0x7f1bc01eea00 2024-02-26T10:39:57.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.239+0000 7f1bb6a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ba006b4d0 0x7f1ba006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:57.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.239+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1bb808a2d0 con 0x7f1bc01eea00 2024-02-26T10:39:57.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.239+0000 7f1bbea00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ba006b4d0 0x7f1ba006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:57.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.241+0000 7f1bbea00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ba006b4d0 0x7f1ba006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006a90 tx=0x7f1bb0006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:57.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.243+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bb8058e80 con 0x7f1bc01eea00 2024-02-26T10:39:57.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:57.486+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1bc0151340 con 0x7f1bc01eea00 2024-02-26T10:39:57.824 INFO:tasks.ceph.mds.c.smithi067.stderr:2024-02-26T10:39:57.823+0000 7f5b3a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:39:58.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.149+0000 7f1bb6a00640 1 -- 172.21.15.67:0/457836718 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 119+0+0 (secure 0 0 0) 0x7f1bb8058780 con 0x7f1bc01eea00 2024-02-26T10:39:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.152+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ba006b4d0 msgr2=0x7f1ba006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.152+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ba006b4d0 0x7f1ba006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006a90 tx=0x7f1bb0006a20 comp rx=0 tx=0).stop 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.152+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 msgr2=0x7f1bc0148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.152+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 0x7f1bc0148850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000f80 tx=0x7f1bb800b0d0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 shutdown_connections 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bc01eea00 0x7f1bc0148850 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bc0072b70 0x7f1bc01eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ba006b4d0 0x7f1ba006d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 --2- 172.21.15.67:0/457836718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ba4004800 0x7f1bc00abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 >> 172.21.15.67:0/457836718 conn(0x7f1bc00c6da0 msgr2=0x7f1bc00c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 shutdown_connections 2024-02-26T10:39:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.153+0000 7f1bc5a00640 1 -- 172.21.15.67:0/457836718 wait complete. 2024-02-26T10:39:58.176 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:39:58.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-02-26T10:39:58.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.405+0000 7f4991800640 1 Processor -- start 2024-02-26T10:39:58.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.406+0000 7f4991800640 1 -- start start 2024-02-26T10:39:58.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.407+0000 7f4991800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 0x7f498c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:58.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.407+0000 7f4991800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f498c155880 0x7f498c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:58.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.407+0000 7f4991800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c06c110 con 0x7f498c06ce30 2024-02-26T10:39:58.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.407+0000 7f4991800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0c72c0 con 0x7f498c155880 2024-02-26T10:39:58.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.407+0000 7f4991800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0c75a0 con 0x7f498c0c5290 2024-02-26T10:39:58.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.408+0000 7f498be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 0x7f498c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:58.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.408+0000 7f498be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 0x7f498c1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:51874/0 (socket says 172.21.15.67:51874) 2024-02-26T10:39:58.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.408+0000 7f498be00640 1 -- 172.21.15.67:0/3177557439 learned_addr learned my addr 172.21.15.67:0/3177557439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:58.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.408+0000 7f498b400640 1 --2- 172.21.15.67:0/3177557439 >> v2:172.21.15.82:3300/0 conn(0x7f498c155880 0x7f498c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:58.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.409+0000 7f498be00640 1 -- 172.21.15.67:0/3177557439 >> v2:172.21.15.82:3300/0 conn(0x7f498c155880 msgr2=0x7f498c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:58.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.409+0000 7f498be00640 1 --2- 172.21.15.67:0/3177557439 >> v2:172.21.15.82:3300/0 conn(0x7f498c155880 0x7f498c06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.409+0000 7f498be00640 1 -- 172.21.15.67:0/3177557439 >> v1:172.21.15.82:6789/0 conn(0x7f498c06ce30 legacy=0x7f498c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:58.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.409+0000 7f498be00640 1 -- 172.21.15.67:0/3177557439 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0c7880 con 0x7f498c0c5290 2024-02-26T10:39:58.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.410+0000 7f498be00640 1 --2- 172.21.15.67:0/3177557439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 0x7f498c1562f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f497c009e50 tx=0x7f497c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15df0807f394c266 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:58.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f498aa00640 1 -- 172.21.15.67:0/3177557439 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c02ca70 con 0x7f498c0c5290 2024-02-26T10:39:58.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f498aa00640 1 -- 172.21.15.67:0/3177557439 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c02cc10 con 0x7f498c0c5290 2024-02-26T10:39:58.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f498aa00640 1 -- 172.21.15.67:0/3177557439 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c0343f0 con 0x7f498c0c5290 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 -- 172.21.15.67:0/3177557439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 msgr2=0x7f498c1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 --2- 172.21.15.67:0/3177557439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 0x7f498c1562f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f497c009e50 tx=0x7f497c02bed0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 -- 172.21.15.67:0/3177557439 shutdown_connections 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 --2- 172.21.15.67:0/3177557439 >> v2:172.21.15.82:3300/0 conn(0x7f498c155880 0x7f498c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 --2- 172.21.15.67:0/3177557439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c5290 0x7f498c1562f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 -- 172.21.15.67:0/3177557439 >> 172.21.15.67:0/3177557439 conn(0x7f498c0ac230 msgr2=0x7f498c0ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:58.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.411+0000 7f4991800640 1 -- 172.21.15.67:0/3177557439 shutdown_connections 2024-02-26T10:39:58.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.412+0000 7f4991800640 1 -- 172.21.15.67:0/3177557439 wait complete. 2024-02-26T10:39:58.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.413+0000 7f4991800640 1 Processor -- start 2024-02-26T10:39:58.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.413+0000 7f4991800640 1 -- start start 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f4991800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498c0c5290 0x7f498c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f4991800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 0x7f498c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f4991800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c45d0 0x7f498c1e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f4991800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0affb0 con 0x7f498c155880 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f4991800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c155cc0 con 0x7f498c0c5290 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f4991800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0b0130 con 0x7f498c0c45d0 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.414+0000 7f498b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 0x7f498c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 0x7f498c0c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54294/0 (socket says 172.21.15.67:54294) 2024-02-26T10:39:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 -- 172.21.15.67:0/639765438 learned_addr learned my addr 172.21.15.67:0/639765438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f4990e00640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c45d0 0x7f498c1e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498be00640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498c0c5290 0x7f498c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 -- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c45d0 msgr2=0x7f498c1e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c45d0 0x7f498c1e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 -- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498c0c5290 msgr2=0x7f498c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498c0c5290 0x7f498c1517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 -- 172.21.15.67:0/639765438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0c18f0 con 0x7f498c155880 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.415+0000 7f498b400640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 0x7f498c0c96b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f497400e9b0 tx=0x7f497400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.416+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974017070 con 0x7f498c155880 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.416+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400c9a0 con 0x7f498c155880 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.416+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400cd90 con 0x7f498c155880 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.416+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c0c1bd0 con 0x7f498c155880 2024-02-26T10:39:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.416+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c0c2190 con 0x7f498c155880 2024-02-26T10:39:58.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.417+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c074a30 con 0x7f498c155880 2024-02-26T10:39:58.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.418+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f497401a070 con 0x7f498c155880 2024-02-26T10:39:58.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.419+0000 7f4983e00640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f496806b4d0 0x7f496806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:58.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.419+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f497408a640 con 0x7f498c155880 2024-02-26T10:39:58.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.419+0000 7f498be00640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f496806b4d0 0x7f496806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:58.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.421+0000 7f498be00640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f496806b4d0 0x7f496806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f497c035000 tx=0x7f497c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.423+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49740590c0 con 0x7f498c155880 2024-02-26T10:39:58.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:58.655+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f498c0c1640 con 0x7f498c155880 2024-02-26T10:39:59.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.163+0000 7f4983e00640 1 -- 172.21.15.67:0/639765438 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f49740589c0 con 0x7f498c155880 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f496806b4d0 msgr2=0x7f496806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f496806b4d0 0x7f496806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f497c035000 tx=0x7f497c030000 comp rx=0 tx=0).stop 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 msgr2=0x7f498c0c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 0x7f498c0c96b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f497400e9b0 tx=0x7f497400ee70 comp rx=0 tx=0).stop 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 shutdown_connections 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498c0c45d0 0x7f498c1e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f498c155880 0x7f498c0c96b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f496806b4d0 0x7f496806d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 --2- 172.21.15.67:0/639765438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498c0c5290 0x7f498c1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 >> 172.21.15.67:0/639765438 conn(0x7f498c0ac230 msgr2=0x7f498c0ab460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:59.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.166+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 shutdown_connections 2024-02-26T10:39:59.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.167+0000 7f4991800640 1 -- 172.21.15.67:0/639765438 wait complete. 2024-02-26T10:39:59.187 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-02-26T10:39:59.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.411+0000 7fa64be00640 1 Processor -- start 2024-02-26T10:39:59.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.411+0000 7fa64be00640 1 -- start start 2024-02-26T10:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa64c153630 0x7fa64c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c155f10 con 0x7fa64c06ac20 2024-02-26T10:39:59.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c153a60 con 0x7fa64c0c2310 2024-02-26T10:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0c28d0 con 0x7fa64c153630 2024-02-26T10:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa64c06ac20 0x7fa64c152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52958/0 (socket says 172.21.15.67:52958) 2024-02-26T10:39:59.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64aa00640 1 -- 172.21.15.67:0/248890001 learned_addr learned my addr 172.21.15.67:0/248890001 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681322535 0 0) 0x7fa64c155f10 con 0x7fa64c06ac20 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.413+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c005730 con 0x7fa64c06ac20 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64b400640 1 --2- 172.21.15.67:0/248890001 >> v2:172.21.15.82:3300/0 conn(0x7fa64c153630 0x7fa64c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2513722007 0 0) 0x7fa62c005730 con 0x7fa64c06ac20 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 >> v2:172.21.15.82:3300/0 conn(0x7fa64c153630 msgr2=0x7fa64c06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 --2- 172.21.15.67:0/248890001 >> v2:172.21.15.82:3300/0 conn(0x7fa64c153630 0x7fa64c06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 >> v1:172.21.15.82:6789/0 conn(0x7fa64c0c2310 legacy=0x7fa64c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c155150 con 0x7fa64c06ac20 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fa63c001670 con 0x7fa64c06ac20 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 >> v1:172.21.15.67:6789/0 conn(0x7fa64c06ac20 legacy=0x7fa64c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa62c005c10 0x7fa62c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa62c006a10 0x7fa62c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c0075d0 0x7fa62c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c28d0 con 0x7fa62c006a10 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c00e4b0 con 0x7fa62c005c10 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c00e6b0 con 0x7fa62c0075d0 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.414+0000 7fa643e00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa62c006a10 0x7fa62c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa64aa00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa62c005c10 0x7fa62c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa64b400640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c0075d0 0x7fa62c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa643e00640 1 -- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c0075d0 msgr2=0x7fa62c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa643e00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c0075d0 0x7fa62c00dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa643e00640 1 -- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa62c005c10 msgr2=0x7fa62c006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa643e00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa62c005c10 0x7fa62c006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa643e00640 1 -- 172.21.15.67:0/248890001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c005730 con 0x7fa62c006a10 2024-02-26T10:39:59.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.415+0000 7fa64b400640 1 -- 172.21.15.67:0/248890001 reap_dead start 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.416+0000 7fa643e00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa62c006a10 0x7fa62c006e90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa634001750 tx=0x7fa62c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b029b13245621 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.416+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634001370 con 0x7fa62c006a10 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.416+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6340052d0 con 0x7fa62c006a10 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.416+0000 7fa64a000640 1 -- 172.21.15.67:0/248890001 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634003830 con 0x7fa62c006a10 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 -- 172.21.15.67:0/248890001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa62c006a10 msgr2=0x7fa62c006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa62c006a10 0x7fa62c006e90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa634001750 tx=0x7fa62c007a40 comp rx=0 tx=0).stop 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 -- 172.21.15.67:0/248890001 shutdown_connections 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 --2- 172.21.15.67:0/248890001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa62c006a10 0x7fa62c006e90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 -- 172.21.15.67:0/248890001 >> 172.21.15.67:0/248890001 conn(0x7fa64c0c63f0 msgr2=0x7fa64c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 -- 172.21.15.67:0/248890001 shutdown_connections 2024-02-26T10:39:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.417+0000 7fa64be00640 1 -- 172.21.15.67:0/248890001 wait complete. 2024-02-26T10:39:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.419+0000 7fa64be00640 1 Processor -- start 2024-02-26T10:39:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.419+0000 7fa64be00640 1 -- start start 2024-02-26T10:39:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c006a10 0x7fa64c0abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 0x7fa64c0aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa64c13f290 0x7fa64c1e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c1559f0 con 0x7fa64c0c5d30 2024-02-26T10:39:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c155710 con 0x7fa64c13f290 2024-02-26T10:39:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c148850 con 0x7fa62c006a10 2024-02-26T10:39:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 0x7fa64c0aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 0x7fa64c0aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54174/0 (socket says 172.21.15.67:54174) 2024-02-26T10:39:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 -- 172.21.15.67:0/3411161485 learned_addr learned my addr 172.21.15.67:0/3411161485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64b400640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa64c13f290 0x7fa64c1e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa64aa00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c006a10 0x7fa64c0abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 -- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c006a10 msgr2=0x7fa64c0abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c006a10 0x7fa64c0abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 -- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa64c13f290 msgr2=0x7fa64c1e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa64c13f290 0x7fa64c1e9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.420+0000 7fa643e00640 1 -- 172.21.15.67:0/3411161485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c142270 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.421+0000 7fa643e00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 0x7fa64c0aaa20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa634001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.421+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634041020 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.421+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634005480 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.421+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634001370 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.421+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c151c70 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.421+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c143ac0 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.422+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c074a30 con 0x7fa64c0c5d30 2024-02-26T10:39:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.423+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa634044020 con 0x7fa64c0c5d30 2024-02-26T10:39:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.425+0000 7fa642a00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa62806b4d0 0x7fa62806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.425+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa6340abf00 con 0x7fa64c0c5d30 2024-02-26T10:39:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.425+0000 7fa64aa00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa62806b4d0 0x7fa62806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.426+0000 7fa64aa00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa62806b4d0 0x7fa62806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001120 tx=0x7fa63c000f50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.429+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa63404b030 con 0x7fa64c0c5d30 2024-02-26T10:39:59.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.660+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa64c0c6120 con 0x7fa64c0c5d30 2024-02-26T10:39:59.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.661+0000 7fa642a00640 1 -- 172.21.15.67:0/3411161485 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+6171 (secure 0 0 0) 0x7fa6340303c0 con 0x7fa64c0c5d30 2024-02-26T10:39:59.661 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:39:59.661 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:39:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 22 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa62806b4d0 msgr2=0x7fa62806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa62806b4d0 0x7fa62806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa63c001120 tx=0x7fa63c000f50 comp rx=0 tx=0).stop 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 msgr2=0x7fa64c0aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 0x7fa64c0aaa20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa634000c00 tx=0x7fa634001120 comp rx=0 tx=0).stop 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 shutdown_connections 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa62806b4d0 0x7fa62806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa64c13f290 0x7fa64c1e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa64c0c5d30 0x7fa64c0aaa20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 --2- 172.21.15.67:0/3411161485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa62c006a10 0x7fa64c0abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 >> 172.21.15.67:0/3411161485 conn(0x7fa64c0c63f0 msgr2=0x7fa64c0c0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.663+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 shutdown_connections 2024-02-26T10:39:59.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.664+0000 7fa64be00640 1 -- 172.21.15.67:0/3411161485 wait complete. 2024-02-26T10:39:59.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:39:59.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.890+0000 7f0612600640 1 Processor -- start 2024-02-26T10:39:59.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.891+0000 7f0612600640 1 -- start start 2024-02-26T10:39:59.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.892+0000 7f0612600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f060c06ac20 0x7f060c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.892+0000 7f0612600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.892+0000 7f0612600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c06b460 con 0x7f060c06b000 2024-02-26T10:39:59.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.892+0000 7f0612600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c06dfb0 con 0x7f060c06ac20 2024-02-26T10:39:59.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.892+0000 7f0612600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0a9a80 con 0x7f060c06db00 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.892+0000 7f0611c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c0abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:35288/0 (socket says 172.21.15.67:35288) 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611c00640 1 -- 172.21.15.67:0/2571133724 learned_addr learned my addr 172.21.15.67:0/2571133724 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611200640 1 --2- 172.21.15.67:0/2571133724 >> v2:172.21.15.82:3300/0 conn(0x7f060c06ac20 0x7f060c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611c00640 1 -- 172.21.15.67:0/2571133724 >> v2:172.21.15.82:3300/0 conn(0x7f060c06ac20 msgr2=0x7f060c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611c00640 1 --2- 172.21.15.67:0/2571133724 >> v2:172.21.15.82:3300/0 conn(0x7f060c06ac20 0x7f060c0a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611c00640 1 -- 172.21.15.67:0/2571133724 >> v1:172.21.15.82:6789/0 conn(0x7f060c06b000 legacy=0x7f060c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.893+0000 7f0611c00640 1 -- 172.21.15.67:0/2571133724 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0a9d60 con 0x7f060c06db00 2024-02-26T10:39:59.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.895+0000 7f0611c00640 1 --2- 172.21.15.67:0/2571133724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c0abf50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0604009870 tx=0x7f060402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c34399040c9e53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.895+0000 7f060b400640 1 -- 172.21.15.67:0/2571133724 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604034070 con 0x7f060c06db00 2024-02-26T10:39:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.895+0000 7f060b400640 1 -- 172.21.15.67:0/2571133724 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604030840 con 0x7f060c06db00 2024-02-26T10:39:59.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.895+0000 7f060b400640 1 -- 172.21.15.67:0/2571133724 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604030c10 con 0x7f060c06db00 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 -- 172.21.15.67:0/2571133724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 msgr2=0x7f060c0abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 --2- 172.21.15.67:0/2571133724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c0abf50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0604009870 tx=0x7f060402b980 comp rx=0 tx=0).stop 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 -- 172.21.15.67:0/2571133724 shutdown_connections 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 --2- 172.21.15.67:0/2571133724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c0abf50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 --2- 172.21.15.67:0/2571133724 >> v2:172.21.15.82:3300/0 conn(0x7f060c06ac20 0x7f060c0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 -- 172.21.15.67:0/2571133724 >> 172.21.15.67:0/2571133724 conn(0x7f060c0c63f0 msgr2=0x7f060c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 -- 172.21.15.67:0/2571133724 shutdown_connections 2024-02-26T10:39:59.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.896+0000 7f0612600640 1 -- 172.21.15.67:0/2571133724 wait complete. 2024-02-26T10:39:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.898+0000 7f0612600640 1 Processor -- start 2024-02-26T10:39:59.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.898+0000 7f0612600640 1 -- start start 2024-02-26T10:39:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0612600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 0x7f060c1eb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0612600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c06b000 0x7f060c0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0612600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0612600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0ac490 con 0x7f060c06ac20 2024-02-26T10:39:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0612600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0aba30 con 0x7f060c06b000 2024-02-26T10:39:59.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0612600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c150410 con 0x7f060c06db00 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 0x7f060c1eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 0x7f060c1eb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54176/0 (socket says 172.21.15.67:54176) 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 -- 172.21.15.67:0/1423179662 learned_addr learned my addr 172.21.15.67:0/1423179662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611c00640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f060be00640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c06b000 0x7f060c0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 -- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 msgr2=0x7f060c152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 -- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c06b000 msgr2=0x7f060c0bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c06b000 0x7f060c0bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.899+0000 7f0611200640 1 -- 172.21.15.67:0/1423179662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c070cf0 con 0x7f060c06ac20 2024-02-26T10:39:59.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.900+0000 7f0611200640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 0x7f060c1eb350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009db0 tx=0x7f05fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:59.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.900+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc018020 con 0x7f060c06ac20 2024-02-26T10:39:59.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.900+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc014480 con 0x7f060c06ac20 2024-02-26T10:39:59.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.900+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc014850 con 0x7f060c06ac20 2024-02-26T10:39:59.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.900+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c072080 con 0x7f060c06ac20 2024-02-26T10:39:59.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.900+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c072640 con 0x7f060c06ac20 2024-02-26T10:39:59.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.901+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c06bea0 con 0x7f060c06ac20 2024-02-26T10:39:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.902+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05fc005d10 con 0x7f060c06ac20 2024-02-26T10:39:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.903+0000 7f0608c00640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05e806b500 0x7f05e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:39:59.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.903+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f05fc08b000 con 0x7f060c06ac20 2024-02-26T10:39:59.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.904+0000 7f060be00640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05e806b500 0x7f05e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:39:59.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.905+0000 7f060be00640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05e806b500 0x7f05e806d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f060c1571f0 tx=0x7f05f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:39:59.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:39:59.908+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f060c06bea0 con 0x7f060c06ac20 2024-02-26T10:40:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.098+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f060c074a30 con 0x7f060c06ac20 2024-02-26T10:40:00.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.102+0000 7f0608c00640 1 -- 172.21.15.67:0/1423179662 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+20323 (secure 0 0 0) 0x7f05fc022070 con 0x7f060c06ac20 2024-02-26T10:40:00.103 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:00.103 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:57.124601+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:00.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05e806b500 msgr2=0x7f05e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05e806b500 0x7f05e806d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f060c1571f0 tx=0x7f05f4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 msgr2=0x7f060c1eb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 0x7f060c1eb350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009db0 tx=0x7f05fc00c490 comp rx=0 tx=0).stop 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 shutdown_connections 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f060c06db00 0x7f060c152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f05e806b500 0x7f05e806d9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c06b000 0x7f060c0bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 --2- 172.21.15.67:0/1423179662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f060c06ac20 0x7f060c1eb350 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 >> 172.21.15.67:0/1423179662 conn(0x7f060c0c63f0 msgr2=0x7f060c14e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 shutdown_connections 2024-02-26T10:40:00.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.105+0000 7f0612600640 1 -- 172.21.15.67:0/1423179662 wait complete. 2024-02-26T10:40:00.124 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:40:00.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:40:00.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:40:00.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:40:00.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.347+0000 7f23ff200640 1 Processor -- start 2024-02-26T10:40:00.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.347+0000 7f23ff200640 1 -- start start 2024-02-26T10:40:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.348+0000 7f23ff200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f240006db00 0x7f24000abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.348+0000 7f23ff200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f24000ac230 0x7f24000716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23ff200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240006e0e0 con 0x7f24000c2710 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23ff200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2400070c50 con 0x7f24000ac230 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23ff200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240006e3c0 con 0x7f240006db00 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f24000ac230 0x7f24000716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f24000ac230 0x7f24000716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54186/0 (socket says 172.21.15.67:54186) 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 -- 172.21.15.67:0/365037062 learned_addr learned my addr 172.21.15.67:0/365037062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:00.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 -- 172.21.15.67:0/365037062 >> v2:172.21.15.82:3300/0 conn(0x7f240006db00 msgr2=0x7f24000abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 --2- 172.21.15.67:0/365037062 >> v2:172.21.15.82:3300/0 conn(0x7f240006db00 0x7f24000abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 -- 172.21.15.67:0/365037062 >> v1:172.21.15.82:6789/0 conn(0x7f24000c2710 legacy=0x7f2400151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.349+0000 7f23fe800640 1 -- 172.21.15.67:0/365037062 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400070ea0 con 0x7f24000ac230 2024-02-26T10:40:00.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.351+0000 7f23fe800640 1 --2- 172.21.15.67:0/365037062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f24000716a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f23f800d150 tx=0x7f23f802dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc8c0a4a29e37fa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.351+0000 7f23fca00640 1 -- 172.21.15.67:0/365037062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f802e970 con 0x7f24000ac230 2024-02-26T10:40:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.351+0000 7f23fca00640 1 -- 172.21.15.67:0/365037062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f802eb10 con 0x7f24000ac230 2024-02-26T10:40:00.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.351+0000 7f23fca00640 1 -- 172.21.15.67:0/365037062 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8009600 con 0x7f24000ac230 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 -- 172.21.15.67:0/365037062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 msgr2=0x7f24000716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 --2- 172.21.15.67:0/365037062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f24000716a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f23f800d150 tx=0x7f23f802dfd0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 -- 172.21.15.67:0/365037062 shutdown_connections 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 --2- 172.21.15.67:0/365037062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f24000716a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 --2- 172.21.15.67:0/365037062 >> v2:172.21.15.82:3300/0 conn(0x7f240006db00 0x7f24000abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 -- 172.21.15.67:0/365037062 >> 172.21.15.67:0/365037062 conn(0x7f24000c63f0 msgr2=0x7f24000c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 -- 172.21.15.67:0/365037062 shutdown_connections 2024-02-26T10:40:00.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.352+0000 7f23ff200640 1 -- 172.21.15.67:0/365037062 wait complete. 2024-02-26T10:40:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.354+0000 7f23ff200640 1 Processor -- start 2024-02-26T10:40:00.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.354+0000 7f23ff200640 1 -- start start 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23ff200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f240006db00 0x7f24000c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23ff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f2400148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23ff200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24000c2710 0x7f2400149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23ff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240006ac20 con 0x7f24000ac230 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23ff200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24000c2af0 con 0x7f24000c2710 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23ff200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240006ada0 con 0x7f240006db00 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23fd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f2400148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23fd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f2400148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54202/0 (socket says 172.21.15.67:54202) 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23fd400640 1 -- 172.21.15.67:0/2292678689 learned_addr learned my addr 172.21.15.67:0/2292678689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23fe800640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24000c2710 0x7f2400149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fd400640 1 -- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f240006db00 msgr2=0x7f24000c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.355+0000 7f23fde00640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f240006db00 0x7f24000c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fd400640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f240006db00 0x7f24000c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fd400640 1 -- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24000c2710 msgr2=0x7f2400149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fd400640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24000c2710 0x7f2400149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fd400640 1 -- 172.21.15.67:0/2292678689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24001ea870 con 0x7f24000ac230 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fe800640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24000c2710 0x7f2400149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23fd400640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f2400148b50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00b410 tx=0x7f23ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec015020 con 0x7f24000ac230 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec011850 con 0x7f24000ac230 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec011c40 con 0x7f24000ac230 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24001eab50 con 0x7f24000ac230 2024-02-26T10:40:00.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.356+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24001eb110 con 0x7f24000ac230 2024-02-26T10:40:00.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.358+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24000c3690 con 0x7f24000ac230 2024-02-26T10:40:00.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.358+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23ec018020 con 0x7f24000ac230 2024-02-26T10:40:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.359+0000 7f23f6000640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23e006b520 0x7f23e006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.360+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f23ec08a460 con 0x7f24000ac230 2024-02-26T10:40:00.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.360+0000 7f23fde00640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23e006b520 0x7f23e006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.361+0000 7f23fde00640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23e006b520 0x7f23e006d9e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f00015d0 tx=0x7f23f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:00.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.365+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23ec01f020 con 0x7f24000ac230 2024-02-26T10:40:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.606+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2400074a30 con 0x7f23e006b520 2024-02-26T10:40:00.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.647+0000 7f23f6000640 1 -- 172.21.15.67:0/2292678689 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490424 (secure 0 0 0) 0x7f2400074a30 con 0x7f23e006b520 2024-02-26T10:40:00.654 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:00.657 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:40:00.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23e006b520 msgr2=0x7f23e006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23e006b520 0x7f23e006d9e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f00015d0 tx=0x7f23f0009040 comp rx=0 tx=0).stop 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 msgr2=0x7f2400148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f2400148b50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00b410 tx=0x7f23ec00b8d0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 shutdown_connections 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23e006b520 0x7f23e006d9e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24000c2710 0x7f2400149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000ac230 0x7f2400148b50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 --2- 172.21.15.67:0/2292678689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f240006db00 0x7f24000c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 >> 172.21.15.67:0/2292678689 conn(0x7f24000c63f0 msgr2=0x7f24000c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 shutdown_connections 2024-02-26T10:40:00.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.655+0000 7f23ff200640 1 -- 172.21.15.67:0/2292678689 wait complete. 2024-02-26T10:40:00.674 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-02-26T10:40:00.352902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":345,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220836,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753884,"statfs":{"total":773094113280,"available":772867977216,"internally_reserved":0,"allocated":4759552,"data_stored":2621334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002061"},"pg_stats":[{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250638+0000","last_change":"2024-02-26T10:39:56.155884+0000","last_active":"2024-02-26T10:39:57.250638+0000","last_peered":"2024-02-26T10:39:57.250638+0000","last_clean":"2024-02-26T10:39:57.250638+0000","last_became_active":"2024-02-26T10:39:56.155539+0000","last_became_peered":"2024-02-26T10:39:56.155539+0000","last_unstale":"2024-02-26T10:39:57.250638+0000","last_undegraded":"2024-02-26T10:39:57.250638+0000","last_fullsized":"2024-02-26T10:39:57.250638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:53.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256791+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:39:57.256791+0000","last_peered":"2024-02-26T10:39:57.256791+0000","last_clean":"2024-02-26T10:39:57.256791+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:39:57.256791+0000","last_undegraded":"2024-02-26T10:39:57.256791+0000","last_fullsized":"2024-02-26T10:39:57.256791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259164+0000","last_change":"2024-02-26T10:39:39.013277+0000","last_active":"2024-02-26T10:39:57.259164+0000","last_peered":"2024-02-26T10:39:57.259164+0000","last_clean":"2024-02-26T10:39:57.259164+0000","last_became_active":"2024-02-26T10:39:39.012073+0000","last_became_peered":"2024-02-26T10:39:39.012073+0000","last_unstale":"2024-02-26T10:39:57.259164+0000","last_undegraded":"2024-02-26T10:39:57.259164+0000","last_fullsized":"2024-02-26T10:39:57.259164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:44.421668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258900+0000","last_change":"2024-02-26T10:39:40.018706+0000","last_active":"2024-02-26T10:39:57.258900+0000","last_peered":"2024-02-26T10:39:57.258900+0000","last_clean":"2024-02-26T10:39:57.258900+0000","last_became_active":"2024-02-26T10:39:40.018417+0000","last_became_peered":"2024-02-26T10:39:40.018417+0000","last_unstale":"2024-02-26T10:39:57.258900+0000","last_undegraded":"2024-02-26T10:39:57.258900+0000","last_fullsized":"2024-02-26T10:39:57.258900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257645+0000","last_change":"2024-02-26T10:39:55.139751+0000","last_active":"2024-02-26T10:39:57.257645+0000","last_peered":"2024-02-26T10:39:57.257645+0000","last_clean":"2024-02-26T10:39:57.257645+0000","last_became_active":"2024-02-26T10:39:55.139085+0000","last_became_peered":"2024-02-26T10:39:55.139085+0000","last_unstale":"2024-02-26T10:39:57.257645+0000","last_undegraded":"2024-02-26T10:39:57.257645+0000","last_fullsized":"2024-02-26T10:39:57.257645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:26:00.601349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139003+0000","last_change":"2024-02-26T10:39:56.133408+0000","last_active":"2024-02-26T10:39:57.139003+0000","last_peered":"2024-02-26T10:39:57.139003+0000","last_clean":"2024-02-26T10:39:57.139003+0000","last_became_active":"2024-02-26T10:39:56.132956+0000","last_became_peered":"2024-02-26T10:39:56.132956+0000","last_unstale":"2024-02-26T10:39:57.139003+0000","last_undegraded":"2024-02-26T10:39:57.139003+0000","last_fullsized":"2024-02-26T10:39:57.139003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:29.273971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138744+0000","last_change":"2024-02-26T10:39:39.006896+0000","last_active":"2024-02-26T10:39:57.138744+0000","last_peered":"2024-02-26T10:39:57.138744+0000","last_clean":"2024-02-26T10:39:57.138744+0000","last_became_active":"2024-02-26T10:39:39.006604+0000","last_became_peered":"2024-02-26T10:39:39.006604+0000","last_unstale":"2024-02-26T10:39:57.138744+0000","last_undegraded":"2024-02-26T10:39:57.138744+0000","last_fullsized":"2024-02-26T10:39:57.138744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143093+0000","last_change":"2024-02-26T10:39:39.995364+0000","last_active":"2024-02-26T10:39:57.143093+0000","last_peered":"2024-02-26T10:39:57.143093+0000","last_clean":"2024-02-26T10:39:57.143093+0000","last_became_active":"2024-02-26T10:39:39.995077+0000","last_became_peered":"2024-02-26T10:39:39.995077+0000","last_unstale":"2024-02-26T10:39:57.143093+0000","last_undegraded":"2024-02-26T10:39:57.143093+0000","last_fullsized":"2024-02-26T10:39:57.143093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144237+0000","last_change":"2024-02-26T10:39:55.138429+0000","last_active":"2024-02-26T10:39:57.144237+0000","last_peered":"2024-02-26T10:39:57.144237+0000","last_clean":"2024-02-26T10:39:57.144237+0000","last_became_active":"2024-02-26T10:39:55.138123+0000","last_became_peered":"2024-02-26T10:39:55.138123+0000","last_unstale":"2024-02-26T10:39:57.144237+0000","last_undegraded":"2024-02-26T10:39:57.144237+0000","last_fullsized":"2024-02-26T10:39:57.144237+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:22.197650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139321+0000","last_change":"2024-02-26T10:39:56.131831+0000","last_active":"2024-02-26T10:39:57.139321+0000","last_peered":"2024-02-26T10:39:57.139321+0000","last_clean":"2024-02-26T10:39:57.139321+0000","last_became_active":"2024-02-26T10:39:56.131566+0000","last_became_peered":"2024-02-26T10:39:56.131566+0000","last_unstale":"2024-02-26T10:39:57.139321+0000","last_undegraded":"2024-02-26T10:39:57.139321+0000","last_fullsized":"2024-02-26T10:39:57.139321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:44:13.749545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143650+0000","last_change":"2024-02-26T10:39:38.999680+0000","last_active":"2024-02-26T10:39:57.143650+0000","last_peered":"2024-02-26T10:39:57.143650+0000","last_clean":"2024-02-26T10:39:57.143650+0000","last_became_active":"2024-02-26T10:39:38.999058+0000","last_became_peered":"2024-02-26T10:39:38.999058+0000","last_unstale":"2024-02-26T10:39:57.143650+0000","last_undegraded":"2024-02-26T10:39:57.143650+0000","last_fullsized":"2024-02-26T10:39:57.143650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:26.840527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138626+0000","last_change":"2024-02-26T10:39:39.997501+0000","last_active":"2024-02-26T10:39:57.138626+0000","last_peered":"2024-02-26T10:39:57.138626+0000","last_clean":"2024-02-26T10:39:57.138626+0000","last_became_active":"2024-02-26T10:39:39.996084+0000","last_became_peered":"2024-02-26T10:39:39.996084+0000","last_unstale":"2024-02-26T10:39:57.138626+0000","last_undegraded":"2024-02-26T10:39:57.138626+0000","last_fullsized":"2024-02-26T10:39:57.138626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257819+0000","last_change":"2024-02-26T10:39:55.147988+0000","last_active":"2024-02-26T10:39:57.257819+0000","last_peered":"2024-02-26T10:39:57.257819+0000","last_clean":"2024-02-26T10:39:57.257819+0000","last_became_active":"2024-02-26T10:39:55.147451+0000","last_became_peered":"2024-02-26T10:39:55.147451+0000","last_unstale":"2024-02-26T10:39:57.257819+0000","last_undegraded":"2024-02-26T10:39:57.257819+0000","last_fullsized":"2024-02-26T10:39:57.257819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:00.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143227+0000","last_change":"2024-02-26T10:39:56.140810+0000","last_active":"2024-02-26T10:39:57.143227+0000","last_peered":"2024-02-26T10:39:57.143227+0000","last_clean":"2024-02-26T10:39:57.143227+0000","last_became_active":"2024-02-26T10:39:56.140541+0000","last_became_peered":"2024-02-26T10:39:56.140541+0000","last_unstale":"2024-02-26T10:39:57.143227+0000","last_undegraded":"2024-02-26T10:39:57.143227+0000","last_fullsized":"2024-02-26T10:39:57.143227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:37.401643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250803+0000","last_change":"2024-02-26T10:39:39.013362+0000","last_active":"2024-02-26T10:39:57.250803+0000","last_peered":"2024-02-26T10:39:57.250803+0000","last_clean":"2024-02-26T10:39:57.250803+0000","last_became_active":"2024-02-26T10:39:39.012819+0000","last_became_peered":"2024-02-26T10:39:39.012819+0000","last_unstale":"2024-02-26T10:39:57.250803+0000","last_undegraded":"2024-02-26T10:39:57.250803+0000","last_fullsized":"2024-02-26T10:39:57.250803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249694+0000","last_change":"2024-02-26T10:39:40.011820+0000","last_active":"2024-02-26T10:39:57.249694+0000","last_peered":"2024-02-26T10:39:57.249694+0000","last_clean":"2024-02-26T10:39:57.249694+0000","last_became_active":"2024-02-26T10:39:40.011511+0000","last_became_peered":"2024-02-26T10:39:40.011511+0000","last_unstale":"2024-02-26T10:39:57.249694+0000","last_undegraded":"2024-02-26T10:39:57.249694+0000","last_fullsized":"2024-02-26T10:39:57.249694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249257+0000","last_change":"2024-02-26T10:39:55.137179+0000","last_active":"2024-02-26T10:39:57.249257+0000","last_peered":"2024-02-26T10:39:57.249257+0000","last_clean":"2024-02-26T10:39:57.249257+0000","last_became_active":"2024-02-26T10:39:55.136803+0000","last_became_peered":"2024-02-26T10:39:55.136803+0000","last_unstale":"2024-02-26T10:39:57.249257+0000","last_undegraded":"2024-02-26T10:39:57.249257+0000","last_fullsized":"2024-02-26T10:39:57.249257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:36.808744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252495+0000","last_change":"2024-02-26T10:39:56.146625+0000","last_active":"2024-02-26T10:39:57.252495+0000","last_peered":"2024-02-26T10:39:57.252495+0000","last_clean":"2024-02-26T10:39:57.252495+0000","last_became_active":"2024-02-26T10:39:56.145895+0000","last_became_peered":"2024-02-26T10:39:56.145895+0000","last_unstale":"2024-02-26T10:39:57.252495+0000","last_undegraded":"2024-02-26T10:39:57.252495+0000","last_fullsized":"2024-02-26T10:39:57.252495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:06.696664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250295+0000","last_change":"2024-02-26T10:39:39.003861+0000","last_active":"2024-02-26T10:39:57.250295+0000","last_peered":"2024-02-26T10:39:57.250295+0000","last_clean":"2024-02-26T10:39:57.250295+0000","last_became_active":"2024-02-26T10:39:39.003370+0000","last_became_peered":"2024-02-26T10:39:39.003370+0000","last_unstale":"2024-02-26T10:39:57.250295+0000","last_undegraded":"2024-02-26T10:39:57.250295+0000","last_fullsized":"2024-02-26T10:39:57.250295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259356+0000","last_change":"2024-02-26T10:39:40.008904+0000","last_active":"2024-02-26T10:39:57.259356+0000","last_peered":"2024-02-26T10:39:57.259356+0000","last_clean":"2024-02-26T10:39:57.259356+0000","last_became_active":"2024-02-26T10:39:40.008631+0000","last_became_peered":"2024-02-26T10:39:40.008631+0000","last_unstale":"2024-02-26T10:39:57.259356+0000","last_undegraded":"2024-02-26T10:39:57.259356+0000","last_fullsized":"2024-02-26T10:39:57.259356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250493+0000","last_change":"2024-02-26T10:39:55.147824+0000","last_active":"2024-02-26T10:39:57.250493+0000","last_peered":"2024-02-26T10:39:57.250493+0000","last_clean":"2024-02-26T10:39:57.250493+0000","last_became_active":"2024-02-26T10:39:55.147535+0000","last_became_peered":"2024-02-26T10:39:55.147535+0000","last_unstale":"2024-02-26T10:39:57.250493+0000","last_undegraded":"2024-02-26T10:39:57.250493+0000","last_fullsized":"2024-02-26T10:39:57.250493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:36.386825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259593+0000","last_change":"2024-02-26T10:39:56.139551+0000","last_active":"2024-02-26T10:39:57.259593+0000","last_peered":"2024-02-26T10:39:57.259593+0000","last_clean":"2024-02-26T10:39:57.259593+0000","last_became_active":"2024-02-26T10:39:56.139259+0000","last_became_peered":"2024-02-26T10:39:56.139259+0000","last_unstale":"2024-02-26T10:39:57.259593+0000","last_undegraded":"2024-02-26T10:39:57.259593+0000","last_fullsized":"2024-02-26T10:39:57.259593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:51:53.159078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251111+0000","last_change":"2024-02-26T10:39:38.996371+0000","last_active":"2024-02-26T10:39:57.251111+0000","last_peered":"2024-02-26T10:39:57.251111+0000","last_clean":"2024-02-26T10:39:57.251111+0000","last_became_active":"2024-02-26T10:39:38.996052+0000","last_became_peered":"2024-02-26T10:39:38.996052+0000","last_unstale":"2024-02-26T10:39:57.251111+0000","last_undegraded":"2024-02-26T10:39:57.251111+0000","last_fullsized":"2024-02-26T10:39:57.251111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249995+0000","last_change":"2024-02-26T10:39:40.009958+0000","last_active":"2024-02-26T10:39:57.249995+0000","last_peered":"2024-02-26T10:39:57.249995+0000","last_clean":"2024-02-26T10:39:57.249995+0000","last_became_active":"2024-02-26T10:39:40.009557+0000","last_became_peered":"2024-02-26T10:39:40.009557+0000","last_unstale":"2024-02-26T10:39:57.249995+0000","last_undegraded":"2024-02-26T10:39:57.249995+0000","last_fullsized":"2024-02-26T10:39:57.249995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:58.124347+0000","last_change":"2024-02-26T10:39:55.143200+0000","last_active":"2024-02-26T10:39:58.124347+0000","last_peered":"2024-02-26T10:39:58.124347+0000","last_clean":"2024-02-26T10:39:58.124347+0000","last_became_active":"2024-02-26T10:39:55.142844+0000","last_became_peered":"2024-02-26T10:39:55.142844+0000","last_unstale":"2024-02-26T10:39:58.124347+0000","last_undegraded":"2024-02-26T10:39:58.124347+0000","last_fullsized":"2024-02-26T10:39:58.124347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:50.940528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257616+0000","last_change":"2024-02-26T10:39:56.148312+0000","last_active":"2024-02-26T10:39:57.257616+0000","last_peered":"2024-02-26T10:39:57.257616+0000","last_clean":"2024-02-26T10:39:57.257616+0000","last_became_active":"2024-02-26T10:39:56.148002+0000","last_became_peered":"2024-02-26T10:39:56.148002+0000","last_unstale":"2024-02-26T10:39:57.257616+0000","last_undegraded":"2024-02-26T10:39:57.257616+0000","last_fullsized":"2024-02-26T10:39:57.257616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:29.247494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250768+0000","last_change":"2024-02-26T10:39:38.992861+0000","last_active":"2024-02-26T10:39:57.250768+0000","last_peered":"2024-02-26T10:39:57.250768+0000","last_clean":"2024-02-26T10:39:57.250768+0000","last_became_active":"2024-02-26T10:39:38.992258+0000","last_became_peered":"2024-02-26T10:39:38.992258+0000","last_unstale":"2024-02-26T10:39:57.250768+0000","last_undegraded":"2024-02-26T10:39:57.250768+0000","last_fullsized":"2024-02-26T10:39:57.250768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254948+0000","last_change":"2024-02-26T10:39:40.009603+0000","last_active":"2024-02-26T10:39:57.254948+0000","last_peered":"2024-02-26T10:39:57.254948+0000","last_clean":"2024-02-26T10:39:57.254948+0000","last_became_active":"2024-02-26T10:39:40.009327+0000","last_became_peered":"2024-02-26T10:39:40.009327+0000","last_unstale":"2024-02-26T10:39:57.254948+0000","last_undegraded":"2024-02-26T10:39:57.254948+0000","last_fullsized":"2024-02-26T10:39:57.254948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250518+0000","last_change":"2024-02-26T10:39:55.125334+0000","last_active":"2024-02-26T10:39:57.250518+0000","last_peered":"2024-02-26T10:39:57.250518+0000","last_clean":"2024-02-26T10:39:57.250518+0000","last_became_active":"2024-02-26T10:39:55.125040+0000","last_became_peered":"2024-02-26T10:39:55.125040+0000","last_unstale":"2024-02-26T10:39:57.250518+0000","last_undegraded":"2024-02-26T10:39:57.250518+0000","last_fullsized":"2024-02-26T10:39:57.250518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:17.565930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249624+0000","last_change":"2024-02-26T10:39:56.151375+0000","last_active":"2024-02-26T10:39:57.249624+0000","last_peered":"2024-02-26T10:39:57.249624+0000","last_clean":"2024-02-26T10:39:57.249624+0000","last_became_active":"2024-02-26T10:39:56.151080+0000","last_became_peered":"2024-02-26T10:39:56.151080+0000","last_unstale":"2024-02-26T10:39:57.249624+0000","last_undegraded":"2024-02-26T10:39:57.249624+0000","last_fullsized":"2024-02-26T10:39:57.249624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:10.708753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251574+0000","last_change":"2024-02-26T10:39:39.009948+0000","last_active":"2024-02-26T10:39:57.251574+0000","last_peered":"2024-02-26T10:39:57.251574+0000","last_clean":"2024-02-26T10:39:57.251574+0000","last_became_active":"2024-02-26T10:39:39.009640+0000","last_became_peered":"2024-02-26T10:39:39.009640+0000","last_unstale":"2024-02-26T10:39:57.251574+0000","last_undegraded":"2024-02-26T10:39:57.251574+0000","last_fullsized":"2024-02-26T10:39:57.251574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:47.192375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140884+0000","last_change":"2024-02-26T10:39:40.022558+0000","last_active":"2024-02-26T10:39:57.140884+0000","last_peered":"2024-02-26T10:39:57.140884+0000","last_clean":"2024-02-26T10:39:57.140884+0000","last_became_active":"2024-02-26T10:39:40.022316+0000","last_became_peered":"2024-02-26T10:39:40.022316+0000","last_unstale":"2024-02-26T10:39:57.140884+0000","last_undegraded":"2024-02-26T10:39:57.140884+0000","last_fullsized":"2024-02-26T10:39:57.140884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:11:06.614146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258080+0000","last_change":"2024-02-26T10:39:55.148037+0000","last_active":"2024-02-26T10:39:57.258080+0000","last_peered":"2024-02-26T10:39:57.258080+0000","last_clean":"2024-02-26T10:39:57.258080+0000","last_became_active":"2024-02-26T10:39:55.147501+0000","last_became_peered":"2024-02-26T10:39:55.147501+0000","last_unstale":"2024-02-26T10:39:57.258080+0000","last_undegraded":"2024-02-26T10:39:57.258080+0000","last_fullsized":"2024-02-26T10:39:57.258080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:51.837369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250733+0000","last_change":"2024-02-26T10:39:56.152002+0000","last_active":"2024-02-26T10:39:57.250733+0000","last_peered":"2024-02-26T10:39:57.250733+0000","last_clean":"2024-02-26T10:39:57.250733+0000","last_became_active":"2024-02-26T10:39:56.151739+0000","last_became_peered":"2024-02-26T10:39:56.151739+0000","last_unstale":"2024-02-26T10:39:57.250733+0000","last_undegraded":"2024-02-26T10:39:57.250733+0000","last_fullsized":"2024-02-26T10:39:57.250733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:15.643339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252645+0000","last_change":"2024-02-26T10:39:39.009824+0000","last_active":"2024-02-26T10:39:57.252645+0000","last_peered":"2024-02-26T10:39:57.252645+0000","last_clean":"2024-02-26T10:39:57.252645+0000","last_became_active":"2024-02-26T10:39:39.009559+0000","last_became_peered":"2024-02-26T10:39:39.009559+0000","last_unstale":"2024-02-26T10:39:57.252645+0000","last_undegraded":"2024-02-26T10:39:57.252645+0000","last_fullsized":"2024-02-26T10:39:57.252645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256931+0000","last_change":"2024-02-26T10:39:40.013051+0000","last_active":"2024-02-26T10:39:57.256931+0000","last_peered":"2024-02-26T10:39:57.256931+0000","last_clean":"2024-02-26T10:39:57.256931+0000","last_became_active":"2024-02-26T10:39:40.012739+0000","last_became_peered":"2024-02-26T10:39:40.012739+0000","last_unstale":"2024-02-26T10:39:57.256931+0000","last_undegraded":"2024-02-26T10:39:57.256931+0000","last_fullsized":"2024-02-26T10:39:57.256931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254892+0000","last_change":"2024-02-26T10:39:55.144555+0000","last_active":"2024-02-26T10:39:57.254892+0000","last_peered":"2024-02-26T10:39:57.254892+0000","last_clean":"2024-02-26T10:39:57.254892+0000","last_became_active":"2024-02-26T10:39:55.143431+0000","last_became_peered":"2024-02-26T10:39:55.143431+0000","last_unstale":"2024-02-26T10:39:57.254892+0000","last_undegraded":"2024-02-26T10:39:57.254892+0000","last_fullsized":"2024-02-26T10:39:57.254892+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:00.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139121+0000","last_change":"2024-02-26T10:39:56.133927+0000","last_active":"2024-02-26T10:39:57.139121+0000","last_peered":"2024-02-26T10:39:57.139121+0000","last_clean":"2024-02-26T10:39:57.139121+0000","last_became_active":"2024-02-26T10:39:56.133652+0000","last_became_peered":"2024-02-26T10:39:56.133652+0000","last_unstale":"2024-02-26T10:39:57.139121+0000","last_undegraded":"2024-02-26T10:39:57.139121+0000","last_fullsized":"2024-02-26T10:39:57.139121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:39.037942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143361+0000","last_change":"2024-02-26T10:39:39.000283+0000","last_active":"2024-02-26T10:39:57.143361+0000","last_peered":"2024-02-26T10:39:57.143361+0000","last_clean":"2024-02-26T10:39:57.143361+0000","last_became_active":"2024-02-26T10:39:38.999392+0000","last_became_peered":"2024-02-26T10:39:38.999392+0000","last_unstale":"2024-02-26T10:39:57.143361+0000","last_undegraded":"2024-02-26T10:39:57.143361+0000","last_fullsized":"2024-02-26T10:39:57.143361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:42.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140120+0000","last_change":"2024-02-26T10:39:40.001953+0000","last_active":"2024-02-26T10:39:57.140120+0000","last_peered":"2024-02-26T10:39:57.140120+0000","last_clean":"2024-02-26T10:39:57.140120+0000","last_became_active":"2024-02-26T10:39:40.001670+0000","last_became_peered":"2024-02-26T10:39:40.001670+0000","last_unstale":"2024-02-26T10:39:57.140120+0000","last_undegraded":"2024-02-26T10:39:57.140120+0000","last_fullsized":"2024-02-26T10:39:57.140120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:07.672401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139446+0000","last_change":"2024-02-26T10:39:55.147668+0000","last_active":"2024-02-26T10:39:57.139446+0000","last_peered":"2024-02-26T10:39:57.139446+0000","last_clean":"2024-02-26T10:39:57.139446+0000","last_became_active":"2024-02-26T10:39:55.147402+0000","last_became_peered":"2024-02-26T10:39:55.147402+0000","last_unstale":"2024-02-26T10:39:57.139446+0000","last_undegraded":"2024-02-26T10:39:57.139446+0000","last_fullsized":"2024-02-26T10:39:57.139446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:05.681846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249175+0000","last_change":"2024-02-26T10:39:56.151212+0000","last_active":"2024-02-26T10:39:57.249175+0000","last_peered":"2024-02-26T10:39:57.249175+0000","last_clean":"2024-02-26T10:39:57.249175+0000","last_became_active":"2024-02-26T10:39:56.150916+0000","last_became_peered":"2024-02-26T10:39:56.150916+0000","last_unstale":"2024-02-26T10:39:57.249175+0000","last_undegraded":"2024-02-26T10:39:57.249175+0000","last_fullsized":"2024-02-26T10:39:57.249175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:26:01.616167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259715+0000","last_change":"2024-02-26T10:39:39.011100+0000","last_active":"2024-02-26T10:39:57.259715+0000","last_peered":"2024-02-26T10:39:57.259715+0000","last_clean":"2024-02-26T10:39:57.259715+0000","last_became_active":"2024-02-26T10:39:39.010704+0000","last_became_peered":"2024-02-26T10:39:39.010704+0000","last_unstale":"2024-02-26T10:39:57.259715+0000","last_undegraded":"2024-02-26T10:39:57.259715+0000","last_fullsized":"2024-02-26T10:39:57.259715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142379+0000","last_change":"2024-02-26T10:39:40.002307+0000","last_active":"2024-02-26T10:39:57.142379+0000","last_peered":"2024-02-26T10:39:57.142379+0000","last_clean":"2024-02-26T10:39:57.142379+0000","last_became_active":"2024-02-26T10:39:40.001899+0000","last_became_peered":"2024-02-26T10:39:40.001899+0000","last_unstale":"2024-02-26T10:39:57.142379+0000","last_undegraded":"2024-02-26T10:39:57.142379+0000","last_fullsized":"2024-02-26T10:39:57.142379+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:23.314983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256394+0000","last_change":"2024-02-26T10:39:55.147262+0000","last_active":"2024-02-26T10:39:57.256394+0000","last_peered":"2024-02-26T10:39:57.256394+0000","last_clean":"2024-02-26T10:39:57.256394+0000","last_became_active":"2024-02-26T10:39:55.146952+0000","last_became_peered":"2024-02-26T10:39:55.146952+0000","last_unstale":"2024-02-26T10:39:57.256394+0000","last_undegraded":"2024-02-26T10:39:57.256394+0000","last_fullsized":"2024-02-26T10:39:57.256394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:49.472864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139544+0000","last_change":"2024-02-26T10:39:56.143318+0000","last_active":"2024-02-26T10:39:57.139544+0000","last_peered":"2024-02-26T10:39:57.139544+0000","last_clean":"2024-02-26T10:39:57.139544+0000","last_became_active":"2024-02-26T10:39:56.142848+0000","last_became_peered":"2024-02-26T10:39:56.142848+0000","last_unstale":"2024-02-26T10:39:57.139544+0000","last_undegraded":"2024-02-26T10:39:57.139544+0000","last_fullsized":"2024-02-26T10:39:57.139544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:44.657930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249952+0000","last_change":"2024-02-26T10:39:38.992320+0000","last_active":"2024-02-26T10:39:57.249952+0000","last_peered":"2024-02-26T10:39:57.249952+0000","last_clean":"2024-02-26T10:39:57.249952+0000","last_became_active":"2024-02-26T10:39:38.991970+0000","last_became_peered":"2024-02-26T10:39:38.991970+0000","last_unstale":"2024-02-26T10:39:57.249952+0000","last_undegraded":"2024-02-26T10:39:57.249952+0000","last_fullsized":"2024-02-26T10:39:57.249952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140449+0000","last_change":"2024-02-26T10:39:39.997097+0000","last_active":"2024-02-26T10:39:57.140449+0000","last_peered":"2024-02-26T10:39:57.140449+0000","last_clean":"2024-02-26T10:39:57.140449+0000","last_became_active":"2024-02-26T10:39:39.996720+0000","last_became_peered":"2024-02-26T10:39:39.996720+0000","last_unstale":"2024-02-26T10:39:57.140449+0000","last_undegraded":"2024-02-26T10:39:57.140449+0000","last_fullsized":"2024-02-26T10:39:57.140449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:12.318898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249123+0000","last_change":"2024-02-26T10:39:55.145833+0000","last_active":"2024-02-26T10:39:57.249123+0000","last_peered":"2024-02-26T10:39:57.249123+0000","last_clean":"2024-02-26T10:39:57.249123+0000","last_became_active":"2024-02-26T10:39:55.145534+0000","last_became_peered":"2024-02-26T10:39:55.145534+0000","last_unstale":"2024-02-26T10:39:57.249123+0000","last_undegraded":"2024-02-26T10:39:57.249123+0000","last_fullsized":"2024-02-26T10:39:57.249123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:11.096461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250250+0000","last_change":"2024-02-26T10:39:56.138607+0000","last_active":"2024-02-26T10:39:57.250250+0000","last_peered":"2024-02-26T10:39:57.250250+0000","last_clean":"2024-02-26T10:39:57.250250+0000","last_became_active":"2024-02-26T10:39:56.138370+0000","last_became_peered":"2024-02-26T10:39:56.138370+0000","last_unstale":"2024-02-26T10:39:57.250250+0000","last_undegraded":"2024-02-26T10:39:57.250250+0000","last_fullsized":"2024-02-26T10:39:57.250250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:46.512272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144723+0000","last_change":"2024-02-26T10:39:39.005544+0000","last_active":"2024-02-26T10:39:57.144723+0000","last_peered":"2024-02-26T10:39:57.144723+0000","last_clean":"2024-02-26T10:39:57.144723+0000","last_became_active":"2024-02-26T10:39:39.005194+0000","last_became_peered":"2024-02-26T10:39:39.005194+0000","last_unstale":"2024-02-26T10:39:57.144723+0000","last_undegraded":"2024-02-26T10:39:57.144723+0000","last_fullsized":"2024-02-26T10:39:57.144723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:51:47.714809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140986+0000","last_change":"2024-02-26T10:39:40.014954+0000","last_active":"2024-02-26T10:39:57.140986+0000","last_peered":"2024-02-26T10:39:57.140986+0000","last_clean":"2024-02-26T10:39:57.140986+0000","last_became_active":"2024-02-26T10:39:40.014657+0000","last_became_peered":"2024-02-26T10:39:40.014657+0000","last_unstale":"2024-02-26T10:39:57.140986+0000","last_undegraded":"2024-02-26T10:39:57.140986+0000","last_fullsized":"2024-02-26T10:39:57.140986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:38:13.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144920+0000","last_change":"2024-02-26T10:39:55.137487+0000","last_active":"2024-02-26T10:39:57.144920+0000","last_peered":"2024-02-26T10:39:57.144920+0000","last_clean":"2024-02-26T10:39:57.144920+0000","last_became_active":"2024-02-26T10:39:55.135747+0000","last_became_peered":"2024-02-26T10:39:55.135747+0000","last_unstale":"2024-02-26T10:39:57.144920+0000","last_undegraded":"2024-02-26T10:39:57.144920+0000","last_fullsized":"2024-02-26T10:39:57.144920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:41.302220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140800+0000","last_change":"2024-02-26T10:39:56.147247+0000","last_active":"2024-02-26T10:39:57.140800+0000","last_peered":"2024-02-26T10:39:57.140800+0000","last_clean":"2024-02-26T10:39:57.140800+0000","last_became_active":"2024-02-26T10:39:56.146951+0000","last_became_peered":"2024-02-26T10:39:56.146951+0000","last_unstale":"2024-02-26T10:39:57.140800+0000","last_undegraded":"2024-02-26T10:39:57.140800+0000","last_fullsized":"2024-02-26T10:39:57.140800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:40:06.774429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145157+0000","last_change":"2024-02-26T10:39:38.997283+0000","last_active":"2024-02-26T10:39:57.145157+0000","last_peered":"2024-02-26T10:39:57.145157+0000","last_clean":"2024-02-26T10:39:57.145157+0000","last_became_active":"2024-02-26T10:39:38.996974+0000","last_became_peered":"2024-02-26T10:39:38.996974+0000","last_unstale":"2024-02-26T10:39:57.145157+0000","last_undegraded":"2024-02-26T10:39:57.145157+0000","last_fullsized":"2024-02-26T10:39:57.145157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:48.227511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250972+0000","last_change":"2024-02-26T10:39:40.005719+0000","last_active":"2024-02-26T10:39:57.250972+0000","last_peered":"2024-02-26T10:39:57.250972+0000","last_clean":"2024-02-26T10:39:57.250972+0000","last_became_active":"2024-02-26T10:39:40.005399+0000","last_became_peered":"2024-02-26T10:39:40.005399+0000","last_unstale":"2024-02-26T10:39:57.250972+0000","last_undegraded":"2024-02-26T10:39:57.250972+0000","last_fullsized":"2024-02-26T10:39:57.250972+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:40.993108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140902+0000","last_change":"2024-02-26T10:39:55.142603+0000","last_active":"2024-02-26T10:39:57.140902+0000","last_peered":"2024-02-26T10:39:57.140902+0000","last_clean":"2024-02-26T10:39:57.140902+0000","last_became_active":"2024-02-26T10:39:55.142285+0000","last_became_peered":"2024-02-26T10:39:55.142285+0000","last_unstale":"2024-02-26T10:39:57.140902+0000","last_undegraded":"2024-02-26T10:39:57.140902+0000","last_fullsized":"2024-02-26T10:39:57.140902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:56.179993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258612+0000","last_change":"2024-02-26T10:39:56.142389+0000","last_active":"2024-02-26T10:39:57.258612+0000","last_peered":"2024-02-26T10:39:57.258612+0000","last_clean":"2024-02-26T10:39:57.258612+0000","last_became_active":"2024-02-26T10:39:56.141659+0000","last_became_peered":"2024-02-26T10:39:56.141659+0000","last_unstale":"2024-02-26T10:39:57.258612+0000","last_undegraded":"2024-02-26T10:39:57.258612+0000","last_fullsized":"2024-02-26T10:39:57.258612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:37.401643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145227+0000","last_change":"2024-02-26T10:39:39.008983+0000","last_active":"2024-02-26T10:39:57.145227+0000","last_peered":"2024-02-26T10:39:57.145227+0000","last_clean":"2024-02-26T10:39:57.145227+0000","last_became_active":"2024-02-26T10:39:39.008687+0000","last_became_peered":"2024-02-26T10:39:39.008687+0000","last_unstale":"2024-02-26T10:39:57.145227+0000","last_undegraded":"2024-02-26T10:39:57.145227+0000","last_fullsized":"2024-02-26T10:39:57.145227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:08:52.607215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254711+0000","last_change":"2024-02-26T10:39:40.017294+0000","last_active":"2024-02-26T10:39:57.254711+0000","last_peered":"2024-02-26T10:39:57.254711+0000","last_clean":"2024-02-26T10:39:57.254711+0000","last_became_active":"2024-02-26T10:39:40.016268+0000","last_became_peered":"2024-02-26T10:39:40.016268+0000","last_unstale":"2024-02-26T10:39:57.254711+0000","last_undegraded":"2024-02-26T10:39:57.254711+0000","last_fullsized":"2024-02-26T10:39:57.254711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140639+0000","last_change":"2024-02-26T10:39:55.142899+0000","last_active":"2024-02-26T10:39:57.140639+0000","last_peered":"2024-02-26T10:39:57.140639+0000","last_clean":"2024-02-26T10:39:57.140639+0000","last_became_active":"2024-02-26T10:39:55.142393+0000","last_became_peered":"2024-02-26T10:39:55.142393+0000","last_unstale":"2024-02-26T10:39:57.140639+0000","last_undegraded":"2024-02-26T10:39:57.140639+0000","last_fullsized":"2024-02-26T10:39:57.140639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:31:53.873062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258250+0000","last_change":"2024-02-26T10:39:56.138495+0000","last_active":"2024-02-26T10:39:57.258250+0000","last_peered":"2024-02-26T10:39:57.258250+0000","last_clean":"2024-02-26T10:39:57.258250+0000","last_became_active":"2024-02-26T10:39:56.138180+0000","last_became_peered":"2024-02-26T10:39:56.138180+0000","last_unstale":"2024-02-26T10:39:57.258250+0000","last_undegraded":"2024-02-26T10:39:57.258250+0000","last_fullsized":"2024-02-26T10:39:57.258250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:45.443279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251494+0000","last_change":"2024-02-26T10:39:39.001118+0000","last_active":"2024-02-26T10:39:57.251494+0000","last_peered":"2024-02-26T10:39:57.251494+0000","last_clean":"2024-02-26T10:39:57.251494+0000","last_became_active":"2024-02-26T10:39:39.000747+0000","last_became_peered":"2024-02-26T10:39:39.000747+0000","last_unstale":"2024-02-26T10:39:57.251494+0000","last_undegraded":"2024-02-26T10:39:57.251494+0000","last_fullsized":"2024-02-26T10:39:57.251494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:54.239568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145051+0000","last_change":"2024-02-26T10:39:40.026290+0000","last_active":"2024-02-26T10:39:57.145051+0000","last_peered":"2024-02-26T10:39:57.145051+0000","last_clean":"2024-02-26T10:39:57.145051+0000","last_became_active":"2024-02-26T10:39:40.026004+0000","last_became_peered":"2024-02-26T10:39:40.026004+0000","last_unstale":"2024-02-26T10:39:57.145051+0000","last_undegraded":"2024-02-26T10:39:57.145051+0000","last_fullsized":"2024-02-26T10:39:57.145051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254052+0000","last_change":"2024-02-26T10:39:55.142017+0000","last_active":"2024-02-26T10:39:57.254052+0000","last_peered":"2024-02-26T10:39:57.254052+0000","last_clean":"2024-02-26T10:39:57.254052+0000","last_became_active":"2024-02-26T10:39:55.140751+0000","last_became_peered":"2024-02-26T10:39:55.140751+0000","last_unstale":"2024-02-26T10:39:57.254052+0000","last_undegraded":"2024-02-26T10:39:57.254052+0000","last_fullsized":"2024-02-26T10:39:57.254052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:49.472864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250177+0000","last_change":"2024-02-26T10:39:56.155913+0000","last_active":"2024-02-26T10:39:57.250177+0000","last_peered":"2024-02-26T10:39:57.250177+0000","last_clean":"2024-02-26T10:39:57.250177+0000","last_became_active":"2024-02-26T10:39:56.155200+0000","last_became_peered":"2024-02-26T10:39:56.155200+0000","last_unstale":"2024-02-26T10:39:57.250177+0000","last_undegraded":"2024-02-26T10:39:57.250177+0000","last_fullsized":"2024-02-26T10:39:57.250177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:06.696664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138720+0000","last_change":"2024-02-26T10:39:39.002498+0000","last_active":"2024-02-26T10:39:57.138720+0000","last_peered":"2024-02-26T10:39:57.138720+0000","last_clean":"2024-02-26T10:39:57.138720+0000","last_became_active":"2024-02-26T10:39:39.002185+0000","last_became_peered":"2024-02-26T10:39:39.002185+0000","last_unstale":"2024-02-26T10:39:57.138720+0000","last_undegraded":"2024-02-26T10:39:57.138720+0000","last_fullsized":"2024-02-26T10:39:57.138720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:18.706456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256484+0000","last_change":"2024-02-26T10:39:40.008604+0000","last_active":"2024-02-26T10:39:57.256484+0000","last_peered":"2024-02-26T10:39:57.256484+0000","last_clean":"2024-02-26T10:39:57.256484+0000","last_became_active":"2024-02-26T10:39:40.008244+0000","last_became_peered":"2024-02-26T10:39:40.008244+0000","last_unstale":"2024-02-26T10:39:57.256484+0000","last_undegraded":"2024-02-26T10:39:57.256484+0000","last_fullsized":"2024-02-26T10:39:57.256484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257141+0000","last_change":"2024-02-26T10:39:55.146646+0000","last_active":"2024-02-26T10:39:57.257141+0000","last_peered":"2024-02-26T10:39:57.257141+0000","last_clean":"2024-02-26T10:39:57.257141+0000","last_became_active":"2024-02-26T10:39:55.146149+0000","last_became_peered":"2024-02-26T10:39:55.146149+0000","last_unstale":"2024-02-26T10:39:57.257141+0000","last_undegraded":"2024-02-26T10:39:57.257141+0000","last_fullsized":"2024-02-26T10:39:57.257141+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:26:00.601349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258945+0000","last_change":"2024-02-26T10:39:56.138500+0000","last_active":"2024-02-26T10:39:57.258945+0000","last_peered":"2024-02-26T10:39:57.258945+0000","last_clean":"2024-02-26T10:39:57.258945+0000","last_became_active":"2024-02-26T10:39:56.138157+0000","last_became_peered":"2024-02-26T10:39:56.138157+0000","last_unstale":"2024-02-26T10:39:57.258945+0000","last_undegraded":"2024-02-26T10:39:57.258945+0000","last_fullsized":"2024-02-26T10:39:57.258945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:44:34.878600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139928+0000","last_change":"2024-02-26T10:39:39.008406+0000","last_active":"2024-02-26T10:39:57.139928+0000","last_peered":"2024-02-26T10:39:57.139928+0000","last_clean":"2024-02-26T10:39:57.139928+0000","last_became_active":"2024-02-26T10:39:39.008086+0000","last_became_peered":"2024-02-26T10:39:39.008086+0000","last_unstale":"2024-02-26T10:39:57.139928+0000","last_undegraded":"2024-02-26T10:39:57.139928+0000","last_fullsized":"2024-02-26T10:39:57.139928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255887+0000","last_change":"2024-02-26T10:39:40.008946+0000","last_active":"2024-02-26T10:39:57.255887+0000","last_peered":"2024-02-26T10:39:57.255887+0000","last_clean":"2024-02-26T10:39:57.255887+0000","last_became_active":"2024-02-26T10:39:40.008676+0000","last_became_peered":"2024-02-26T10:39:40.008676+0000","last_unstale":"2024-02-26T10:39:57.255887+0000","last_undegraded":"2024-02-26T10:39:57.255887+0000","last_fullsized":"2024-02-26T10:39:57.255887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139206+0000","last_change":"2024-02-26T10:39:55.144193+0000","last_active":"2024-02-26T10:39:57.139206+0000","last_peered":"2024-02-26T10:39:57.139206+0000","last_clean":"2024-02-26T10:39:57.139206+0000","last_became_active":"2024-02-26T10:39:55.143853+0000","last_became_peered":"2024-02-26T10:39:55.143853+0000","last_unstale":"2024-02-26T10:39:57.139206+0000","last_undegraded":"2024-02-26T10:39:57.139206+0000","last_fullsized":"2024-02-26T10:39:57.139206+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:13.048005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256470+0000","last_change":"2024-02-26T10:39:56.145501+0000","last_active":"2024-02-26T10:39:57.256470+0000","last_peered":"2024-02-26T10:39:57.256470+0000","last_clean":"2024-02-26T10:39:57.256470+0000","last_became_active":"2024-02-26T10:39:56.145170+0000","last_became_peered":"2024-02-26T10:39:56.145170+0000","last_unstale":"2024-02-26T10:39:57.256470+0000","last_undegraded":"2024-02-26T10:39:57.256470+0000","last_fullsized":"2024-02-26T10:39:57.256470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:20.079475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250686+0000","last_change":"2024-02-26T10:39:39.000729+0000","last_active":"2024-02-26T10:39:57.250686+0000","last_peered":"2024-02-26T10:39:57.250686+0000","last_clean":"2024-02-26T10:39:57.250686+0000","last_became_active":"2024-02-26T10:39:38.999811+0000","last_became_peered":"2024-02-26T10:39:38.999811+0000","last_unstale":"2024-02-26T10:39:57.250686+0000","last_undegraded":"2024-02-26T10:39:57.250686+0000","last_fullsized":"2024-02-26T10:39:57.250686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139997+0000","last_change":"2024-02-26T10:39:40.006750+0000","last_active":"2024-02-26T10:39:57.139997+0000","last_peered":"2024-02-26T10:39:57.139997+0000","last_clean":"2024-02-26T10:39:57.139997+0000","last_became_active":"2024-02-26T10:39:40.005913+0000","last_became_peered":"2024-02-26T10:39:40.005913+0000","last_unstale":"2024-02-26T10:39:57.139997+0000","last_undegraded":"2024-02-26T10:39:57.139997+0000","last_fullsized":"2024-02-26T10:39:57.139997+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249263+0000","last_change":"2024-02-26T10:39:55.127857+0000","last_active":"2024-02-26T10:39:57.249263+0000","last_peered":"2024-02-26T10:39:57.249263+0000","last_clean":"2024-02-26T10:39:57.249263+0000","last_became_active":"2024-02-26T10:39:55.127539+0000","last_became_peered":"2024-02-26T10:39:55.127539+0000","last_unstale":"2024-02-26T10:39:57.249263+0000","last_undegraded":"2024-02-26T10:39:57.249263+0000","last_fullsized":"2024-02-26T10:39:57.249263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:41.302220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138635+0000","last_change":"2024-02-26T10:39:56.146679+0000","last_active":"2024-02-26T10:39:57.138635+0000","last_peered":"2024-02-26T10:39:57.138635+0000","last_clean":"2024-02-26T10:39:57.138635+0000","last_became_active":"2024-02-26T10:39:56.145640+0000","last_became_peered":"2024-02-26T10:39:56.145640+0000","last_unstale":"2024-02-26T10:39:57.138635+0000","last_undegraded":"2024-02-26T10:39:57.138635+0000","last_fullsized":"2024-02-26T10:39:57.138635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:13.404306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144281+0000","last_change":"2024-02-26T10:39:39.010735+0000","last_active":"2024-02-26T10:39:57.144281+0000","last_peered":"2024-02-26T10:39:57.144281+0000","last_clean":"2024-02-26T10:39:57.144281+0000","last_became_active":"2024-02-26T10:39:39.010394+0000","last_became_peered":"2024-02-26T10:39:39.010394+0000","last_unstale":"2024-02-26T10:39:57.144281+0000","last_undegraded":"2024-02-26T10:39:57.144281+0000","last_fullsized":"2024-02-26T10:39:57.144281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:41:16.074102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250575+0000","last_change":"2024-02-26T10:39:40.018996+0000","last_active":"2024-02-26T10:39:57.250575+0000","last_peered":"2024-02-26T10:39:57.250575+0000","last_clean":"2024-02-26T10:39:57.250575+0000","last_became_active":"2024-02-26T10:39:40.018734+0000","last_became_peered":"2024-02-26T10:39:40.018734+0000","last_unstale":"2024-02-26T10:39:57.250575+0000","last_undegraded":"2024-02-26T10:39:57.250575+0000","last_fullsized":"2024-02-26T10:39:57.250575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:26.470295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139109+0000","last_change":"2024-02-26T10:39:55.144297+0000","last_active":"2024-02-26T10:39:57.139109+0000","last_peered":"2024-02-26T10:39:57.139109+0000","last_clean":"2024-02-26T10:39:57.139109+0000","last_became_active":"2024-02-26T10:39:55.143891+0000","last_became_peered":"2024-02-26T10:39:55.143891+0000","last_unstale":"2024-02-26T10:39:57.139109+0000","last_undegraded":"2024-02-26T10:39:57.139109+0000","last_fullsized":"2024-02-26T10:39:57.139109+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:52.910544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.260580+0000","last_change":"2024-02-26T10:39:56.149171+0000","last_active":"2024-02-26T10:39:57.260580+0000","last_peered":"2024-02-26T10:39:57.260580+0000","last_clean":"2024-02-26T10:39:57.260580+0000","last_became_active":"2024-02-26T10:39:56.148052+0000","last_became_peered":"2024-02-26T10:39:56.148052+0000","last_unstale":"2024-02-26T10:39:57.260580+0000","last_undegraded":"2024-02-26T10:39:57.260580+0000","last_fullsized":"2024-02-26T10:39:57.260580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:08:19.286446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"34'8","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141091+0000","last_change":"2024-02-26T10:39:38.999382+0000","last_active":"2024-02-26T10:39:57.141091+0000","last_peered":"2024-02-26T10:39:57.141091+0000","last_clean":"2024-02-26T10:39:57.141091+0000","last_became_active":"2024-02-26T10:39:38.999045+0000","last_became_peered":"2024-02-26T10:39:38.999045+0000","last_unstale":"2024-02-26T10:39:57.141091+0000","last_undegraded":"2024-02-26T10:39:57.141091+0000","last_fullsized":"2024-02-26T10:39:57.141091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:36.656258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3115,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253980+0000","last_change":"2024-02-26T10:39:40.009580+0000","last_active":"2024-02-26T10:39:57.253980+0000","last_peered":"2024-02-26T10:39:57.253980+0000","last_clean":"2024-02-26T10:39:57.253980+0000","last_became_active":"2024-02-26T10:39:40.009277+0000","last_became_peered":"2024-02-26T10:39:40.009277+0000","last_unstale":"2024-02-26T10:39:57.253980+0000","last_undegraded":"2024-02-26T10:39:57.253980+0000","last_fullsized":"2024-02-26T10:39:57.253980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250463+0000","last_change":"2024-02-26T10:39:55.143834+0000","last_active":"2024-02-26T10:39:57.250463+0000","last_peered":"2024-02-26T10:39:57.250463+0000","last_clean":"2024-02-26T10:39:57.250463+0000","last_became_active":"2024-02-26T10:39:55.143492+0000","last_became_peered":"2024-02-26T10:39:55.143492+0000","last_unstale":"2024-02-26T10:39:57.250463+0000","last_undegraded":"2024-02-26T10:39:57.250463+0000","last_fullsized":"2024-02-26T10:39:57.250463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:19.434585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144795+0000","last_change":"2024-02-26T10:39:56.159534+0000","last_active":"2024-02-26T10:39:57.144795+0000","last_peered":"2024-02-26T10:39:57.144795+0000","last_clean":"2024-02-26T10:39:57.144795+0000","last_became_active":"2024-02-26T10:39:56.159278+0000","last_became_peered":"2024-02-26T10:39:56.159278+0000","last_unstale":"2024-02-26T10:39:57.144795+0000","last_undegraded":"2024-02-26T10:39:57.144795+0000","last_fullsized":"2024-02-26T10:39:57.144795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:02.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254268+0000","last_change":"2024-02-26T10:39:39.010001+0000","last_active":"2024-02-26T10:39:57.254268+0000","last_peered":"2024-02-26T10:39:57.254268+0000","last_clean":"2024-02-26T10:39:57.254268+0000","last_became_active":"2024-02-26T10:39:39.009754+0000","last_became_peered":"2024-02-26T10:39:39.009754+0000","last_unstale":"2024-02-26T10:39:57.254268+0000","last_undegraded":"2024-02-26T10:39:57.254268+0000","last_fullsized":"2024-02-26T10:39:57.254268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140949+0000","last_change":"2024-02-26T10:39:40.006537+0000","last_active":"2024-02-26T10:39:57.140949+0000","last_peered":"2024-02-26T10:39:57.140949+0000","last_clean":"2024-02-26T10:39:57.140949+0000","last_became_active":"2024-02-26T10:39:40.006231+0000","last_became_peered":"2024-02-26T10:39:40.006231+0000","last_unstale":"2024-02-26T10:39:57.140949+0000","last_undegraded":"2024-02-26T10:39:57.140949+0000","last_fullsized":"2024-02-26T10:39:57.140949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145891+0000","last_change":"2024-02-26T10:39:55.136727+0000","last_active":"2024-02-26T10:39:57.145891+0000","last_peered":"2024-02-26T10:39:57.145891+0000","last_clean":"2024-02-26T10:39:57.145891+0000","last_became_active":"2024-02-26T10:39:55.136446+0000","last_became_peered":"2024-02-26T10:39:55.136446+0000","last_unstale":"2024-02-26T10:39:57.145891+0000","last_undegraded":"2024-02-26T10:39:57.145891+0000","last_fullsized":"2024-02-26T10:39:57.145891+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:11.096461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257158+0000","last_change":"2024-02-26T10:39:56.151308+0000","last_active":"2024-02-26T10:39:57.257158+0000","last_peered":"2024-02-26T10:39:57.257158+0000","last_clean":"2024-02-26T10:39:57.257158+0000","last_became_active":"2024-02-26T10:39:56.150991+0000","last_became_peered":"2024-02-26T10:39:56.150991+0000","last_unstale":"2024-02-26T10:39:57.257158+0000","last_undegraded":"2024-02-26T10:39:57.257158+0000","last_fullsized":"2024-02-26T10:39:57.257158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:53.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250864+0000","last_change":"2024-02-26T10:39:39.004487+0000","last_active":"2024-02-26T10:39:57.250864+0000","last_peered":"2024-02-26T10:39:57.250864+0000","last_clean":"2024-02-26T10:39:57.250864+0000","last_became_active":"2024-02-26T10:39:39.004135+0000","last_became_peered":"2024-02-26T10:39:39.004135+0000","last_unstale":"2024-02-26T10:39:57.250864+0000","last_undegraded":"2024-02-26T10:39:57.250864+0000","last_fullsized":"2024-02-26T10:39:57.250864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141998+0000","last_change":"2024-02-26T10:39:40.006041+0000","last_active":"2024-02-26T10:39:57.141998+0000","last_peered":"2024-02-26T10:39:57.141998+0000","last_clean":"2024-02-26T10:39:57.141998+0000","last_became_active":"2024-02-26T10:39:40.002138+0000","last_became_peered":"2024-02-26T10:39:40.002138+0000","last_unstale":"2024-02-26T10:39:57.141998+0000","last_undegraded":"2024-02-26T10:39:57.141998+0000","last_fullsized":"2024-02-26T10:39:57.141998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:38.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141190+0000","last_change":"2024-02-26T10:39:55.139164+0000","last_active":"2024-02-26T10:39:57.141190+0000","last_peered":"2024-02-26T10:39:57.141190+0000","last_clean":"2024-02-26T10:39:57.141190+0000","last_became_active":"2024-02-26T10:39:55.138554+0000","last_became_peered":"2024-02-26T10:39:55.138554+0000","last_unstale":"2024-02-26T10:39:57.141190+0000","last_undegraded":"2024-02-26T10:39:57.141190+0000","last_fullsized":"2024-02-26T10:39:57.141190+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:07:23.177818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141229+0000","last_change":"2024-02-26T10:39:56.136419+0000","last_active":"2024-02-26T10:39:57.141229+0000","last_peered":"2024-02-26T10:39:57.141229+0000","last_clean":"2024-02-26T10:39:57.141229+0000","last_became_active":"2024-02-26T10:39:56.134559+0000","last_became_peered":"2024-02-26T10:39:56.134559+0000","last_unstale":"2024-02-26T10:39:57.141229+0000","last_undegraded":"2024-02-26T10:39:57.141229+0000","last_fullsized":"2024-02-26T10:39:57.141229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.659293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142138+0000","last_change":"2024-02-26T10:39:39.008501+0000","last_active":"2024-02-26T10:39:57.142138+0000","last_peered":"2024-02-26T10:39:57.142138+0000","last_clean":"2024-02-26T10:39:57.142138+0000","last_became_active":"2024-02-26T10:39:39.008223+0000","last_became_peered":"2024-02-26T10:39:39.008223+0000","last_unstale":"2024-02-26T10:39:57.142138+0000","last_undegraded":"2024-02-26T10:39:57.142138+0000","last_fullsized":"2024-02-26T10:39:57.142138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:48.232676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255904+0000","last_change":"2024-02-26T10:39:40.025178+0000","last_active":"2024-02-26T10:39:57.255904+0000","last_peered":"2024-02-26T10:39:57.255904+0000","last_clean":"2024-02-26T10:39:57.255904+0000","last_became_active":"2024-02-26T10:39:40.024856+0000","last_became_peered":"2024-02-26T10:39:40.024856+0000","last_unstale":"2024-02-26T10:39:57.255904+0000","last_undegraded":"2024-02-26T10:39:57.255904+0000","last_fullsized":"2024-02-26T10:39:57.255904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257906+0000","last_change":"2024-02-26T10:39:55.150872+0000","last_active":"2024-02-26T10:39:57.257906+0000","last_peered":"2024-02-26T10:39:57.257906+0000","last_clean":"2024-02-26T10:39:57.257906+0000","last_became_active":"2024-02-26T10:39:55.150170+0000","last_became_peered":"2024-02-26T10:39:55.150170+0000","last_unstale":"2024-02-26T10:39:57.257906+0000","last_undegraded":"2024-02-26T10:39:57.257906+0000","last_fullsized":"2024-02-26T10:39:57.257906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:10.985831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249900+0000","last_change":"2024-02-26T10:39:56.151022+0000","last_active":"2024-02-26T10:39:57.249900+0000","last_peered":"2024-02-26T10:39:57.249900+0000","last_clean":"2024-02-26T10:39:57.249900+0000","last_became_active":"2024-02-26T10:39:56.143642+0000","last_became_peered":"2024-02-26T10:39:56.143642+0000","last_unstale":"2024-02-26T10:39:57.249900+0000","last_undegraded":"2024-02-26T10:39:57.249900+0000","last_fullsized":"2024-02-26T10:39:57.249900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:31:54.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142753+0000","last_change":"2024-02-26T10:39:39.003513+0000","last_active":"2024-02-26T10:39:57.142753+0000","last_peered":"2024-02-26T10:39:57.142753+0000","last_clean":"2024-02-26T10:39:57.142753+0000","last_became_active":"2024-02-26T10:39:39.002989+0000","last_became_peered":"2024-02-26T10:39:39.002989+0000","last_unstale":"2024-02-26T10:39:57.142753+0000","last_undegraded":"2024-02-26T10:39:57.142753+0000","last_fullsized":"2024-02-26T10:39:57.142753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:13.624048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250412+0000","last_change":"2024-02-26T10:39:40.025109+0000","last_active":"2024-02-26T10:39:57.250412+0000","last_peered":"2024-02-26T10:39:57.250412+0000","last_clean":"2024-02-26T10:39:57.250412+0000","last_became_active":"2024-02-26T10:39:40.024213+0000","last_became_peered":"2024-02-26T10:39:40.024213+0000","last_unstale":"2024-02-26T10:39:57.250412+0000","last_undegraded":"2024-02-26T10:39:57.250412+0000","last_fullsized":"2024-02-26T10:39:57.250412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:48.049981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143925+0000","last_change":"2024-02-26T10:39:55.143200+0000","last_active":"2024-02-26T10:39:57.143925+0000","last_peered":"2024-02-26T10:39:57.143925+0000","last_clean":"2024-02-26T10:39:57.143925+0000","last_became_active":"2024-02-26T10:39:55.142811+0000","last_became_peered":"2024-02-26T10:39:55.142811+0000","last_unstale":"2024-02-26T10:39:57.143925+0000","last_undegraded":"2024-02-26T10:39:57.143925+0000","last_fullsized":"2024-02-26T10:39:57.143925+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:26.522970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143592+0000","last_change":"2024-02-26T10:39:56.145300+0000","last_active":"2024-02-26T10:39:57.143592+0000","last_peered":"2024-02-26T10:39:57.143592+0000","last_clean":"2024-02-26T10:39:57.143592+0000","last_became_active":"2024-02-26T10:39:56.144515+0000","last_became_peered":"2024-02-26T10:39:56.144515+0000","last_unstale":"2024-02-26T10:39:57.143592+0000","last_undegraded":"2024-02-26T10:39:57.143592+0000","last_fullsized":"2024-02-26T10:39:57.143592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:44:34.878600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139805+0000","last_change":"2024-02-26T10:39:38.998681+0000","last_active":"2024-02-26T10:39:57.139805+0000","last_peered":"2024-02-26T10:39:57.139805+0000","last_clean":"2024-02-26T10:39:57.139805+0000","last_became_active":"2024-02-26T10:39:38.998355+0000","last_became_peered":"2024-02-26T10:39:38.998355+0000","last_unstale":"2024-02-26T10:39:57.139805+0000","last_undegraded":"2024-02-26T10:39:57.139805+0000","last_fullsized":"2024-02-26T10:39:57.139805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:54.191021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142732+0000","last_change":"2024-02-26T10:39:40.014474+0000","last_active":"2024-02-26T10:39:57.142732+0000","last_peered":"2024-02-26T10:39:57.142732+0000","last_clean":"2024-02-26T10:39:57.142732+0000","last_became_active":"2024-02-26T10:39:40.014129+0000","last_became_peered":"2024-02-26T10:39:40.014129+0000","last_unstale":"2024-02-26T10:39:57.142732+0000","last_undegraded":"2024-02-26T10:39:57.142732+0000","last_fullsized":"2024-02-26T10:39:57.142732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:55.255246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140095+0000","last_change":"2024-02-26T10:39:55.147338+0000","last_active":"2024-02-26T10:39:57.140095+0000","last_peered":"2024-02-26T10:39:57.140095+0000","last_clean":"2024-02-26T10:39:57.140095+0000","last_became_active":"2024-02-26T10:39:55.147106+0000","last_became_peered":"2024-02-26T10:39:55.147106+0000","last_unstale":"2024-02-26T10:39:57.140095+0000","last_undegraded":"2024-02-26T10:39:57.140095+0000","last_fullsized":"2024-02-26T10:39:57.140095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:19.064657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255083+0000","last_change":"2024-02-26T10:39:56.148427+0000","last_active":"2024-02-26T10:39:57.255083+0000","last_peered":"2024-02-26T10:39:57.255083+0000","last_clean":"2024-02-26T10:39:57.255083+0000","last_became_active":"2024-02-26T10:39:56.148146+0000","last_became_peered":"2024-02-26T10:39:56.148146+0000","last_unstale":"2024-02-26T10:39:57.255083+0000","last_undegraded":"2024-02-26T10:39:57.255083+0000","last_fullsized":"2024-02-26T10:39:57.255083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:26.929354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144317+0000","last_change":"2024-02-26T10:39:39.007073+0000","last_active":"2024-02-26T10:39:57.144317+0000","last_peered":"2024-02-26T10:39:57.144317+0000","last_clean":"2024-02-26T10:39:57.144317+0000","last_became_active":"2024-02-26T10:39:39.004616+0000","last_became_peered":"2024-02-26T10:39:39.004616+0000","last_unstale":"2024-02-26T10:39:57.144317+0000","last_undegraded":"2024-02-26T10:39:57.144317+0000","last_fullsized":"2024-02-26T10:39:57.144317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:07.037054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252940+0000","last_change":"2024-02-26T10:39:40.024744+0000","last_active":"2024-02-26T10:39:57.252940+0000","last_peered":"2024-02-26T10:39:57.252940+0000","last_clean":"2024-02-26T10:39:57.252940+0000","last_became_active":"2024-02-26T10:39:40.023928+0000","last_became_peered":"2024-02-26T10:39:40.023928+0000","last_unstale":"2024-02-26T10:39:57.252940+0000","last_undegraded":"2024-02-26T10:39:57.252940+0000","last_fullsized":"2024-02-26T10:39:57.252940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249140+0000","last_change":"2024-02-26T10:39:55.134727+0000","last_active":"2024-02-26T10:39:57.249140+0000","last_peered":"2024-02-26T10:39:57.249140+0000","last_clean":"2024-02-26T10:39:57.249140+0000","last_became_active":"2024-02-26T10:39:55.132818+0000","last_became_peered":"2024-02-26T10:39:55.132818+0000","last_unstale":"2024-02-26T10:39:57.249140+0000","last_undegraded":"2024-02-26T10:39:57.249140+0000","last_fullsized":"2024-02-26T10:39:57.249140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:22.197650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249126+0000","last_change":"2024-02-26T10:39:56.141946+0000","last_active":"2024-02-26T10:39:57.249126+0000","last_peered":"2024-02-26T10:39:57.249126+0000","last_clean":"2024-02-26T10:39:57.249126+0000","last_became_active":"2024-02-26T10:39:56.141311+0000","last_became_peered":"2024-02-26T10:39:56.141311+0000","last_unstale":"2024-02-26T10:39:57.249126+0000","last_undegraded":"2024-02-26T10:39:57.249126+0000","last_fullsized":"2024-02-26T10:39:57.249126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:50.487682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143993+0000","last_change":"2024-02-26T10:39:39.005253+0000","last_active":"2024-02-26T10:39:57.143993+0000","last_peered":"2024-02-26T10:39:57.143993+0000","last_clean":"2024-02-26T10:39:57.143993+0000","last_became_active":"2024-02-26T10:39:39.004947+0000","last_became_peered":"2024-02-26T10:39:39.004947+0000","last_unstale":"2024-02-26T10:39:57.143993+0000","last_undegraded":"2024-02-26T10:39:57.143993+0000","last_fullsized":"2024-02-26T10:39:57.143993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:19:13.361650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257547+0000","last_change":"2024-02-26T10:39:40.017365+0000","last_active":"2024-02-26T10:39:57.257547+0000","last_peered":"2024-02-26T10:39:57.257547+0000","last_clean":"2024-02-26T10:39:57.257547+0000","last_became_active":"2024-02-26T10:39:40.010336+0000","last_became_peered":"2024-02-26T10:39:40.010336+0000","last_unstale":"2024-02-26T10:39:57.257547+0000","last_undegraded":"2024-02-26T10:39:57.257547+0000","last_fullsized":"2024-02-26T10:39:57.257547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253111+0000","last_change":"2024-02-26T10:39:55.139886+0000","last_active":"2024-02-26T10:39:57.253111+0000","last_peered":"2024-02-26T10:39:57.253111+0000","last_clean":"2024-02-26T10:39:57.253111+0000","last_became_active":"2024-02-26T10:39:55.139576+0000","last_became_peered":"2024-02-26T10:39:55.139576+0000","last_unstale":"2024-02-26T10:39:57.253111+0000","last_undegraded":"2024-02-26T10:39:57.253111+0000","last_fullsized":"2024-02-26T10:39:57.253111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:01.886808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258390+0000","last_change":"2024-02-26T10:39:56.150239+0000","last_active":"2024-02-26T10:39:57.258390+0000","last_peered":"2024-02-26T10:39:57.258390+0000","last_clean":"2024-02-26T10:39:57.258390+0000","last_became_active":"2024-02-26T10:39:56.147771+0000","last_became_peered":"2024-02-26T10:39:56.147771+0000","last_unstale":"2024-02-26T10:39:57.258390+0000","last_undegraded":"2024-02-26T10:39:57.258390+0000","last_fullsized":"2024-02-26T10:39:57.258390+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:18.580748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145066+0000","last_change":"2024-02-26T10:39:55.140948+0000","last_active":"2024-02-26T10:39:57.145066+0000","last_peered":"2024-02-26T10:39:57.145066+0000","last_clean":"2024-02-26T10:39:57.145066+0000","last_became_active":"2024-02-26T10:39:55.140674+0000","last_became_peered":"2024-02-26T10:39:55.140674+0000","last_unstale":"2024-02-26T10:39:57.145066+0000","last_undegraded":"2024-02-26T10:39:57.145066+0000","last_fullsized":"2024-02-26T10:39:57.145066+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:36.808744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141400+0000","last_change":"2024-02-26T10:39:38.993592+0000","last_active":"2024-02-26T10:39:57.141400+0000","last_peered":"2024-02-26T10:39:57.141400+0000","last_clean":"2024-02-26T10:39:57.141400+0000","last_became_active":"2024-02-26T10:39:38.993042+0000","last_became_peered":"2024-02-26T10:39:38.993042+0000","last_unstale":"2024-02-26T10:39:57.141400+0000","last_undegraded":"2024-02-26T10:39:57.141400+0000","last_fullsized":"2024-02-26T10:39:57.141400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:06:10.297346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251154+0000","last_change":"2024-02-26T10:39:40.008958+0000","last_active":"2024-02-26T10:39:57.251154+0000","last_peered":"2024-02-26T10:39:57.251154+0000","last_clean":"2024-02-26T10:39:57.251154+0000","last_became_active":"2024-02-26T10:39:40.008566+0000","last_became_peered":"2024-02-26T10:39:40.008566+0000","last_unstale":"2024-02-26T10:39:57.251154+0000","last_undegraded":"2024-02-26T10:39:57.251154+0000","last_fullsized":"2024-02-26T10:39:57.251154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251377+0000","last_change":"2024-02-26T10:39:56.139143+0000","last_active":"2024-02-26T10:39:57.251377+0000","last_peered":"2024-02-26T10:39:57.251377+0000","last_clean":"2024-02-26T10:39:57.251377+0000","last_became_active":"2024-02-26T10:39:56.138676+0000","last_became_peered":"2024-02-26T10:39:56.138676+0000","last_unstale":"2024-02-26T10:39:57.251377+0000","last_undegraded":"2024-02-26T10:39:57.251377+0000","last_fullsized":"2024-02-26T10:39:57.251377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:02.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142124+0000","last_change":"2024-02-26T10:39:55.151001+0000","last_active":"2024-02-26T10:39:57.142124+0000","last_peered":"2024-02-26T10:39:57.142124+0000","last_clean":"2024-02-26T10:39:57.142124+0000","last_became_active":"2024-02-26T10:39:55.150677+0000","last_became_peered":"2024-02-26T10:39:55.150677+0000","last_unstale":"2024-02-26T10:39:57.142124+0000","last_undegraded":"2024-02-26T10:39:57.142124+0000","last_fullsized":"2024-02-26T10:39:57.142124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:45.497453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141113+0000","last_change":"2024-02-26T10:39:39.000798+0000","last_active":"2024-02-26T10:39:57.141113+0000","last_peered":"2024-02-26T10:39:57.141113+0000","last_clean":"2024-02-26T10:39:57.141113+0000","last_became_active":"2024-02-26T10:39:38.998438+0000","last_became_peered":"2024-02-26T10:39:38.998438+0000","last_unstale":"2024-02-26T10:39:57.141113+0000","last_undegraded":"2024-02-26T10:39:57.141113+0000","last_fullsized":"2024-02-26T10:39:57.141113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:29:40.306221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144467+0000","last_change":"2024-02-26T10:39:39.992637+0000","last_active":"2024-02-26T10:39:57.144467+0000","last_peered":"2024-02-26T10:39:57.144467+0000","last_clean":"2024-02-26T10:39:57.144467+0000","last_became_active":"2024-02-26T10:39:39.991960+0000","last_became_peered":"2024-02-26T10:39:39.991960+0000","last_unstale":"2024-02-26T10:39:57.144467+0000","last_undegraded":"2024-02-26T10:39:57.144467+0000","last_fullsized":"2024-02-26T10:39:57.144467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251535+0000","last_change":"2024-02-26T10:39:56.144807+0000","last_active":"2024-02-26T10:39:57.251535+0000","last_peered":"2024-02-26T10:39:57.251535+0000","last_clean":"2024-02-26T10:39:57.251535+0000","last_became_active":"2024-02-26T10:39:56.144227+0000","last_became_peered":"2024-02-26T10:39:56.144227+0000","last_unstale":"2024-02-26T10:39:57.251535+0000","last_undegraded":"2024-02-26T10:39:57.251535+0000","last_fullsized":"2024-02-26T10:39:57.251535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:01.709690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250666+0000","last_change":"2024-02-26T10:39:39.013171+0000","last_active":"2024-02-26T10:39:57.250666+0000","last_peered":"2024-02-26T10:39:57.250666+0000","last_clean":"2024-02-26T10:39:57.250666+0000","last_became_active":"2024-02-26T10:39:39.012785+0000","last_became_peered":"2024-02-26T10:39:39.012785+0000","last_unstale":"2024-02-26T10:39:57.250666+0000","last_undegraded":"2024-02-26T10:39:57.250666+0000","last_fullsized":"2024-02-26T10:39:57.250666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:04:56.455641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259712+0000","last_change":"2024-02-26T10:39:40.011479+0000","last_active":"2024-02-26T10:39:57.259712+0000","last_peered":"2024-02-26T10:39:57.259712+0000","last_clean":"2024-02-26T10:39:57.259712+0000","last_became_active":"2024-02-26T10:39:40.011123+0000","last_became_peered":"2024-02-26T10:39:40.011123+0000","last_unstale":"2024-02-26T10:39:57.259712+0000","last_undegraded":"2024-02-26T10:39:57.259712+0000","last_fullsized":"2024-02-26T10:39:57.259712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258691+0000","last_change":"2024-02-26T10:39:55.148876+0000","last_active":"2024-02-26T10:39:57.258691+0000","last_peered":"2024-02-26T10:39:57.258691+0000","last_clean":"2024-02-26T10:39:57.258691+0000","last_became_active":"2024-02-26T10:39:55.148547+0000","last_became_peered":"2024-02-26T10:39:55.148547+0000","last_unstale":"2024-02-26T10:39:57.258691+0000","last_undegraded":"2024-02-26T10:39:57.258691+0000","last_fullsized":"2024-02-26T10:39:57.258691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:01.886808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254068+0000","last_change":"2024-02-26T10:39:56.152573+0000","last_active":"2024-02-26T10:39:57.254068+0000","last_peered":"2024-02-26T10:39:57.254068+0000","last_clean":"2024-02-26T10:39:57.254068+0000","last_became_active":"2024-02-26T10:39:56.152297+0000","last_became_peered":"2024-02-26T10:39:56.152297+0000","last_unstale":"2024-02-26T10:39:57.254068+0000","last_undegraded":"2024-02-26T10:39:57.254068+0000","last_fullsized":"2024-02-26T10:39:57.254068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:29.247494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259949+0000","last_change":"2024-02-26T10:39:39.014072+0000","last_active":"2024-02-26T10:39:57.259949+0000","last_peered":"2024-02-26T10:39:57.259949+0000","last_clean":"2024-02-26T10:39:57.259949+0000","last_became_active":"2024-02-26T10:39:39.013810+0000","last_became_peered":"2024-02-26T10:39:39.013810+0000","last_unstale":"2024-02-26T10:39:57.259949+0000","last_undegraded":"2024-02-26T10:39:57.259949+0000","last_fullsized":"2024-02-26T10:39:57.259949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:12:03.804658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250528+0000","last_change":"2024-02-26T10:39:40.021369+0000","last_active":"2024-02-26T10:39:57.250528+0000","last_peered":"2024-02-26T10:39:57.250528+0000","last_clean":"2024-02-26T10:39:57.250528+0000","last_became_active":"2024-02-26T10:39:40.021061+0000","last_became_peered":"2024-02-26T10:39:40.021061+0000","last_unstale":"2024-02-26T10:39:57.250528+0000","last_undegraded":"2024-02-26T10:39:57.250528+0000","last_fullsized":"2024-02-26T10:39:57.250528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251509+0000","last_change":"2024-02-26T10:39:55.128982+0000","last_active":"2024-02-26T10:39:57.251509+0000","last_peered":"2024-02-26T10:39:57.251509+0000","last_clean":"2024-02-26T10:39:57.251509+0000","last_became_active":"2024-02-26T10:39:55.126448+0000","last_became_peered":"2024-02-26T10:39:55.126448+0000","last_unstale":"2024-02-26T10:39:57.251509+0000","last_undegraded":"2024-02-26T10:39:57.251509+0000","last_fullsized":"2024-02-26T10:39:57.251509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:53.558326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143449+0000","last_change":"2024-02-26T10:39:56.149841+0000","last_active":"2024-02-26T10:39:57.143449+0000","last_peered":"2024-02-26T10:39:57.143449+0000","last_clean":"2024-02-26T10:39:57.143449+0000","last_became_active":"2024-02-26T10:39:56.148212+0000","last_became_peered":"2024-02-26T10:39:56.148212+0000","last_unstale":"2024-02-26T10:39:57.143449+0000","last_undegraded":"2024-02-26T10:39:57.143449+0000","last_fullsized":"2024-02-26T10:39:57.143449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:54.573144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255382+0000","last_change":"2024-02-26T10:39:40.013245+0000","last_active":"2024-02-26T10:39:57.255382+0000","last_peered":"2024-02-26T10:39:57.255382+0000","last_clean":"2024-02-26T10:39:57.255382+0000","last_became_active":"2024-02-26T10:39:40.008999+0000","last_became_peered":"2024-02-26T10:39:40.008999+0000","last_unstale":"2024-02-26T10:39:57.255382+0000","last_undegraded":"2024-02-26T10:39:57.255382+0000","last_fullsized":"2024-02-26T10:39:57.255382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256915+0000","last_change":"2024-02-26T10:39:56.140813+0000","last_active":"2024-02-26T10:39:57.256915+0000","last_peered":"2024-02-26T10:39:57.256915+0000","last_clean":"2024-02-26T10:39:57.256915+0000","last_became_active":"2024-02-26T10:39:56.140525+0000","last_became_peered":"2024-02-26T10:39:56.140525+0000","last_unstale":"2024-02-26T10:39:57.256915+0000","last_undegraded":"2024-02-26T10:39:57.256915+0000","last_fullsized":"2024-02-26T10:39:57.256915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:40.467226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255695+0000","last_change":"2024-02-26T10:39:40.008658+0000","last_active":"2024-02-26T10:39:57.255695+0000","last_peered":"2024-02-26T10:39:57.255695+0000","last_clean":"2024-02-26T10:39:57.255695+0000","last_became_active":"2024-02-26T10:39:40.008354+0000","last_became_peered":"2024-02-26T10:39:40.008354+0000","last_unstale":"2024-02-26T10:39:57.255695+0000","last_undegraded":"2024-02-26T10:39:57.255695+0000","last_fullsized":"2024-02-26T10:39:57.255695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140075+0000","last_change":"2024-02-26T10:39:56.132854+0000","last_active":"2024-02-26T10:39:57.140075+0000","last_peered":"2024-02-26T10:39:57.140075+0000","last_clean":"2024-02-26T10:39:57.140075+0000","last_became_active":"2024-02-26T10:39:56.130809+0000","last_became_peered":"2024-02-26T10:39:56.130809+0000","last_unstale":"2024-02-26T10:39:57.140075+0000","last_undegraded":"2024-02-26T10:39:57.140075+0000","last_fullsized":"2024-02-26T10:39:57.140075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:20.020460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257284+0000","last_change":"2024-02-26T10:39:40.010029+0000","last_active":"2024-02-26T10:39:57.257284+0000","last_peered":"2024-02-26T10:39:57.257284+0000","last_clean":"2024-02-26T10:39:57.257284+0000","last_became_active":"2024-02-26T10:39:40.009745+0000","last_became_peered":"2024-02-26T10:39:40.009745+0000","last_unstale":"2024-02-26T10:39:57.257284+0000","last_undegraded":"2024-02-26T10:39:57.257284+0000","last_fullsized":"2024-02-26T10:39:57.257284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138983+0000","last_change":"2024-02-26T10:39:56.140807+0000","last_active":"2024-02-26T10:39:57.138983+0000","last_peered":"2024-02-26T10:39:57.138983+0000","last_clean":"2024-02-26T10:39:57.138983+0000","last_became_active":"2024-02-26T10:39:56.140429+0000","last_became_peered":"2024-02-26T10:39:56.140429+0000","last_unstale":"2024-02-26T10:39:57.138983+0000","last_undegraded":"2024-02-26T10:39:57.138983+0000","last_fullsized":"2024-02-26T10:39:57.138983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:53.194566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143736+0000","last_change":"2024-02-26T10:39:39.994088+0000","last_active":"2024-02-26T10:39:57.143736+0000","last_peered":"2024-02-26T10:39:57.143736+0000","last_clean":"2024-02-26T10:39:57.143736+0000","last_became_active":"2024-02-26T10:39:39.993717+0000","last_became_peered":"2024-02-26T10:39:39.993717+0000","last_unstale":"2024-02-26T10:39:57.143736+0000","last_undegraded":"2024-02-26T10:39:57.143736+0000","last_fullsized":"2024-02-26T10:39:57.143736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140571+0000","last_change":"2024-02-26T10:39:56.133103+0000","last_active":"2024-02-26T10:39:57.140571+0000","last_peered":"2024-02-26T10:39:57.140571+0000","last_clean":"2024-02-26T10:39:57.140571+0000","last_became_active":"2024-02-26T10:39:56.132794+0000","last_became_peered":"2024-02-26T10:39:56.132794+0000","last_unstale":"2024-02-26T10:39:57.140571+0000","last_undegraded":"2024-02-26T10:39:57.140571+0000","last_fullsized":"2024-02-26T10:39:57.140571+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:06:13.230409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254546+0000","last_change":"2024-02-26T10:39:40.019372+0000","last_active":"2024-02-26T10:39:57.254546+0000","last_peered":"2024-02-26T10:39:57.254546+0000","last_clean":"2024-02-26T10:39:57.254546+0000","last_became_active":"2024-02-26T10:39:40.019025+0000","last_became_peered":"2024-02-26T10:39:40.019025+0000","last_unstale":"2024-02-26T10:39:57.254546+0000","last_undegraded":"2024-02-26T10:39:57.254546+0000","last_fullsized":"2024-02-26T10:39:57.254546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258943+0000","last_change":"2024-02-26T10:39:56.141645+0000","last_active":"2024-02-26T10:39:57.258943+0000","last_peered":"2024-02-26T10:39:57.258943+0000","last_clean":"2024-02-26T10:39:57.258943+0000","last_became_active":"2024-02-26T10:39:56.140872+0000","last_became_peered":"2024-02-26T10:39:56.140872+0000","last_unstale":"2024-02-26T10:39:57.258943+0000","last_undegraded":"2024-02-26T10:39:57.258943+0000","last_fullsized":"2024-02-26T10:39:57.258943+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:54.573144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251233+0000","last_change":"2024-02-26T10:39:40.008980+0000","last_active":"2024-02-26T10:39:57.251233+0000","last_peered":"2024-02-26T10:39:57.251233+0000","last_clean":"2024-02-26T10:39:57.251233+0000","last_became_active":"2024-02-26T10:39:40.008531+0000","last_became_peered":"2024-02-26T10:39:40.008531+0000","last_unstale":"2024-02-26T10:39:57.251233+0000","last_undegraded":"2024-02-26T10:39:57.251233+0000","last_fullsized":"2024-02-26T10:39:57.251233+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:11:31.399212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253631+0000","last_change":"2024-02-26T10:39:56.146232+0000","last_active":"2024-02-26T10:39:57.253631+0000","last_peered":"2024-02-26T10:39:57.253631+0000","last_clean":"2024-02-26T10:39:57.253631+0000","last_became_active":"2024-02-26T10:39:56.143256+0000","last_became_peered":"2024-02-26T10:39:56.143256+0000","last_unstale":"2024-02-26T10:39:57.253631+0000","last_undegraded":"2024-02-26T10:39:57.253631+0000","last_fullsized":"2024-02-26T10:39:57.253631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:53.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141785+0000","last_change":"2024-02-26T10:39:40.007272+0000","last_active":"2024-02-26T10:39:57.141785+0000","last_peered":"2024-02-26T10:39:57.141785+0000","last_clean":"2024-02-26T10:39:57.141785+0000","last_became_active":"2024-02-26T10:39:40.006923+0000","last_became_peered":"2024-02-26T10:39:40.006923+0000","last_unstale":"2024-02-26T10:39:57.141785+0000","last_undegraded":"2024-02-26T10:39:57.141785+0000","last_fullsized":"2024-02-26T10:39:57.141785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:21.270698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144250+0000","last_change":"2024-02-26T10:39:56.149837+0000","last_active":"2024-02-26T10:39:57.144250+0000","last_peered":"2024-02-26T10:39:57.144250+0000","last_clean":"2024-02-26T10:39:57.144250+0000","last_became_active":"2024-02-26T10:39:56.147660+0000","last_became_peered":"2024-02-26T10:39:56.147660+0000","last_unstale":"2024-02-26T10:39:57.144250+0000","last_undegraded":"2024-02-26T10:39:57.144250+0000","last_fullsized":"2024-02-26T10:39:57.144250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:46.652301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251424+0000","last_change":"2024-02-26T10:39:40.015014+0000","last_active":"2024-02-26T10:39:57.251424+0000","last_peered":"2024-02-26T10:39:57.251424+0000","last_clean":"2024-02-26T10:39:57.251424+0000","last_became_active":"2024-02-26T10:39:40.014611+0000","last_became_peered":"2024-02-26T10:39:40.014611+0000","last_unstale":"2024-02-26T10:39:57.251424+0000","last_undegraded":"2024-02-26T10:39:57.251424+0000","last_fullsized":"2024-02-26T10:39:57.251424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249191+0000","last_change":"2024-02-26T10:39:56.150318+0000","last_active":"2024-02-26T10:39:57.249191+0000","last_peered":"2024-02-26T10:39:57.249191+0000","last_clean":"2024-02-26T10:39:57.249191+0000","last_became_active":"2024-02-26T10:39:56.150020+0000","last_became_peered":"2024-02-26T10:39:56.150020+0000","last_unstale":"2024-02-26T10:39:57.249191+0000","last_undegraded":"2024-02-26T10:39:57.249191+0000","last_fullsized":"2024-02-26T10:39:57.249191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:52.852187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252380+0000","last_change":"2024-02-26T10:39:40.017086+0000","last_active":"2024-02-26T10:39:57.252380+0000","last_peered":"2024-02-26T10:39:57.252380+0000","last_clean":"2024-02-26T10:39:57.252380+0000","last_became_active":"2024-02-26T10:39:40.016779+0000","last_became_peered":"2024-02-26T10:39:40.016779+0000","last_unstale":"2024-02-26T10:39:57.252380+0000","last_undegraded":"2024-02-26T10:39:57.252380+0000","last_fullsized":"2024-02-26T10:39:57.252380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139808+0000","last_change":"2024-02-26T10:39:56.154203+0000","last_active":"2024-02-26T10:39:57.139808+0000","last_peered":"2024-02-26T10:39:57.139808+0000","last_clean":"2024-02-26T10:39:57.139808+0000","last_became_active":"2024-02-26T10:39:56.151496+0000","last_became_peered":"2024-02-26T10:39:56.151496+0000","last_unstale":"2024-02-26T10:39:57.139808+0000","last_undegraded":"2024-02-26T10:39:57.139808+0000","last_fullsized":"2024-02-26T10:39:57.139808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:37.087257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249183+0000","last_change":"2024-02-26T10:39:40.022805+0000","last_active":"2024-02-26T10:39:57.249183+0000","last_peered":"2024-02-26T10:39:57.249183+0000","last_clean":"2024-02-26T10:39:57.249183+0000","last_became_active":"2024-02-26T10:39:40.022510+0000","last_became_peered":"2024-02-26T10:39:40.022510+0000","last_unstale":"2024-02-26T10:39:57.249183+0000","last_undegraded":"2024-02-26T10:39:57.249183+0000","last_fullsized":"2024-02-26T10:39:57.249183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254107+0000","last_change":"2024-02-26T10:39:56.151748+0000","last_active":"2024-02-26T10:39:57.254107+0000","last_peered":"2024-02-26T10:39:57.254107+0000","last_clean":"2024-02-26T10:39:57.254107+0000","last_became_active":"2024-02-26T10:39:56.150645+0000","last_became_peered":"2024-02-26T10:39:56.150645+0000","last_unstale":"2024-02-26T10:39:57.254107+0000","last_undegraded":"2024-02-26T10:39:57.254107+0000","last_fullsized":"2024-02-26T10:39:57.254107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:52.489061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144063+0000","last_change":"2024-02-26T10:39:39.990872+0000","last_active":"2024-02-26T10:39:57.144063+0000","last_peered":"2024-02-26T10:39:57.144063+0000","last_clean":"2024-02-26T10:39:57.144063+0000","last_became_active":"2024-02-26T10:39:39.990134+0000","last_became_peered":"2024-02-26T10:39:39.990134+0000","last_unstale":"2024-02-26T10:39:57.144063+0000","last_undegraded":"2024-02-26T10:39:57.144063+0000","last_fullsized":"2024-02-26T10:39:57.144063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256202+0000","last_change":"2024-02-26T10:39:56.151370+0000","last_active":"2024-02-26T10:39:57.256202+0000","last_peered":"2024-02-26T10:39:57.256202+0000","last_clean":"2024-02-26T10:39:57.256202+0000","last_became_active":"2024-02-26T10:39:56.150807+0000","last_became_peered":"2024-02-26T10:39:56.150807+0000","last_unstale":"2024-02-26T10:39:57.256202+0000","last_undegraded":"2024-02-26T10:39:57.256202+0000","last_fullsized":"2024-02-26T10:39:57.256202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:18.352022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143431+0000","last_change":"2024-02-26T10:39:40.003035+0000","last_active":"2024-02-26T10:39:57.143431+0000","last_peered":"2024-02-26T10:39:57.143431+0000","last_clean":"2024-02-26T10:39:57.143431+0000","last_became_active":"2024-02-26T10:39:40.002724+0000","last_became_peered":"2024-02-26T10:39:40.002724+0000","last_unstale":"2024-02-26T10:39:57.143431+0000","last_undegraded":"2024-02-26T10:39:57.143431+0000","last_fullsized":"2024-02-26T10:39:57.143431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144597+0000","last_change":"2024-02-26T10:39:56.163643+0000","last_active":"2024-02-26T10:39:57.144597+0000","last_peered":"2024-02-26T10:39:57.144597+0000","last_clean":"2024-02-26T10:39:57.144597+0000","last_became_active":"2024-02-26T10:39:56.163310+0000","last_became_peered":"2024-02-26T10:39:56.163310+0000","last_unstale":"2024-02-26T10:39:57.144597+0000","last_undegraded":"2024-02-26T10:39:57.144597+0000","last_fullsized":"2024-02-26T10:39:57.144597+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:51:53.159078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141323+0000","last_change":"2024-02-26T10:39:40.028123+0000","last_active":"2024-02-26T10:39:57.141323+0000","last_peered":"2024-02-26T10:39:57.141323+0000","last_clean":"2024-02-26T10:39:57.141323+0000","last_became_active":"2024-02-26T10:39:40.027299+0000","last_became_peered":"2024-02-26T10:39:40.027299+0000","last_unstale":"2024-02-26T10:39:57.141323+0000","last_undegraded":"2024-02-26T10:39:57.141323+0000","last_fullsized":"2024-02-26T10:39:57.141323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:54:29.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145385+0000","last_change":"2024-02-26T10:39:56.150941+0000","last_active":"2024-02-26T10:39:57.145385+0000","last_peered":"2024-02-26T10:39:57.145385+0000","last_clean":"2024-02-26T10:39:57.145385+0000","last_became_active":"2024-02-26T10:39:56.148004+0000","last_became_peered":"2024-02-26T10:39:56.148004+0000","last_unstale":"2024-02-26T10:39:57.145385+0000","last_undegraded":"2024-02-26T10:39:57.145385+0000","last_fullsized":"2024-02-26T10:39:57.145385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:01.709690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140342+0000","last_change":"2024-02-26T10:39:39.996782+0000","last_active":"2024-02-26T10:39:57.140342+0000","last_peered":"2024-02-26T10:39:57.140342+0000","last_clean":"2024-02-26T10:39:57.140342+0000","last_became_active":"2024-02-26T10:39:39.996469+0000","last_became_peered":"2024-02-26T10:39:39.996469+0000","last_unstale":"2024-02-26T10:39:57.140342+0000","last_undegraded":"2024-02-26T10:39:57.140342+0000","last_fullsized":"2024-02-26T10:39:57.140342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140977+0000","last_change":"2024-02-26T10:39:56.144562+0000","last_active":"2024-02-26T10:39:57.140977+0000","last_peered":"2024-02-26T10:39:57.140977+0000","last_clean":"2024-02-26T10:39:57.140977+0000","last_became_active":"2024-02-26T10:39:56.143770+0000","last_became_peered":"2024-02-26T10:39:56.143770+0000","last_unstale":"2024-02-26T10:39:57.140977+0000","last_undegraded":"2024-02-26T10:39:57.140977+0000","last_fullsized":"2024-02-26T10:39:57.140977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:25:09.826997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141686+0000","last_change":"2024-02-26T10:39:39.998011+0000","last_active":"2024-02-26T10:39:57.141686+0000","last_peered":"2024-02-26T10:39:57.141686+0000","last_clean":"2024-02-26T10:39:57.141686+0000","last_became_active":"2024-02-26T10:39:39.997725+0000","last_became_peered":"2024-02-26T10:39:39.997725+0000","last_unstale":"2024-02-26T10:39:57.141686+0000","last_undegraded":"2024-02-26T10:39:57.141686+0000","last_fullsized":"2024-02-26T10:39:57.141686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:02.449804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259034+0000","last_change":"2024-02-26T10:39:56.142150+0000","last_active":"2024-02-26T10:39:57.259034+0000","last_peered":"2024-02-26T10:39:57.259034+0000","last_clean":"2024-02-26T10:39:57.259034+0000","last_became_active":"2024-02-26T10:39:56.141273+0000","last_became_peered":"2024-02-26T10:39:56.141273+0000","last_unstale":"2024-02-26T10:39:57.259034+0000","last_undegraded":"2024-02-26T10:39:57.259034+0000","last_fullsized":"2024-02-26T10:39:57.259034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:46.652301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.260262+0000","last_change":"2024-02-26T10:39:40.017107+0000","last_active":"2024-02-26T10:39:57.260262+0000","last_peered":"2024-02-26T10:39:57.260262+0000","last_clean":"2024-02-26T10:39:57.260262+0000","last_became_active":"2024-02-26T10:39:40.016813+0000","last_became_peered":"2024-02-26T10:39:40.016813+0000","last_unstale":"2024-02-26T10:39:57.260262+0000","last_undegraded":"2024-02-26T10:39:57.260262+0000","last_fullsized":"2024-02-26T10:39:57.260262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249178+0000","last_change":"2024-02-26T10:39:56.134719+0000","last_active":"2024-02-26T10:39:57.249178+0000","last_peered":"2024-02-26T10:39:57.249178+0000","last_clean":"2024-02-26T10:39:57.249178+0000","last_became_active":"2024-02-26T10:39:56.134420+0000","last_became_peered":"2024-02-26T10:39:56.134420+0000","last_unstale":"2024-02-26T10:39:57.249178+0000","last_undegraded":"2024-02-26T10:39:57.249178+0000","last_fullsized":"2024-02-26T10:39:57.249178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:12.000649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256536+0000","last_change":"2024-02-26T10:39:40.011104+0000","last_active":"2024-02-26T10:39:57.256536+0000","last_peered":"2024-02-26T10:39:57.256536+0000","last_clean":"2024-02-26T10:39:57.256536+0000","last_became_active":"2024-02-26T10:39:40.010774+0000","last_became_peered":"2024-02-26T10:39:40.010774+0000","last_unstale":"2024-02-26T10:39:57.256536+0000","last_undegraded":"2024-02-26T10:39:57.256536+0000","last_fullsized":"2024-02-26T10:39:57.256536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142302+0000","last_change":"2024-02-26T10:39:56.131881+0000","last_active":"2024-02-26T10:39:57.142302+0000","last_peered":"2024-02-26T10:39:57.142302+0000","last_clean":"2024-02-26T10:39:57.142302+0000","last_became_active":"2024-02-26T10:39:56.131517+0000","last_became_peered":"2024-02-26T10:39:56.131517+0000","last_unstale":"2024-02-26T10:39:57.142302+0000","last_undegraded":"2024-02-26T10:39:57.142302+0000","last_fullsized":"2024-02-26T10:39:57.142302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:29:39.425379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252741+0000","last_change":"2024-02-26T10:39:40.024788+0000","last_active":"2024-02-26T10:39:57.252741+0000","last_peered":"2024-02-26T10:39:57.252741+0000","last_clean":"2024-02-26T10:39:57.252741+0000","last_became_active":"2024-02-26T10:39:40.024454+0000","last_became_peered":"2024-02-26T10:39:40.024454+0000","last_unstale":"2024-02-26T10:39:57.252741+0000","last_undegraded":"2024-02-26T10:39:57.252741+0000","last_fullsized":"2024-02-26T10:39:57.252741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254552+0000","last_change":"2024-02-26T10:39:56.142843+0000","last_active":"2024-02-26T10:39:57.254552+0000","last_peered":"2024-02-26T10:39:57.254552+0000","last_clean":"2024-02-26T10:39:57.254552+0000","last_became_active":"2024-02-26T10:39:56.142576+0000","last_became_peered":"2024-02-26T10:39:56.142576+0000","last_unstale":"2024-02-26T10:39:57.254552+0000","last_undegraded":"2024-02-26T10:39:57.254552+0000","last_fullsized":"2024-02-26T10:39:57.254552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:28.003726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139752+0000","last_change":"2024-02-26T10:39:39.999249+0000","last_active":"2024-02-26T10:39:57.139752+0000","last_peered":"2024-02-26T10:39:57.139752+0000","last_clean":"2024-02-26T10:39:57.139752+0000","last_became_active":"2024-02-26T10:39:39.998319+0000","last_became_peered":"2024-02-26T10:39:39.998319+0000","last_unstale":"2024-02-26T10:39:57.139752+0000","last_undegraded":"2024-02-26T10:39:57.139752+0000","last_fullsized":"2024-02-26T10:39:57.139752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:29.312334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138807+0000","last_change":"2024-02-26T10:39:56.142489+0000","last_active":"2024-02-26T10:39:57.138807+0000","last_peered":"2024-02-26T10:39:57.138807+0000","last_clean":"2024-02-26T10:39:57.138807+0000","last_became_active":"2024-02-26T10:39:56.141992+0000","last_became_peered":"2024-02-26T10:39:56.141992+0000","last_unstale":"2024-02-26T10:39:57.138807+0000","last_undegraded":"2024-02-26T10:39:57.138807+0000","last_fullsized":"2024-02-26T10:39:57.138807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:37.709990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142357+0000","last_change":"2024-02-26T10:39:40.000902+0000","last_active":"2024-02-26T10:39:57.142357+0000","last_peered":"2024-02-26T10:39:57.142357+0000","last_clean":"2024-02-26T10:39:57.142357+0000","last_became_active":"2024-02-26T10:39:39.999441+0000","last_became_peered":"2024-02-26T10:39:39.999441+0000","last_unstale":"2024-02-26T10:39:57.142357+0000","last_undegraded":"2024-02-26T10:39:57.142357+0000","last_fullsized":"2024-02-26T10:39:57.142357+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:27.937806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250346+0000","last_change":"2024-02-26T10:39:56.155269+0000","last_active":"2024-02-26T10:39:57.250346+0000","last_peered":"2024-02-26T10:39:57.250346+0000","last_clean":"2024-02-26T10:39:57.250346+0000","last_became_active":"2024-02-26T10:39:56.149555+0000","last_became_peered":"2024-02-26T10:39:56.149555+0000","last_unstale":"2024-02-26T10:39:57.250346+0000","last_undegraded":"2024-02-26T10:39:57.250346+0000","last_fullsized":"2024-02-26T10:39:57.250346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:20.079475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254507+0000","last_change":"2024-02-26T10:39:40.011833+0000","last_active":"2024-02-26T10:39:57.254507+0000","last_peered":"2024-02-26T10:39:57.254507+0000","last_clean":"2024-02-26T10:39:57.254507+0000","last_became_active":"2024-02-26T10:39:40.011581+0000","last_became_peered":"2024-02-26T10:39:40.011581+0000","last_unstale":"2024-02-26T10:39:57.254507+0000","last_undegraded":"2024-02-26T10:39:57.254507+0000","last_fullsized":"2024-02-26T10:39:57.254507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250085+0000","last_change":"2024-02-26T10:39:56.150850+0000","last_active":"2024-02-26T10:39:57.250085+0000","last_peered":"2024-02-26T10:39:57.250085+0000","last_clean":"2024-02-26T10:39:57.250085+0000","last_became_active":"2024-02-26T10:39:56.150551+0000","last_became_peered":"2024-02-26T10:39:56.150551+0000","last_unstale":"2024-02-26T10:39:57.250085+0000","last_undegraded":"2024-02-26T10:39:57.250085+0000","last_fullsized":"2024-02-26T10:39:57.250085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:51.955346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258388+0000","last_change":"2024-02-26T10:39:40.017115+0000","last_active":"2024-02-26T10:39:57.258388+0000","last_peered":"2024-02-26T10:39:57.258388+0000","last_clean":"2024-02-26T10:39:57.258388+0000","last_became_active":"2024-02-26T10:39:40.016876+0000","last_became_peered":"2024-02-26T10:39:40.016876+0000","last_unstale":"2024-02-26T10:39:57.258388+0000","last_undegraded":"2024-02-26T10:39:57.258388+0000","last_fullsized":"2024-02-26T10:39:57.258388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253180+0000","last_change":"2024-02-26T10:39:56.146988+0000","last_active":"2024-02-26T10:39:57.253180+0000","last_peered":"2024-02-26T10:39:57.253180+0000","last_clean":"2024-02-26T10:39:57.253180+0000","last_became_active":"2024-02-26T10:39:56.146609+0000","last_became_peered":"2024-02-26T10:39:56.146609+0000","last_unstale":"2024-02-26T10:39:57.253180+0000","last_undegraded":"2024-02-26T10:39:57.253180+0000","last_fullsized":"2024-02-26T10:39:57.253180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:20.079475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251419+0000","last_change":"2024-02-26T10:39:39.995327+0000","last_active":"2024-02-26T10:39:57.251419+0000","last_peered":"2024-02-26T10:39:57.251419+0000","last_clean":"2024-02-26T10:39:57.251419+0000","last_became_active":"2024-02-26T10:39:39.994245+0000","last_became_peered":"2024-02-26T10:39:39.994245+0000","last_unstale":"2024-02-26T10:39:57.251419+0000","last_undegraded":"2024-02-26T10:39:57.251419+0000","last_fullsized":"2024-02-26T10:39:57.251419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249940+0000","last_change":"2024-02-26T10:39:56.136269+0000","last_active":"2024-02-26T10:39:57.249940+0000","last_peered":"2024-02-26T10:39:57.249940+0000","last_clean":"2024-02-26T10:39:57.249940+0000","last_became_active":"2024-02-26T10:39:56.134890+0000","last_became_peered":"2024-02-26T10:39:56.134890+0000","last_unstale":"2024-02-26T10:39:57.249940+0000","last_undegraded":"2024-02-26T10:39:57.249940+0000","last_fullsized":"2024-02-26T10:39:57.249940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:14.062824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.146252+0000","last_change":"2024-02-26T10:39:40.006430+0000","last_active":"2024-02-26T10:39:57.146252+0000","last_peered":"2024-02-26T10:39:57.146252+0000","last_clean":"2024-02-26T10:39:57.146252+0000","last_became_active":"2024-02-26T10:39:40.004846+0000","last_became_peered":"2024-02-26T10:39:40.004846+0000","last_unstale":"2024-02-26T10:39:57.146252+0000","last_undegraded":"2024-02-26T10:39:57.146252+0000","last_fullsized":"2024-02-26T10:39:57.146252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138901+0000","last_change":"2024-02-26T10:39:56.155042+0000","last_active":"2024-02-26T10:39:57.138901+0000","last_peered":"2024-02-26T10:39:57.138901+0000","last_clean":"2024-02-26T10:39:57.138901+0000","last_became_active":"2024-02-26T10:39:56.152151+0000","last_became_peered":"2024-02-26T10:39:56.152151+0000","last_unstale":"2024-02-26T10:39:57.138901+0000","last_undegraded":"2024-02-26T10:39:57.138901+0000","last_fullsized":"2024-02-26T10:39:57.138901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.967204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255104+0000","last_change":"2024-02-26T10:39:40.005845+0000","last_active":"2024-02-26T10:39:57.255104+0000","last_peered":"2024-02-26T10:39:57.255104+0000","last_clean":"2024-02-26T10:39:57.255104+0000","last_became_active":"2024-02-26T10:39:40.005497+0000","last_became_peered":"2024-02-26T10:39:40.005497+0000","last_unstale":"2024-02-26T10:39:57.255104+0000","last_undegraded":"2024-02-26T10:39:57.255104+0000","last_fullsized":"2024-02-26T10:39:57.255104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253128+0000","last_change":"2024-02-26T10:39:56.155017+0000","last_active":"2024-02-26T10:39:57.253128+0000","last_peered":"2024-02-26T10:39:57.253128+0000","last_clean":"2024-02-26T10:39:57.253128+0000","last_became_active":"2024-02-26T10:39:56.154611+0000","last_became_peered":"2024-02-26T10:39:56.154611+0000","last_unstale":"2024-02-26T10:39:57.253128+0000","last_undegraded":"2024-02-26T10:39:57.253128+0000","last_fullsized":"2024-02-26T10:39:57.253128+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:46.512272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257358+0000","last_change":"2024-02-26T10:39:40.014885+0000","last_active":"2024-02-26T10:39:57.257358+0000","last_peered":"2024-02-26T10:39:57.257358+0000","last_clean":"2024-02-26T10:39:57.257358+0000","last_became_active":"2024-02-26T10:39:40.014675+0000","last_became_peered":"2024-02-26T10:39:40.014675+0000","last_unstale":"2024-02-26T10:39:57.257358+0000","last_undegraded":"2024-02-26T10:39:57.257358+0000","last_fullsized":"2024-02-26T10:39:57.257358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249016+0000","last_change":"2024-02-26T10:39:56.141979+0000","last_active":"2024-02-26T10:39:57.249016+0000","last_peered":"2024-02-26T10:39:57.249016+0000","last_clean":"2024-02-26T10:39:57.249016+0000","last_became_active":"2024-02-26T10:39:56.141657+0000","last_became_peered":"2024-02-26T10:39:56.141657+0000","last_unstale":"2024-02-26T10:39:57.249016+0000","last_undegraded":"2024-02-26T10:39:57.249016+0000","last_fullsized":"2024-02-26T10:39:57.249016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:01.709690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139623+0000","last_change":"2024-02-26T10:39:40.025212+0000","last_active":"2024-02-26T10:39:57.139623+0000","last_peered":"2024-02-26T10:39:57.139623+0000","last_clean":"2024-02-26T10:39:57.139623+0000","last_became_active":"2024-02-26T10:39:40.024642+0000","last_became_peered":"2024-02-26T10:39:40.024642+0000","last_unstale":"2024-02-26T10:39:57.139623+0000","last_undegraded":"2024-02-26T10:39:57.139623+0000","last_fullsized":"2024-02-26T10:39:57.139623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256056+0000","last_change":"2024-02-26T10:39:56.146889+0000","last_active":"2024-02-26T10:39:57.256056+0000","last_peered":"2024-02-26T10:39:57.256056+0000","last_clean":"2024-02-26T10:39:57.256056+0000","last_became_active":"2024-02-26T10:39:56.145742+0000","last_became_peered":"2024-02-26T10:39:56.145742+0000","last_unstale":"2024-02-26T10:39:57.256056+0000","last_undegraded":"2024-02-26T10:39:57.256056+0000","last_fullsized":"2024-02-26T10:39:57.256056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:46.512272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142239+0000","last_change":"2024-02-26T10:39:39.996527+0000","last_active":"2024-02-26T10:39:57.142239+0000","last_peered":"2024-02-26T10:39:57.142239+0000","last_clean":"2024-02-26T10:39:57.142239+0000","last_became_active":"2024-02-26T10:39:39.995687+0000","last_became_peered":"2024-02-26T10:39:39.995687+0000","last_unstale":"2024-02-26T10:39:57.142239+0000","last_undegraded":"2024-02-26T10:39:57.142239+0000","last_fullsized":"2024-02-26T10:39:57.142239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:29.964072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255484+0000","last_change":"2024-02-26T10:39:56.163773+0000","last_active":"2024-02-26T10:39:57.255484+0000","last_peered":"2024-02-26T10:39:57.255484+0000","last_clean":"2024-02-26T10:39:57.255484+0000","last_became_active":"2024-02-26T10:39:56.163436+0000","last_became_peered":"2024-02-26T10:39:56.163436+0000","last_unstale":"2024-02-26T10:39:57.255484+0000","last_undegraded":"2024-02-26T10:39:57.255484+0000","last_fullsized":"2024-02-26T10:39:57.255484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:09:37.102414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250921+0000","last_change":"2024-02-26T10:39:40.009578+0000","last_active":"2024-02-26T10:39:57.250921+0000","last_peered":"2024-02-26T10:39:57.250921+0000","last_clean":"2024-02-26T10:39:57.250921+0000","last_became_active":"2024-02-26T10:39:40.008770+0000","last_became_peered":"2024-02-26T10:39:40.008770+0000","last_unstale":"2024-02-26T10:39:57.250921+0000","last_undegraded":"2024-02-26T10:39:57.250921+0000","last_fullsized":"2024-02-26T10:39:57.250921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:48:42.311486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259802+0000","last_change":"2024-02-26T10:39:56.148486+0000","last_active":"2024-02-26T10:39:57.259802+0000","last_peered":"2024-02-26T10:39:57.259802+0000","last_clean":"2024-02-26T10:39:57.259802+0000","last_became_active":"2024-02-26T10:39:56.147714+0000","last_became_peered":"2024-02-26T10:39:56.147714+0000","last_unstale":"2024-02-26T10:39:57.259802+0000","last_undegraded":"2024-02-26T10:39:57.259802+0000","last_fullsized":"2024-02-26T10:39:57.259802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:09:37.102414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254189+0000","last_change":"2024-02-26T10:39:40.017208+0000","last_active":"2024-02-26T10:39:57.254189+0000","last_peered":"2024-02-26T10:39:57.254189+0000","last_clean":"2024-02-26T10:39:57.254189+0000","last_became_active":"2024-02-26T10:39:40.016350+0000","last_became_peered":"2024-02-26T10:39:40.016350+0000","last_unstale":"2024-02-26T10:39:57.254189+0000","last_undegraded":"2024-02-26T10:39:57.254189+0000","last_fullsized":"2024-02-26T10:39:57.254189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249756+0000","last_change":"2024-02-26T10:39:56.142505+0000","last_active":"2024-02-26T10:39:57.249756+0000","last_peered":"2024-02-26T10:39:57.249756+0000","last_clean":"2024-02-26T10:39:57.249756+0000","last_became_active":"2024-02-26T10:39:56.142202+0000","last_became_peered":"2024-02-26T10:39:56.142202+0000","last_unstale":"2024-02-26T10:39:57.249756+0000","last_undegraded":"2024-02-26T10:39:57.249756+0000","last_fullsized":"2024-02-26T10:39:57.249756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:59.513549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141660+0000","last_change":"2024-02-26T10:39:40.000691+0000","last_active":"2024-02-26T10:39:57.141660+0000","last_peered":"2024-02-26T10:39:57.141660+0000","last_clean":"2024-02-26T10:39:57.141660+0000","last_became_active":"2024-02-26T10:39:40.000419+0000","last_became_peered":"2024-02-26T10:39:40.000419+0000","last_unstale":"2024-02-26T10:39:57.141660+0000","last_undegraded":"2024-02-26T10:39:57.141660+0000","last_fullsized":"2024-02-26T10:39:57.141660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144703+0000","last_change":"2024-02-26T10:39:56.144006+0000","last_active":"2024-02-26T10:39:57.144703+0000","last_peered":"2024-02-26T10:39:57.144703+0000","last_clean":"2024-02-26T10:39:57.144703+0000","last_became_active":"2024-02-26T10:39:56.143570+0000","last_became_peered":"2024-02-26T10:39:56.143570+0000","last_unstale":"2024-02-26T10:39:57.144703+0000","last_undegraded":"2024-02-26T10:39:57.144703+0000","last_fullsized":"2024-02-26T10:39:57.144703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:15.643339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259084+0000","last_change":"2024-02-26T10:39:40.026044+0000","last_active":"2024-02-26T10:39:57.259084+0000","last_peered":"2024-02-26T10:39:57.259084+0000","last_clean":"2024-02-26T10:39:57.259084+0000","last_became_active":"2024-02-26T10:39:40.025716+0000","last_became_peered":"2024-02-26T10:39:40.025716+0000","last_unstale":"2024-02-26T10:39:57.259084+0000","last_undegraded":"2024-02-26T10:39:57.259084+0000","last_fullsized":"2024-02-26T10:39:57.259084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3913,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":769042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":179782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":179236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":180210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":182269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":180198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":181701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1028096,"data_stored":768896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:40:00.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:40:00.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.934+0000 7f1adb200640 1 Processor -- start 2024-02-26T10:40:00.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.934+0000 7f1adb200640 1 -- start start 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1adb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ad4071700 0x7f1ad4071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1adb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4155c00 con 0x7f1ad414f3d0 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1adb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4155ee0 con 0x7f1ad40c2870 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1adb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad406dd80 con 0x7f1ad4071700 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1ad3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1ad414f3d0 0x7f1ad406e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52970/0 (socket says 172.21.15.67:52970) 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1ad3e00640 1 -- 172.21.15.67:0/1717821767 learned_addr learned my addr 172.21.15.67:0/1717821767 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886460167 0 0) 0x7f1ad4155c00 con 0x7f1ad414f3d0 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.936+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ab8003520 con 0x7f1ad414f3d0 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1717821767 >> v2:172.21.15.82:3300/0 conn(0x7f1ad4071700 0x7f1ad4071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 757581426 0 0) 0x7f1ab8003520 con 0x7f1ad414f3d0 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 >> v2:172.21.15.82:3300/0 conn(0x7f1ad4071700 msgr2=0x7f1ad4071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 --2- 172.21.15.67:0/1717821767 >> v2:172.21.15.82:3300/0 conn(0x7f1ad4071700 0x7f1ad4071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 >> v1:172.21.15.82:6789/0 conn(0x7f1ad40c2870 legacy=0x7f1ad4155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad406dfd0 con 0x7f1ad414f3d0 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1ac4001670 con 0x7f1ad414f3d0 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 >> v1:172.21.15.67:6789/0 conn(0x7f1ad414f3d0 legacy=0x7f1ad406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ab80039b0 0x7f1ab8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab8004800 0x7f1ab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ab80053c0 0x7f1ab800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad406dd80 con 0x7f1ab80053c0 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab800e470 con 0x7f1ab8004800 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab800e670 con 0x7f1ab80039b0 2024-02-26T10:40:00.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.937+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ab80053c0 0x7f1ab800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1acb400640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab8004800 0x7f1ab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1ad9200640 1 -- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ab80039b0 msgr2=0x7f1ab8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ab80039b0 0x7f1ab8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1ad9200640 1 -- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab8004800 msgr2=0x7f1ab8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab8004800 0x7f1ab8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1ad9200640 1 -- 172.21.15.67:0/1717821767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8003520 con 0x7f1ab80053c0 2024-02-26T10:40:00.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.938+0000 7f1ad9200640 1 -- 172.21.15.67:0/1717821767 reap_dead start 2024-02-26T10:40:00.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.939+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ab80053c0 0x7f1ab800dbd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1acc007f60 tx=0x7f1ab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edf39e7a72ae392 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:00.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.939+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc007b30 con 0x7f1ab80053c0 2024-02-26T10:40:00.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.939+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc033600 con 0x7f1ab80053c0 2024-02-26T10:40:00.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.939+0000 7f1ad3400640 1 -- 172.21.15.67:0/1717821767 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0339d0 con 0x7f1ab80053c0 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.939+0000 7f1adb200640 1 -- 172.21.15.67:0/1717821767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ab80053c0 msgr2=0x7f1ab800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.939+0000 7f1adb200640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ab80053c0 0x7f1ab800dbd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1acc007f60 tx=0x7f1ab8005830 comp rx=0 tx=0).stop 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.940+0000 7f1adb200640 1 -- 172.21.15.67:0/1717821767 shutdown_connections 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.940+0000 7f1adb200640 1 --2- 172.21.15.67:0/1717821767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ab80053c0 0x7f1ab800dbd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.940+0000 7f1adb200640 1 -- 172.21.15.67:0/1717821767 >> 172.21.15.67:0/1717821767 conn(0x7f1ad40a84a0 msgr2=0x7f1ad40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.940+0000 7f1adb200640 1 -- 172.21.15.67:0/1717821767 shutdown_connections 2024-02-26T10:40:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.940+0000 7f1adb200640 1 -- 172.21.15.67:0/1717821767 wait complete. 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.941+0000 7f1adb200640 1 Processor -- start 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 -- start start 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab80053c0 0x7f1ad414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ad414b140 0x7f1ad40c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 0x7f1ad4143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c2cb0 con 0x7f1ad4143920 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c1810 con 0x7f1ab80053c0 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.942+0000 7f1adb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c2e30 con 0x7f1ad414b140 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 0x7f1ad4143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 0x7f1ad4143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54226/0 (socket says 172.21.15.67:54226) 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 -- 172.21.15.67:0/1230034240 learned_addr learned my addr 172.21.15.67:0/1230034240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1acb400640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ad414b140 0x7f1ad40c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad3e00640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab80053c0 0x7f1ad414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 -- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ad414b140 msgr2=0x7f1ad40c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ad414b140 0x7f1ad40c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 -- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab80053c0 msgr2=0x7f1ad414b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab80053c0 0x7f1ad414b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.943+0000 7f1ad9200640 1 -- 172.21.15.67:0/1230034240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad41eb0f0 con 0x7f1ad4143920 2024-02-26T10:40:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.944+0000 7f1ad9200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 0x7f1ad4143da0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c400 tx=0x7f1acc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.944+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc007b30 con 0x7f1ad4143920 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.944+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc046070 con 0x7f1ad4143920 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.944+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc032b60 con 0x7f1ad4143920 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.944+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad41eb3d0 con 0x7f1ad4143920 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.944+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad40c5720 con 0x7f1ad4143920 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.945+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad4074a30 con 0x7f1ad4143920 2024-02-26T10:40:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.946+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1acc03b040 con 0x7f1ad4143920 2024-02-26T10:40:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.947+0000 7f1ad0c00640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ab006b4d0 0x7f1ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.947+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1acc0ad340 con 0x7f1ad4143920 2024-02-26T10:40:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.947+0000 7f1ad3e00640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ab006b4d0 0x7f1ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.948+0000 7f1ad3e00640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ab006b4d0 0x7f1ab006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000fd0 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:00.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:00.951+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1acc07be50 con 0x7f1ad4143920 2024-02-26T10:40:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.146+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ad40c13f0 con 0x7f1ab006b4d0 2024-02-26T10:40:01.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.182+0000 7f1ad0c00640 1 -- 172.21.15.67:0/1230034240 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490424 (secure 0 0 0) 0x7f1ad40c13f0 con 0x7f1ab006b4d0 2024-02-26T10:40:01.188 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:40:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ab006b4d0 msgr2=0x7f1ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ab006b4d0 0x7f1ab006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000fd0 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 msgr2=0x7f1ad4143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 0x7f1ad4143da0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c400 tx=0x7f1acc004ab0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 shutdown_connections 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4143920 0x7f1ad4143da0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ad414b140 0x7f1ad40c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ab006b4d0 0x7f1ab006d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.188+0000 7f1adb200640 1 --2- 172.21.15.67:0/1230034240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ab80053c0 0x7f1ad414b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.189+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 >> 172.21.15.67:0/1230034240 conn(0x7f1ad40a84a0 msgr2=0x7f1ad414dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.189+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 shutdown_connections 2024-02-26T10:40:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.189+0000 7f1adb200640 1 -- 172.21.15.67:0/1230034240 wait complete. 2024-02-26T10:40:01.207 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-02-26T10:40:00.352902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":345,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220836,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753884,"statfs":{"total":773094113280,"available":772867977216,"internally_reserved":0,"allocated":4759552,"data_stored":2621334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002061"},"pg_stats":[{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250638+0000","last_change":"2024-02-26T10:39:56.155884+0000","last_active":"2024-02-26T10:39:57.250638+0000","last_peered":"2024-02-26T10:39:57.250638+0000","last_clean":"2024-02-26T10:39:57.250638+0000","last_became_active":"2024-02-26T10:39:56.155539+0000","last_became_peered":"2024-02-26T10:39:56.155539+0000","last_unstale":"2024-02-26T10:39:57.250638+0000","last_undegraded":"2024-02-26T10:39:57.250638+0000","last_fullsized":"2024-02-26T10:39:57.250638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:53.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256791+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:39:57.256791+0000","last_peered":"2024-02-26T10:39:57.256791+0000","last_clean":"2024-02-26T10:39:57.256791+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:39:57.256791+0000","last_undegraded":"2024-02-26T10:39:57.256791+0000","last_fullsized":"2024-02-26T10:39:57.256791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259164+0000","last_change":"2024-02-26T10:39:39.013277+0000","last_active":"2024-02-26T10:39:57.259164+0000","last_peered":"2024-02-26T10:39:57.259164+0000","last_clean":"2024-02-26T10:39:57.259164+0000","last_became_active":"2024-02-26T10:39:39.012073+0000","last_became_peered":"2024-02-26T10:39:39.012073+0000","last_unstale":"2024-02-26T10:39:57.259164+0000","last_undegraded":"2024-02-26T10:39:57.259164+0000","last_fullsized":"2024-02-26T10:39:57.259164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:44:44.421668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258900+0000","last_change":"2024-02-26T10:39:40.018706+0000","last_active":"2024-02-26T10:39:57.258900+0000","last_peered":"2024-02-26T10:39:57.258900+0000","last_clean":"2024-02-26T10:39:57.258900+0000","last_became_active":"2024-02-26T10:39:40.018417+0000","last_became_peered":"2024-02-26T10:39:40.018417+0000","last_unstale":"2024-02-26T10:39:57.258900+0000","last_undegraded":"2024-02-26T10:39:57.258900+0000","last_fullsized":"2024-02-26T10:39:57.258900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257645+0000","last_change":"2024-02-26T10:39:55.139751+0000","last_active":"2024-02-26T10:39:57.257645+0000","last_peered":"2024-02-26T10:39:57.257645+0000","last_clean":"2024-02-26T10:39:57.257645+0000","last_became_active":"2024-02-26T10:39:55.139085+0000","last_became_peered":"2024-02-26T10:39:55.139085+0000","last_unstale":"2024-02-26T10:39:57.257645+0000","last_undegraded":"2024-02-26T10:39:57.257645+0000","last_fullsized":"2024-02-26T10:39:57.257645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:26:00.601349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139003+0000","last_change":"2024-02-26T10:39:56.133408+0000","last_active":"2024-02-26T10:39:57.139003+0000","last_peered":"2024-02-26T10:39:57.139003+0000","last_clean":"2024-02-26T10:39:57.139003+0000","last_became_active":"2024-02-26T10:39:56.132956+0000","last_became_peered":"2024-02-26T10:39:56.132956+0000","last_unstale":"2024-02-26T10:39:57.139003+0000","last_undegraded":"2024-02-26T10:39:57.139003+0000","last_fullsized":"2024-02-26T10:39:57.139003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:29.273971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138744+0000","last_change":"2024-02-26T10:39:39.006896+0000","last_active":"2024-02-26T10:39:57.138744+0000","last_peered":"2024-02-26T10:39:57.138744+0000","last_clean":"2024-02-26T10:39:57.138744+0000","last_became_active":"2024-02-26T10:39:39.006604+0000","last_became_peered":"2024-02-26T10:39:39.006604+0000","last_unstale":"2024-02-26T10:39:57.138744+0000","last_undegraded":"2024-02-26T10:39:57.138744+0000","last_fullsized":"2024-02-26T10:39:57.138744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143093+0000","last_change":"2024-02-26T10:39:39.995364+0000","last_active":"2024-02-26T10:39:57.143093+0000","last_peered":"2024-02-26T10:39:57.143093+0000","last_clean":"2024-02-26T10:39:57.143093+0000","last_became_active":"2024-02-26T10:39:39.995077+0000","last_became_peered":"2024-02-26T10:39:39.995077+0000","last_unstale":"2024-02-26T10:39:57.143093+0000","last_undegraded":"2024-02-26T10:39:57.143093+0000","last_fullsized":"2024-02-26T10:39:57.143093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144237+0000","last_change":"2024-02-26T10:39:55.138429+0000","last_active":"2024-02-26T10:39:57.144237+0000","last_peered":"2024-02-26T10:39:57.144237+0000","last_clean":"2024-02-26T10:39:57.144237+0000","last_became_active":"2024-02-26T10:39:55.138123+0000","last_became_peered":"2024-02-26T10:39:55.138123+0000","last_unstale":"2024-02-26T10:39:57.144237+0000","last_undegraded":"2024-02-26T10:39:57.144237+0000","last_fullsized":"2024-02-26T10:39:57.144237+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:22.197650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139321+0000","last_change":"2024-02-26T10:39:56.131831+0000","last_active":"2024-02-26T10:39:57.139321+0000","last_peered":"2024-02-26T10:39:57.139321+0000","last_clean":"2024-02-26T10:39:57.139321+0000","last_became_active":"2024-02-26T10:39:56.131566+0000","last_became_peered":"2024-02-26T10:39:56.131566+0000","last_unstale":"2024-02-26T10:39:57.139321+0000","last_undegraded":"2024-02-26T10:39:57.139321+0000","last_fullsized":"2024-02-26T10:39:57.139321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:44:13.749545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143650+0000","last_change":"2024-02-26T10:39:38.999680+0000","last_active":"2024-02-26T10:39:57.143650+0000","last_peered":"2024-02-26T10:39:57.143650+0000","last_clean":"2024-02-26T10:39:57.143650+0000","last_became_active":"2024-02-26T10:39:38.999058+0000","last_became_peered":"2024-02-26T10:39:38.999058+0000","last_unstale":"2024-02-26T10:39:57.143650+0000","last_undegraded":"2024-02-26T10:39:57.143650+0000","last_fullsized":"2024-02-26T10:39:57.143650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:16:26.840527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138626+0000","last_change":"2024-02-26T10:39:39.997501+0000","last_active":"2024-02-26T10:39:57.138626+0000","last_peered":"2024-02-26T10:39:57.138626+0000","last_clean":"2024-02-26T10:39:57.138626+0000","last_became_active":"2024-02-26T10:39:39.996084+0000","last_became_peered":"2024-02-26T10:39:39.996084+0000","last_unstale":"2024-02-26T10:39:57.138626+0000","last_undegraded":"2024-02-26T10:39:57.138626+0000","last_fullsized":"2024-02-26T10:39:57.138626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257819+0000","last_change":"2024-02-26T10:39:55.147988+0000","last_active":"2024-02-26T10:39:57.257819+0000","last_peered":"2024-02-26T10:39:57.257819+0000","last_clean":"2024-02-26T10:39:57.257819+0000","last_became_active":"2024-02-26T10:39:55.147451+0000","last_became_peered":"2024-02-26T10:39:55.147451+0000","last_unstale":"2024-02-26T10:39:57.257819+0000","last_undegraded":"2024-02-26T10:39:57.257819+0000","last_fullsized":"2024-02-26T10:39:57.257819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:00.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143227+0000","last_change":"2024-02-26T10:39:56.140810+0000","last_active":"2024-02-26T10:39:57.143227+0000","last_peered":"2024-02-26T10:39:57.143227+0000","last_clean":"2024-02-26T10:39:57.143227+0000","last_became_active":"2024-02-26T10:39:56.140541+0000","last_became_peered":"2024-02-26T10:39:56.140541+0000","last_unstale":"2024-02-26T10:39:57.143227+0000","last_undegraded":"2024-02-26T10:39:57.143227+0000","last_fullsized":"2024-02-26T10:39:57.143227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:37.401643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250803+0000","last_change":"2024-02-26T10:39:39.013362+0000","last_active":"2024-02-26T10:39:57.250803+0000","last_peered":"2024-02-26T10:39:57.250803+0000","last_clean":"2024-02-26T10:39:57.250803+0000","last_became_active":"2024-02-26T10:39:39.012819+0000","last_became_peered":"2024-02-26T10:39:39.012819+0000","last_unstale":"2024-02-26T10:39:57.250803+0000","last_undegraded":"2024-02-26T10:39:57.250803+0000","last_fullsized":"2024-02-26T10:39:57.250803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249694+0000","last_change":"2024-02-26T10:39:40.011820+0000","last_active":"2024-02-26T10:39:57.249694+0000","last_peered":"2024-02-26T10:39:57.249694+0000","last_clean":"2024-02-26T10:39:57.249694+0000","last_became_active":"2024-02-26T10:39:40.011511+0000","last_became_peered":"2024-02-26T10:39:40.011511+0000","last_unstale":"2024-02-26T10:39:57.249694+0000","last_undegraded":"2024-02-26T10:39:57.249694+0000","last_fullsized":"2024-02-26T10:39:57.249694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249257+0000","last_change":"2024-02-26T10:39:55.137179+0000","last_active":"2024-02-26T10:39:57.249257+0000","last_peered":"2024-02-26T10:39:57.249257+0000","last_clean":"2024-02-26T10:39:57.249257+0000","last_became_active":"2024-02-26T10:39:55.136803+0000","last_became_peered":"2024-02-26T10:39:55.136803+0000","last_unstale":"2024-02-26T10:39:57.249257+0000","last_undegraded":"2024-02-26T10:39:57.249257+0000","last_fullsized":"2024-02-26T10:39:57.249257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:36.808744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252495+0000","last_change":"2024-02-26T10:39:56.146625+0000","last_active":"2024-02-26T10:39:57.252495+0000","last_peered":"2024-02-26T10:39:57.252495+0000","last_clean":"2024-02-26T10:39:57.252495+0000","last_became_active":"2024-02-26T10:39:56.145895+0000","last_became_peered":"2024-02-26T10:39:56.145895+0000","last_unstale":"2024-02-26T10:39:57.252495+0000","last_undegraded":"2024-02-26T10:39:57.252495+0000","last_fullsized":"2024-02-26T10:39:57.252495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:06.696664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250295+0000","last_change":"2024-02-26T10:39:39.003861+0000","last_active":"2024-02-26T10:39:57.250295+0000","last_peered":"2024-02-26T10:39:57.250295+0000","last_clean":"2024-02-26T10:39:57.250295+0000","last_became_active":"2024-02-26T10:39:39.003370+0000","last_became_peered":"2024-02-26T10:39:39.003370+0000","last_unstale":"2024-02-26T10:39:57.250295+0000","last_undegraded":"2024-02-26T10:39:57.250295+0000","last_fullsized":"2024-02-26T10:39:57.250295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259356+0000","last_change":"2024-02-26T10:39:40.008904+0000","last_active":"2024-02-26T10:39:57.259356+0000","last_peered":"2024-02-26T10:39:57.259356+0000","last_clean":"2024-02-26T10:39:57.259356+0000","last_became_active":"2024-02-26T10:39:40.008631+0000","last_became_peered":"2024-02-26T10:39:40.008631+0000","last_unstale":"2024-02-26T10:39:57.259356+0000","last_undegraded":"2024-02-26T10:39:57.259356+0000","last_fullsized":"2024-02-26T10:39:57.259356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250493+0000","last_change":"2024-02-26T10:39:55.147824+0000","last_active":"2024-02-26T10:39:57.250493+0000","last_peered":"2024-02-26T10:39:57.250493+0000","last_clean":"2024-02-26T10:39:57.250493+0000","last_became_active":"2024-02-26T10:39:55.147535+0000","last_became_peered":"2024-02-26T10:39:55.147535+0000","last_unstale":"2024-02-26T10:39:57.250493+0000","last_undegraded":"2024-02-26T10:39:57.250493+0000","last_fullsized":"2024-02-26T10:39:57.250493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:36.386825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259593+0000","last_change":"2024-02-26T10:39:56.139551+0000","last_active":"2024-02-26T10:39:57.259593+0000","last_peered":"2024-02-26T10:39:57.259593+0000","last_clean":"2024-02-26T10:39:57.259593+0000","last_became_active":"2024-02-26T10:39:56.139259+0000","last_became_peered":"2024-02-26T10:39:56.139259+0000","last_unstale":"2024-02-26T10:39:57.259593+0000","last_undegraded":"2024-02-26T10:39:57.259593+0000","last_fullsized":"2024-02-26T10:39:57.259593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:51:53.159078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251111+0000","last_change":"2024-02-26T10:39:38.996371+0000","last_active":"2024-02-26T10:39:57.251111+0000","last_peered":"2024-02-26T10:39:57.251111+0000","last_clean":"2024-02-26T10:39:57.251111+0000","last_became_active":"2024-02-26T10:39:38.996052+0000","last_became_peered":"2024-02-26T10:39:38.996052+0000","last_unstale":"2024-02-26T10:39:57.251111+0000","last_undegraded":"2024-02-26T10:39:57.251111+0000","last_fullsized":"2024-02-26T10:39:57.251111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:15:47.071582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249995+0000","last_change":"2024-02-26T10:39:40.009958+0000","last_active":"2024-02-26T10:39:57.249995+0000","last_peered":"2024-02-26T10:39:57.249995+0000","last_clean":"2024-02-26T10:39:57.249995+0000","last_became_active":"2024-02-26T10:39:40.009557+0000","last_became_peered":"2024-02-26T10:39:40.009557+0000","last_unstale":"2024-02-26T10:39:57.249995+0000","last_undegraded":"2024-02-26T10:39:57.249995+0000","last_fullsized":"2024-02-26T10:39:57.249995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:58.124347+0000","last_change":"2024-02-26T10:39:55.143200+0000","last_active":"2024-02-26T10:39:58.124347+0000","last_peered":"2024-02-26T10:39:58.124347+0000","last_clean":"2024-02-26T10:39:58.124347+0000","last_became_active":"2024-02-26T10:39:55.142844+0000","last_became_peered":"2024-02-26T10:39:55.142844+0000","last_unstale":"2024-02-26T10:39:58.124347+0000","last_undegraded":"2024-02-26T10:39:58.124347+0000","last_fullsized":"2024-02-26T10:39:58.124347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:50.940528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257616+0000","last_change":"2024-02-26T10:39:56.148312+0000","last_active":"2024-02-26T10:39:57.257616+0000","last_peered":"2024-02-26T10:39:57.257616+0000","last_clean":"2024-02-26T10:39:57.257616+0000","last_became_active":"2024-02-26T10:39:56.148002+0000","last_became_peered":"2024-02-26T10:39:56.148002+0000","last_unstale":"2024-02-26T10:39:57.257616+0000","last_undegraded":"2024-02-26T10:39:57.257616+0000","last_fullsized":"2024-02-26T10:39:57.257616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:29.247494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250768+0000","last_change":"2024-02-26T10:39:38.992861+0000","last_active":"2024-02-26T10:39:57.250768+0000","last_peered":"2024-02-26T10:39:57.250768+0000","last_clean":"2024-02-26T10:39:57.250768+0000","last_became_active":"2024-02-26T10:39:38.992258+0000","last_became_peered":"2024-02-26T10:39:38.992258+0000","last_unstale":"2024-02-26T10:39:57.250768+0000","last_undegraded":"2024-02-26T10:39:57.250768+0000","last_fullsized":"2024-02-26T10:39:57.250768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:59:38.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254948+0000","last_change":"2024-02-26T10:39:40.009603+0000","last_active":"2024-02-26T10:39:57.254948+0000","last_peered":"2024-02-26T10:39:57.254948+0000","last_clean":"2024-02-26T10:39:57.254948+0000","last_became_active":"2024-02-26T10:39:40.009327+0000","last_became_peered":"2024-02-26T10:39:40.009327+0000","last_unstale":"2024-02-26T10:39:57.254948+0000","last_undegraded":"2024-02-26T10:39:57.254948+0000","last_fullsized":"2024-02-26T10:39:57.254948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250518+0000","last_change":"2024-02-26T10:39:55.125334+0000","last_active":"2024-02-26T10:39:57.250518+0000","last_peered":"2024-02-26T10:39:57.250518+0000","last_clean":"2024-02-26T10:39:57.250518+0000","last_became_active":"2024-02-26T10:39:55.125040+0000","last_became_peered":"2024-02-26T10:39:55.125040+0000","last_unstale":"2024-02-26T10:39:57.250518+0000","last_undegraded":"2024-02-26T10:39:57.250518+0000","last_fullsized":"2024-02-26T10:39:57.250518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:17.565930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249624+0000","last_change":"2024-02-26T10:39:56.151375+0000","last_active":"2024-02-26T10:39:57.249624+0000","last_peered":"2024-02-26T10:39:57.249624+0000","last_clean":"2024-02-26T10:39:57.249624+0000","last_became_active":"2024-02-26T10:39:56.151080+0000","last_became_peered":"2024-02-26T10:39:56.151080+0000","last_unstale":"2024-02-26T10:39:57.249624+0000","last_undegraded":"2024-02-26T10:39:57.249624+0000","last_fullsized":"2024-02-26T10:39:57.249624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:10.708753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251574+0000","last_change":"2024-02-26T10:39:39.009948+0000","last_active":"2024-02-26T10:39:57.251574+0000","last_peered":"2024-02-26T10:39:57.251574+0000","last_clean":"2024-02-26T10:39:57.251574+0000","last_became_active":"2024-02-26T10:39:39.009640+0000","last_became_peered":"2024-02-26T10:39:39.009640+0000","last_unstale":"2024-02-26T10:39:57.251574+0000","last_undegraded":"2024-02-26T10:39:57.251574+0000","last_fullsized":"2024-02-26T10:39:57.251574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:47.192375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140884+0000","last_change":"2024-02-26T10:39:40.022558+0000","last_active":"2024-02-26T10:39:57.140884+0000","last_peered":"2024-02-26T10:39:57.140884+0000","last_clean":"2024-02-26T10:39:57.140884+0000","last_became_active":"2024-02-26T10:39:40.022316+0000","last_became_peered":"2024-02-26T10:39:40.022316+0000","last_unstale":"2024-02-26T10:39:57.140884+0000","last_undegraded":"2024-02-26T10:39:57.140884+0000","last_fullsized":"2024-02-26T10:39:57.140884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:11:06.614146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258080+0000","last_change":"2024-02-26T10:39:55.148037+0000","last_active":"2024-02-26T10:39:57.258080+0000","last_peered":"2024-02-26T10:39:57.258080+0000","last_clean":"2024-02-26T10:39:57.258080+0000","last_became_active":"2024-02-26T10:39:55.147501+0000","last_became_peered":"2024-02-26T10:39:55.147501+0000","last_unstale":"2024-02-26T10:39:57.258080+0000","last_undegraded":"2024-02-26T10:39:57.258080+0000","last_fullsized":"2024-02-26T10:39:57.258080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:51.837369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250733+0000","last_change":"2024-02-26T10:39:56.152002+0000","last_active":"2024-02-26T10:39:57.250733+0000","last_peered":"2024-02-26T10:39:57.250733+0000","last_clean":"2024-02-26T10:39:57.250733+0000","last_became_active":"2024-02-26T10:39:56.151739+0000","last_became_peered":"2024-02-26T10:39:56.151739+0000","last_unstale":"2024-02-26T10:39:57.250733+0000","last_undegraded":"2024-02-26T10:39:57.250733+0000","last_fullsized":"2024-02-26T10:39:57.250733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:15.643339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252645+0000","last_change":"2024-02-26T10:39:39.009824+0000","last_active":"2024-02-26T10:39:57.252645+0000","last_peered":"2024-02-26T10:39:57.252645+0000","last_clean":"2024-02-26T10:39:57.252645+0000","last_became_active":"2024-02-26T10:39:39.009559+0000","last_became_peered":"2024-02-26T10:39:39.009559+0000","last_unstale":"2024-02-26T10:39:57.252645+0000","last_undegraded":"2024-02-26T10:39:57.252645+0000","last_fullsized":"2024-02-26T10:39:57.252645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256931+0000","last_change":"2024-02-26T10:39:40.013051+0000","last_active":"2024-02-26T10:39:57.256931+0000","last_peered":"2024-02-26T10:39:57.256931+0000","last_clean":"2024-02-26T10:39:57.256931+0000","last_became_active":"2024-02-26T10:39:40.012739+0000","last_became_peered":"2024-02-26T10:39:40.012739+0000","last_unstale":"2024-02-26T10:39:57.256931+0000","last_undegraded":"2024-02-26T10:39:57.256931+0000","last_fullsized":"2024-02-26T10:39:57.256931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254892+0000","last_change":"2024-02-26T10:39:55.144555+0000","last_active":"2024-02-26T10:39:57.254892+0000","last_peered":"2024-02-26T10:39:57.254892+0000","last_clean":"2024-02-26T10:39:57.254892+0000","last_became_active":"2024-02-26T10:39:55.143431+0000","last_became_peered":"2024-02-26T10:39:55.143431+0000","last_unstale":"2024-02-26T10:39:57.254892+0000","last_undegraded":"2024-02-26T10:39:57.254892+0000","last_fullsized":"2024-02-26T10:39:57.254892+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:00.694872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139121+0000","last_change":"2024-02-26T10:39:56.133927+0000","last_active":"2024-02-26T10:39:57.139121+0000","last_peered":"2024-02-26T10:39:57.139121+0000","last_clean":"2024-02-26T10:39:57.139121+0000","last_became_active":"2024-02-26T10:39:56.133652+0000","last_became_peered":"2024-02-26T10:39:56.133652+0000","last_unstale":"2024-02-26T10:39:57.139121+0000","last_undegraded":"2024-02-26T10:39:57.139121+0000","last_fullsized":"2024-02-26T10:39:57.139121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:21:39.037942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143361+0000","last_change":"2024-02-26T10:39:39.000283+0000","last_active":"2024-02-26T10:39:57.143361+0000","last_peered":"2024-02-26T10:39:57.143361+0000","last_clean":"2024-02-26T10:39:57.143361+0000","last_became_active":"2024-02-26T10:39:38.999392+0000","last_became_peered":"2024-02-26T10:39:38.999392+0000","last_unstale":"2024-02-26T10:39:57.143361+0000","last_undegraded":"2024-02-26T10:39:57.143361+0000","last_fullsized":"2024-02-26T10:39:57.143361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:42.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140120+0000","last_change":"2024-02-26T10:39:40.001953+0000","last_active":"2024-02-26T10:39:57.140120+0000","last_peered":"2024-02-26T10:39:57.140120+0000","last_clean":"2024-02-26T10:39:57.140120+0000","last_became_active":"2024-02-26T10:39:40.001670+0000","last_became_peered":"2024-02-26T10:39:40.001670+0000","last_unstale":"2024-02-26T10:39:57.140120+0000","last_undegraded":"2024-02-26T10:39:57.140120+0000","last_fullsized":"2024-02-26T10:39:57.140120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:07.672401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139446+0000","last_change":"2024-02-26T10:39:55.147668+0000","last_active":"2024-02-26T10:39:57.139446+0000","last_peered":"2024-02-26T10:39:57.139446+0000","last_clean":"2024-02-26T10:39:57.139446+0000","last_became_active":"2024-02-26T10:39:55.147402+0000","last_became_peered":"2024-02-26T10:39:55.147402+0000","last_unstale":"2024-02-26T10:39:57.139446+0000","last_undegraded":"2024-02-26T10:39:57.139446+0000","last_fullsized":"2024-02-26T10:39:57.139446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:05.681846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249175+0000","last_change":"2024-02-26T10:39:56.151212+0000","last_active":"2024-02-26T10:39:57.249175+0000","last_peered":"2024-02-26T10:39:57.249175+0000","last_clean":"2024-02-26T10:39:57.249175+0000","last_became_active":"2024-02-26T10:39:56.150916+0000","last_became_peered":"2024-02-26T10:39:56.150916+0000","last_unstale":"2024-02-26T10:39:57.249175+0000","last_undegraded":"2024-02-26T10:39:57.249175+0000","last_fullsized":"2024-02-26T10:39:57.249175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:26:01.616167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259715+0000","last_change":"2024-02-26T10:39:39.011100+0000","last_active":"2024-02-26T10:39:57.259715+0000","last_peered":"2024-02-26T10:39:57.259715+0000","last_clean":"2024-02-26T10:39:57.259715+0000","last_became_active":"2024-02-26T10:39:39.010704+0000","last_became_peered":"2024-02-26T10:39:39.010704+0000","last_unstale":"2024-02-26T10:39:57.259715+0000","last_undegraded":"2024-02-26T10:39:57.259715+0000","last_fullsized":"2024-02-26T10:39:57.259715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142379+0000","last_change":"2024-02-26T10:39:40.002307+0000","last_active":"2024-02-26T10:39:57.142379+0000","last_peered":"2024-02-26T10:39:57.142379+0000","last_clean":"2024-02-26T10:39:57.142379+0000","last_became_active":"2024-02-26T10:39:40.001899+0000","last_became_peered":"2024-02-26T10:39:40.001899+0000","last_unstale":"2024-02-26T10:39:57.142379+0000","last_undegraded":"2024-02-26T10:39:57.142379+0000","last_fullsized":"2024-02-26T10:39:57.142379+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:39:23.314983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256394+0000","last_change":"2024-02-26T10:39:55.147262+0000","last_active":"2024-02-26T10:39:57.256394+0000","last_peered":"2024-02-26T10:39:57.256394+0000","last_clean":"2024-02-26T10:39:57.256394+0000","last_became_active":"2024-02-26T10:39:55.146952+0000","last_became_peered":"2024-02-26T10:39:55.146952+0000","last_unstale":"2024-02-26T10:39:57.256394+0000","last_undegraded":"2024-02-26T10:39:57.256394+0000","last_fullsized":"2024-02-26T10:39:57.256394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:49.472864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139544+0000","last_change":"2024-02-26T10:39:56.143318+0000","last_active":"2024-02-26T10:39:57.139544+0000","last_peered":"2024-02-26T10:39:57.139544+0000","last_clean":"2024-02-26T10:39:57.139544+0000","last_became_active":"2024-02-26T10:39:56.142848+0000","last_became_peered":"2024-02-26T10:39:56.142848+0000","last_unstale":"2024-02-26T10:39:57.139544+0000","last_undegraded":"2024-02-26T10:39:57.139544+0000","last_fullsized":"2024-02-26T10:39:57.139544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:18:44.657930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249952+0000","last_change":"2024-02-26T10:39:38.992320+0000","last_active":"2024-02-26T10:39:57.249952+0000","last_peered":"2024-02-26T10:39:57.249952+0000","last_clean":"2024-02-26T10:39:57.249952+0000","last_became_active":"2024-02-26T10:39:38.991970+0000","last_became_peered":"2024-02-26T10:39:38.991970+0000","last_unstale":"2024-02-26T10:39:57.249952+0000","last_undegraded":"2024-02-26T10:39:57.249952+0000","last_fullsized":"2024-02-26T10:39:57.249952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:22:50.405236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140449+0000","last_change":"2024-02-26T10:39:39.997097+0000","last_active":"2024-02-26T10:39:57.140449+0000","last_peered":"2024-02-26T10:39:57.140449+0000","last_clean":"2024-02-26T10:39:57.140449+0000","last_became_active":"2024-02-26T10:39:39.996720+0000","last_became_peered":"2024-02-26T10:39:39.996720+0000","last_unstale":"2024-02-26T10:39:57.140449+0000","last_undegraded":"2024-02-26T10:39:57.140449+0000","last_fullsized":"2024-02-26T10:39:57.140449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:12.318898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249123+0000","last_change":"2024-02-26T10:39:55.145833+0000","last_active":"2024-02-26T10:39:57.249123+0000","last_peered":"2024-02-26T10:39:57.249123+0000","last_clean":"2024-02-26T10:39:57.249123+0000","last_became_active":"2024-02-26T10:39:55.145534+0000","last_became_peered":"2024-02-26T10:39:55.145534+0000","last_unstale":"2024-02-26T10:39:57.249123+0000","last_undegraded":"2024-02-26T10:39:57.249123+0000","last_fullsized":"2024-02-26T10:39:57.249123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:11.096461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250250+0000","last_change":"2024-02-26T10:39:56.138607+0000","last_active":"2024-02-26T10:39:57.250250+0000","last_peered":"2024-02-26T10:39:57.250250+0000","last_clean":"2024-02-26T10:39:57.250250+0000","last_became_active":"2024-02-26T10:39:56.138370+0000","last_became_peered":"2024-02-26T10:39:56.138370+0000","last_unstale":"2024-02-26T10:39:57.250250+0000","last_undegraded":"2024-02-26T10:39:57.250250+0000","last_fullsized":"2024-02-26T10:39:57.250250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:46.512272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144723+0000","last_change":"2024-02-26T10:39:39.005544+0000","last_active":"2024-02-26T10:39:57.144723+0000","last_peered":"2024-02-26T10:39:57.144723+0000","last_clean":"2024-02-26T10:39:57.144723+0000","last_became_active":"2024-02-26T10:39:39.005194+0000","last_became_peered":"2024-02-26T10:39:39.005194+0000","last_unstale":"2024-02-26T10:39:57.144723+0000","last_undegraded":"2024-02-26T10:39:57.144723+0000","last_fullsized":"2024-02-26T10:39:57.144723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:51:47.714809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140986+0000","last_change":"2024-02-26T10:39:40.014954+0000","last_active":"2024-02-26T10:39:57.140986+0000","last_peered":"2024-02-26T10:39:57.140986+0000","last_clean":"2024-02-26T10:39:57.140986+0000","last_became_active":"2024-02-26T10:39:40.014657+0000","last_became_peered":"2024-02-26T10:39:40.014657+0000","last_unstale":"2024-02-26T10:39:57.140986+0000","last_undegraded":"2024-02-26T10:39:57.140986+0000","last_fullsized":"2024-02-26T10:39:57.140986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:38:13.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144920+0000","last_change":"2024-02-26T10:39:55.137487+0000","last_active":"2024-02-26T10:39:57.144920+0000","last_peered":"2024-02-26T10:39:57.144920+0000","last_clean":"2024-02-26T10:39:57.144920+0000","last_became_active":"2024-02-26T10:39:55.135747+0000","last_became_peered":"2024-02-26T10:39:55.135747+0000","last_unstale":"2024-02-26T10:39:57.144920+0000","last_undegraded":"2024-02-26T10:39:57.144920+0000","last_fullsized":"2024-02-26T10:39:57.144920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:41.302220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140800+0000","last_change":"2024-02-26T10:39:56.147247+0000","last_active":"2024-02-26T10:39:57.140800+0000","last_peered":"2024-02-26T10:39:57.140800+0000","last_clean":"2024-02-26T10:39:57.140800+0000","last_became_active":"2024-02-26T10:39:56.146951+0000","last_became_peered":"2024-02-26T10:39:56.146951+0000","last_unstale":"2024-02-26T10:39:57.140800+0000","last_undegraded":"2024-02-26T10:39:57.140800+0000","last_fullsized":"2024-02-26T10:39:57.140800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:40:06.774429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145157+0000","last_change":"2024-02-26T10:39:38.997283+0000","last_active":"2024-02-26T10:39:57.145157+0000","last_peered":"2024-02-26T10:39:57.145157+0000","last_clean":"2024-02-26T10:39:57.145157+0000","last_became_active":"2024-02-26T10:39:38.996974+0000","last_became_peered":"2024-02-26T10:39:38.996974+0000","last_unstale":"2024-02-26T10:39:57.145157+0000","last_undegraded":"2024-02-26T10:39:57.145157+0000","last_fullsized":"2024-02-26T10:39:57.145157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:48.227511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250972+0000","last_change":"2024-02-26T10:39:40.005719+0000","last_active":"2024-02-26T10:39:57.250972+0000","last_peered":"2024-02-26T10:39:57.250972+0000","last_clean":"2024-02-26T10:39:57.250972+0000","last_became_active":"2024-02-26T10:39:40.005399+0000","last_became_peered":"2024-02-26T10:39:40.005399+0000","last_unstale":"2024-02-26T10:39:57.250972+0000","last_undegraded":"2024-02-26T10:39:57.250972+0000","last_fullsized":"2024-02-26T10:39:57.250972+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:32:40.993108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140902+0000","last_change":"2024-02-26T10:39:55.142603+0000","last_active":"2024-02-26T10:39:57.140902+0000","last_peered":"2024-02-26T10:39:57.140902+0000","last_clean":"2024-02-26T10:39:57.140902+0000","last_became_active":"2024-02-26T10:39:55.142285+0000","last_became_peered":"2024-02-26T10:39:55.142285+0000","last_unstale":"2024-02-26T10:39:57.140902+0000","last_undegraded":"2024-02-26T10:39:57.140902+0000","last_fullsized":"2024-02-26T10:39:57.140902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:56.179993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258612+0000","last_change":"2024-02-26T10:39:56.142389+0000","last_active":"2024-02-26T10:39:57.258612+0000","last_peered":"2024-02-26T10:39:57.258612+0000","last_clean":"2024-02-26T10:39:57.258612+0000","last_became_active":"2024-02-26T10:39:56.141659+0000","last_became_peered":"2024-02-26T10:39:56.141659+0000","last_unstale":"2024-02-26T10:39:57.258612+0000","last_undegraded":"2024-02-26T10:39:57.258612+0000","last_fullsized":"2024-02-26T10:39:57.258612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:37.401643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145227+0000","last_change":"2024-02-26T10:39:39.008983+0000","last_active":"2024-02-26T10:39:57.145227+0000","last_peered":"2024-02-26T10:39:57.145227+0000","last_clean":"2024-02-26T10:39:57.145227+0000","last_became_active":"2024-02-26T10:39:39.008687+0000","last_became_peered":"2024-02-26T10:39:39.008687+0000","last_unstale":"2024-02-26T10:39:57.145227+0000","last_undegraded":"2024-02-26T10:39:57.145227+0000","last_fullsized":"2024-02-26T10:39:57.145227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:08:52.607215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254711+0000","last_change":"2024-02-26T10:39:40.017294+0000","last_active":"2024-02-26T10:39:57.254711+0000","last_peered":"2024-02-26T10:39:57.254711+0000","last_clean":"2024-02-26T10:39:57.254711+0000","last_became_active":"2024-02-26T10:39:40.016268+0000","last_became_peered":"2024-02-26T10:39:40.016268+0000","last_unstale":"2024-02-26T10:39:57.254711+0000","last_undegraded":"2024-02-26T10:39:57.254711+0000","last_fullsized":"2024-02-26T10:39:57.254711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140639+0000","last_change":"2024-02-26T10:39:55.142899+0000","last_active":"2024-02-26T10:39:57.140639+0000","last_peered":"2024-02-26T10:39:57.140639+0000","last_clean":"2024-02-26T10:39:57.140639+0000","last_became_active":"2024-02-26T10:39:55.142393+0000","last_became_peered":"2024-02-26T10:39:55.142393+0000","last_unstale":"2024-02-26T10:39:57.140639+0000","last_undegraded":"2024-02-26T10:39:57.140639+0000","last_fullsized":"2024-02-26T10:39:57.140639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:31:53.873062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258250+0000","last_change":"2024-02-26T10:39:56.138495+0000","last_active":"2024-02-26T10:39:57.258250+0000","last_peered":"2024-02-26T10:39:57.258250+0000","last_clean":"2024-02-26T10:39:57.258250+0000","last_became_active":"2024-02-26T10:39:56.138180+0000","last_became_peered":"2024-02-26T10:39:56.138180+0000","last_unstale":"2024-02-26T10:39:57.258250+0000","last_undegraded":"2024-02-26T10:39:57.258250+0000","last_fullsized":"2024-02-26T10:39:57.258250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:45.443279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251494+0000","last_change":"2024-02-26T10:39:39.001118+0000","last_active":"2024-02-26T10:39:57.251494+0000","last_peered":"2024-02-26T10:39:57.251494+0000","last_clean":"2024-02-26T10:39:57.251494+0000","last_became_active":"2024-02-26T10:39:39.000747+0000","last_became_peered":"2024-02-26T10:39:39.000747+0000","last_unstale":"2024-02-26T10:39:57.251494+0000","last_undegraded":"2024-02-26T10:39:57.251494+0000","last_fullsized":"2024-02-26T10:39:57.251494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:54.239568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145051+0000","last_change":"2024-02-26T10:39:40.026290+0000","last_active":"2024-02-26T10:39:57.145051+0000","last_peered":"2024-02-26T10:39:57.145051+0000","last_clean":"2024-02-26T10:39:57.145051+0000","last_became_active":"2024-02-26T10:39:40.026004+0000","last_became_peered":"2024-02-26T10:39:40.026004+0000","last_unstale":"2024-02-26T10:39:57.145051+0000","last_undegraded":"2024-02-26T10:39:57.145051+0000","last_fullsized":"2024-02-26T10:39:57.145051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:47.508007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254052+0000","last_change":"2024-02-26T10:39:55.142017+0000","last_active":"2024-02-26T10:39:57.254052+0000","last_peered":"2024-02-26T10:39:57.254052+0000","last_clean":"2024-02-26T10:39:57.254052+0000","last_became_active":"2024-02-26T10:39:55.140751+0000","last_became_peered":"2024-02-26T10:39:55.140751+0000","last_unstale":"2024-02-26T10:39:57.254052+0000","last_undegraded":"2024-02-26T10:39:57.254052+0000","last_fullsized":"2024-02-26T10:39:57.254052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:49.472864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250177+0000","last_change":"2024-02-26T10:39:56.155913+0000","last_active":"2024-02-26T10:39:57.250177+0000","last_peered":"2024-02-26T10:39:57.250177+0000","last_clean":"2024-02-26T10:39:57.250177+0000","last_became_active":"2024-02-26T10:39:56.155200+0000","last_became_peered":"2024-02-26T10:39:56.155200+0000","last_unstale":"2024-02-26T10:39:57.250177+0000","last_undegraded":"2024-02-26T10:39:57.250177+0000","last_fullsized":"2024-02-26T10:39:57.250177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:37:06.696664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138720+0000","last_change":"2024-02-26T10:39:39.002498+0000","last_active":"2024-02-26T10:39:57.138720+0000","last_peered":"2024-02-26T10:39:57.138720+0000","last_clean":"2024-02-26T10:39:57.138720+0000","last_became_active":"2024-02-26T10:39:39.002185+0000","last_became_peered":"2024-02-26T10:39:39.002185+0000","last_unstale":"2024-02-26T10:39:57.138720+0000","last_undegraded":"2024-02-26T10:39:57.138720+0000","last_fullsized":"2024-02-26T10:39:57.138720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:18.706456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256484+0000","last_change":"2024-02-26T10:39:40.008604+0000","last_active":"2024-02-26T10:39:57.256484+0000","last_peered":"2024-02-26T10:39:57.256484+0000","last_clean":"2024-02-26T10:39:57.256484+0000","last_became_active":"2024-02-26T10:39:40.008244+0000","last_became_peered":"2024-02-26T10:39:40.008244+0000","last_unstale":"2024-02-26T10:39:57.256484+0000","last_undegraded":"2024-02-26T10:39:57.256484+0000","last_fullsized":"2024-02-26T10:39:57.256484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:48.208053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257141+0000","last_change":"2024-02-26T10:39:55.146646+0000","last_active":"2024-02-26T10:39:57.257141+0000","last_peered":"2024-02-26T10:39:57.257141+0000","last_clean":"2024-02-26T10:39:57.257141+0000","last_became_active":"2024-02-26T10:39:55.146149+0000","last_became_peered":"2024-02-26T10:39:55.146149+0000","last_unstale":"2024-02-26T10:39:57.257141+0000","last_undegraded":"2024-02-26T10:39:57.257141+0000","last_fullsized":"2024-02-26T10:39:57.257141+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:26:00.601349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258945+0000","last_change":"2024-02-26T10:39:56.138500+0000","last_active":"2024-02-26T10:39:57.258945+0000","last_peered":"2024-02-26T10:39:57.258945+0000","last_clean":"2024-02-26T10:39:57.258945+0000","last_became_active":"2024-02-26T10:39:56.138157+0000","last_became_peered":"2024-02-26T10:39:56.138157+0000","last_unstale":"2024-02-26T10:39:57.258945+0000","last_undegraded":"2024-02-26T10:39:57.258945+0000","last_fullsized":"2024-02-26T10:39:57.258945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:44:34.878600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139928+0000","last_change":"2024-02-26T10:39:39.008406+0000","last_active":"2024-02-26T10:39:57.139928+0000","last_peered":"2024-02-26T10:39:57.139928+0000","last_clean":"2024-02-26T10:39:57.139928+0000","last_became_active":"2024-02-26T10:39:39.008086+0000","last_became_peered":"2024-02-26T10:39:39.008086+0000","last_unstale":"2024-02-26T10:39:57.139928+0000","last_undegraded":"2024-02-26T10:39:57.139928+0000","last_fullsized":"2024-02-26T10:39:57.139928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:25:46.492328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255887+0000","last_change":"2024-02-26T10:39:40.008946+0000","last_active":"2024-02-26T10:39:57.255887+0000","last_peered":"2024-02-26T10:39:57.255887+0000","last_clean":"2024-02-26T10:39:57.255887+0000","last_became_active":"2024-02-26T10:39:40.008676+0000","last_became_peered":"2024-02-26T10:39:40.008676+0000","last_unstale":"2024-02-26T10:39:57.255887+0000","last_undegraded":"2024-02-26T10:39:57.255887+0000","last_fullsized":"2024-02-26T10:39:57.255887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139206+0000","last_change":"2024-02-26T10:39:55.144193+0000","last_active":"2024-02-26T10:39:57.139206+0000","last_peered":"2024-02-26T10:39:57.139206+0000","last_clean":"2024-02-26T10:39:57.139206+0000","last_became_active":"2024-02-26T10:39:55.143853+0000","last_became_peered":"2024-02-26T10:39:55.143853+0000","last_unstale":"2024-02-26T10:39:57.139206+0000","last_undegraded":"2024-02-26T10:39:57.139206+0000","last_fullsized":"2024-02-26T10:39:57.139206+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:13.048005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256470+0000","last_change":"2024-02-26T10:39:56.145501+0000","last_active":"2024-02-26T10:39:57.256470+0000","last_peered":"2024-02-26T10:39:57.256470+0000","last_clean":"2024-02-26T10:39:57.256470+0000","last_became_active":"2024-02-26T10:39:56.145170+0000","last_became_peered":"2024-02-26T10:39:56.145170+0000","last_unstale":"2024-02-26T10:39:57.256470+0000","last_undegraded":"2024-02-26T10:39:57.256470+0000","last_fullsized":"2024-02-26T10:39:57.256470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:20.079475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250686+0000","last_change":"2024-02-26T10:39:39.000729+0000","last_active":"2024-02-26T10:39:57.250686+0000","last_peered":"2024-02-26T10:39:57.250686+0000","last_clean":"2024-02-26T10:39:57.250686+0000","last_became_active":"2024-02-26T10:39:38.999811+0000","last_became_peered":"2024-02-26T10:39:38.999811+0000","last_unstale":"2024-02-26T10:39:57.250686+0000","last_undegraded":"2024-02-26T10:39:57.250686+0000","last_fullsized":"2024-02-26T10:39:57.250686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139997+0000","last_change":"2024-02-26T10:39:40.006750+0000","last_active":"2024-02-26T10:39:57.139997+0000","last_peered":"2024-02-26T10:39:57.139997+0000","last_clean":"2024-02-26T10:39:57.139997+0000","last_became_active":"2024-02-26T10:39:40.005913+0000","last_became_peered":"2024-02-26T10:39:40.005913+0000","last_unstale":"2024-02-26T10:39:57.139997+0000","last_undegraded":"2024-02-26T10:39:57.139997+0000","last_fullsized":"2024-02-26T10:39:57.139997+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249263+0000","last_change":"2024-02-26T10:39:55.127857+0000","last_active":"2024-02-26T10:39:57.249263+0000","last_peered":"2024-02-26T10:39:57.249263+0000","last_clean":"2024-02-26T10:39:57.249263+0000","last_became_active":"2024-02-26T10:39:55.127539+0000","last_became_peered":"2024-02-26T10:39:55.127539+0000","last_unstale":"2024-02-26T10:39:57.249263+0000","last_undegraded":"2024-02-26T10:39:57.249263+0000","last_fullsized":"2024-02-26T10:39:57.249263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:41.302220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138635+0000","last_change":"2024-02-26T10:39:56.146679+0000","last_active":"2024-02-26T10:39:57.138635+0000","last_peered":"2024-02-26T10:39:57.138635+0000","last_clean":"2024-02-26T10:39:57.138635+0000","last_became_active":"2024-02-26T10:39:56.145640+0000","last_became_peered":"2024-02-26T10:39:56.145640+0000","last_unstale":"2024-02-26T10:39:57.138635+0000","last_undegraded":"2024-02-26T10:39:57.138635+0000","last_fullsized":"2024-02-26T10:39:57.138635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:03:13.404306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144281+0000","last_change":"2024-02-26T10:39:39.010735+0000","last_active":"2024-02-26T10:39:57.144281+0000","last_peered":"2024-02-26T10:39:57.144281+0000","last_clean":"2024-02-26T10:39:57.144281+0000","last_became_active":"2024-02-26T10:39:39.010394+0000","last_became_peered":"2024-02-26T10:39:39.010394+0000","last_unstale":"2024-02-26T10:39:57.144281+0000","last_undegraded":"2024-02-26T10:39:57.144281+0000","last_fullsized":"2024-02-26T10:39:57.144281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:41:16.074102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250575+0000","last_change":"2024-02-26T10:39:40.018996+0000","last_active":"2024-02-26T10:39:57.250575+0000","last_peered":"2024-02-26T10:39:57.250575+0000","last_clean":"2024-02-26T10:39:57.250575+0000","last_became_active":"2024-02-26T10:39:40.018734+0000","last_became_peered":"2024-02-26T10:39:40.018734+0000","last_unstale":"2024-02-26T10:39:57.250575+0000","last_undegraded":"2024-02-26T10:39:57.250575+0000","last_fullsized":"2024-02-26T10:39:57.250575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:26.470295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139109+0000","last_change":"2024-02-26T10:39:55.144297+0000","last_active":"2024-02-26T10:39:57.139109+0000","last_peered":"2024-02-26T10:39:57.139109+0000","last_clean":"2024-02-26T10:39:57.139109+0000","last_became_active":"2024-02-26T10:39:55.143891+0000","last_became_peered":"2024-02-26T10:39:55.143891+0000","last_unstale":"2024-02-26T10:39:57.139109+0000","last_undegraded":"2024-02-26T10:39:57.139109+0000","last_fullsized":"2024-02-26T10:39:57.139109+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:52.910544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.260580+0000","last_change":"2024-02-26T10:39:56.149171+0000","last_active":"2024-02-26T10:39:57.260580+0000","last_peered":"2024-02-26T10:39:57.260580+0000","last_clean":"2024-02-26T10:39:57.260580+0000","last_became_active":"2024-02-26T10:39:56.148052+0000","last_became_peered":"2024-02-26T10:39:56.148052+0000","last_unstale":"2024-02-26T10:39:57.260580+0000","last_undegraded":"2024-02-26T10:39:57.260580+0000","last_fullsized":"2024-02-26T10:39:57.260580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:08:19.286446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"34'8","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141091+0000","last_change":"2024-02-26T10:39:38.999382+0000","last_active":"2024-02-26T10:39:57.141091+0000","last_peered":"2024-02-26T10:39:57.141091+0000","last_clean":"2024-02-26T10:39:57.141091+0000","last_became_active":"2024-02-26T10:39:38.999045+0000","last_became_peered":"2024-02-26T10:39:38.999045+0000","last_unstale":"2024-02-26T10:39:57.141091+0000","last_undegraded":"2024-02-26T10:39:57.141091+0000","last_fullsized":"2024-02-26T10:39:57.141091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:36.656258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3115,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253980+0000","last_change":"2024-02-26T10:39:40.009580+0000","last_active":"2024-02-26T10:39:57.253980+0000","last_peered":"2024-02-26T10:39:57.253980+0000","last_clean":"2024-02-26T10:39:57.253980+0000","last_became_active":"2024-02-26T10:39:40.009277+0000","last_became_peered":"2024-02-26T10:39:40.009277+0000","last_unstale":"2024-02-26T10:39:57.253980+0000","last_undegraded":"2024-02-26T10:39:57.253980+0000","last_fullsized":"2024-02-26T10:39:57.253980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:21.692618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250463+0000","last_change":"2024-02-26T10:39:55.143834+0000","last_active":"2024-02-26T10:39:57.250463+0000","last_peered":"2024-02-26T10:39:57.250463+0000","last_clean":"2024-02-26T10:39:57.250463+0000","last_became_active":"2024-02-26T10:39:55.143492+0000","last_became_peered":"2024-02-26T10:39:55.143492+0000","last_unstale":"2024-02-26T10:39:57.250463+0000","last_undegraded":"2024-02-26T10:39:57.250463+0000","last_fullsized":"2024-02-26T10:39:57.250463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:19.434585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144795+0000","last_change":"2024-02-26T10:39:56.159534+0000","last_active":"2024-02-26T10:39:57.144795+0000","last_peered":"2024-02-26T10:39:57.144795+0000","last_clean":"2024-02-26T10:39:57.144795+0000","last_became_active":"2024-02-26T10:39:56.159278+0000","last_became_peered":"2024-02-26T10:39:56.159278+0000","last_unstale":"2024-02-26T10:39:57.144795+0000","last_undegraded":"2024-02-26T10:39:57.144795+0000","last_fullsized":"2024-02-26T10:39:57.144795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:02.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254268+0000","last_change":"2024-02-26T10:39:39.010001+0000","last_active":"2024-02-26T10:39:57.254268+0000","last_peered":"2024-02-26T10:39:57.254268+0000","last_clean":"2024-02-26T10:39:57.254268+0000","last_became_active":"2024-02-26T10:39:39.009754+0000","last_became_peered":"2024-02-26T10:39:39.009754+0000","last_unstale":"2024-02-26T10:39:57.254268+0000","last_undegraded":"2024-02-26T10:39:57.254268+0000","last_fullsized":"2024-02-26T10:39:57.254268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:12.786080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140949+0000","last_change":"2024-02-26T10:39:40.006537+0000","last_active":"2024-02-26T10:39:57.140949+0000","last_peered":"2024-02-26T10:39:57.140949+0000","last_clean":"2024-02-26T10:39:57.140949+0000","last_became_active":"2024-02-26T10:39:40.006231+0000","last_became_peered":"2024-02-26T10:39:40.006231+0000","last_unstale":"2024-02-26T10:39:57.140949+0000","last_undegraded":"2024-02-26T10:39:57.140949+0000","last_fullsized":"2024-02-26T10:39:57.140949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145891+0000","last_change":"2024-02-26T10:39:55.136727+0000","last_active":"2024-02-26T10:39:57.145891+0000","last_peered":"2024-02-26T10:39:57.145891+0000","last_clean":"2024-02-26T10:39:57.145891+0000","last_became_active":"2024-02-26T10:39:55.136446+0000","last_became_peered":"2024-02-26T10:39:55.136446+0000","last_unstale":"2024-02-26T10:39:57.145891+0000","last_undegraded":"2024-02-26T10:39:57.145891+0000","last_fullsized":"2024-02-26T10:39:57.145891+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:59:11.096461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257158+0000","last_change":"2024-02-26T10:39:56.151308+0000","last_active":"2024-02-26T10:39:57.257158+0000","last_peered":"2024-02-26T10:39:57.257158+0000","last_clean":"2024-02-26T10:39:57.257158+0000","last_became_active":"2024-02-26T10:39:56.150991+0000","last_became_peered":"2024-02-26T10:39:56.150991+0000","last_unstale":"2024-02-26T10:39:57.257158+0000","last_undegraded":"2024-02-26T10:39:57.257158+0000","last_fullsized":"2024-02-26T10:39:57.257158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:53.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250864+0000","last_change":"2024-02-26T10:39:39.004487+0000","last_active":"2024-02-26T10:39:57.250864+0000","last_peered":"2024-02-26T10:39:57.250864+0000","last_clean":"2024-02-26T10:39:57.250864+0000","last_became_active":"2024-02-26T10:39:39.004135+0000","last_became_peered":"2024-02-26T10:39:39.004135+0000","last_unstale":"2024-02-26T10:39:57.250864+0000","last_undegraded":"2024-02-26T10:39:57.250864+0000","last_fullsized":"2024-02-26T10:39:57.250864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:34:34.876134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141998+0000","last_change":"2024-02-26T10:39:40.006041+0000","last_active":"2024-02-26T10:39:57.141998+0000","last_peered":"2024-02-26T10:39:57.141998+0000","last_clean":"2024-02-26T10:39:57.141998+0000","last_became_active":"2024-02-26T10:39:40.002138+0000","last_became_peered":"2024-02-26T10:39:40.002138+0000","last_unstale":"2024-02-26T10:39:57.141998+0000","last_undegraded":"2024-02-26T10:39:57.141998+0000","last_fullsized":"2024-02-26T10:39:57.141998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:38.442199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141190+0000","last_change":"2024-02-26T10:39:55.139164+0000","last_active":"2024-02-26T10:39:57.141190+0000","last_peered":"2024-02-26T10:39:57.141190+0000","last_clean":"2024-02-26T10:39:57.141190+0000","last_became_active":"2024-02-26T10:39:55.138554+0000","last_became_peered":"2024-02-26T10:39:55.138554+0000","last_unstale":"2024-02-26T10:39:57.141190+0000","last_undegraded":"2024-02-26T10:39:57.141190+0000","last_fullsized":"2024-02-26T10:39:57.141190+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:07:23.177818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141229+0000","last_change":"2024-02-26T10:39:56.136419+0000","last_active":"2024-02-26T10:39:57.141229+0000","last_peered":"2024-02-26T10:39:57.141229+0000","last_clean":"2024-02-26T10:39:57.141229+0000","last_became_active":"2024-02-26T10:39:56.134559+0000","last_became_peered":"2024-02-26T10:39:56.134559+0000","last_unstale":"2024-02-26T10:39:57.141229+0000","last_undegraded":"2024-02-26T10:39:57.141229+0000","last_fullsized":"2024-02-26T10:39:57.141229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:45.659293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142138+0000","last_change":"2024-02-26T10:39:39.008501+0000","last_active":"2024-02-26T10:39:57.142138+0000","last_peered":"2024-02-26T10:39:57.142138+0000","last_clean":"2024-02-26T10:39:57.142138+0000","last_became_active":"2024-02-26T10:39:39.008223+0000","last_became_peered":"2024-02-26T10:39:39.008223+0000","last_unstale":"2024-02-26T10:39:57.142138+0000","last_undegraded":"2024-02-26T10:39:57.142138+0000","last_fullsized":"2024-02-26T10:39:57.142138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:43:48.232676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255904+0000","last_change":"2024-02-26T10:39:40.025178+0000","last_active":"2024-02-26T10:39:57.255904+0000","last_peered":"2024-02-26T10:39:57.255904+0000","last_clean":"2024-02-26T10:39:57.255904+0000","last_became_active":"2024-02-26T10:39:40.024856+0000","last_became_peered":"2024-02-26T10:39:40.024856+0000","last_unstale":"2024-02-26T10:39:57.255904+0000","last_undegraded":"2024-02-26T10:39:57.255904+0000","last_fullsized":"2024-02-26T10:39:57.255904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:24.336281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257906+0000","last_change":"2024-02-26T10:39:55.150872+0000","last_active":"2024-02-26T10:39:57.257906+0000","last_peered":"2024-02-26T10:39:57.257906+0000","last_clean":"2024-02-26T10:39:57.257906+0000","last_became_active":"2024-02-26T10:39:55.150170+0000","last_became_peered":"2024-02-26T10:39:55.150170+0000","last_unstale":"2024-02-26T10:39:57.257906+0000","last_undegraded":"2024-02-26T10:39:57.257906+0000","last_fullsized":"2024-02-26T10:39:57.257906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:10.985831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249900+0000","last_change":"2024-02-26T10:39:56.151022+0000","last_active":"2024-02-26T10:39:57.249900+0000","last_peered":"2024-02-26T10:39:57.249900+0000","last_clean":"2024-02-26T10:39:57.249900+0000","last_became_active":"2024-02-26T10:39:56.143642+0000","last_became_peered":"2024-02-26T10:39:56.143642+0000","last_unstale":"2024-02-26T10:39:57.249900+0000","last_undegraded":"2024-02-26T10:39:57.249900+0000","last_fullsized":"2024-02-26T10:39:57.249900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:31:54.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142753+0000","last_change":"2024-02-26T10:39:39.003513+0000","last_active":"2024-02-26T10:39:57.142753+0000","last_peered":"2024-02-26T10:39:57.142753+0000","last_clean":"2024-02-26T10:39:57.142753+0000","last_became_active":"2024-02-26T10:39:39.002989+0000","last_became_peered":"2024-02-26T10:39:39.002989+0000","last_unstale":"2024-02-26T10:39:57.142753+0000","last_undegraded":"2024-02-26T10:39:57.142753+0000","last_fullsized":"2024-02-26T10:39:57.142753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:50:13.624048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250412+0000","last_change":"2024-02-26T10:39:40.025109+0000","last_active":"2024-02-26T10:39:57.250412+0000","last_peered":"2024-02-26T10:39:57.250412+0000","last_clean":"2024-02-26T10:39:57.250412+0000","last_became_active":"2024-02-26T10:39:40.024213+0000","last_became_peered":"2024-02-26T10:39:40.024213+0000","last_unstale":"2024-02-26T10:39:57.250412+0000","last_undegraded":"2024-02-26T10:39:57.250412+0000","last_fullsized":"2024-02-26T10:39:57.250412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:48.049981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143925+0000","last_change":"2024-02-26T10:39:55.143200+0000","last_active":"2024-02-26T10:39:57.143925+0000","last_peered":"2024-02-26T10:39:57.143925+0000","last_clean":"2024-02-26T10:39:57.143925+0000","last_became_active":"2024-02-26T10:39:55.142811+0000","last_became_peered":"2024-02-26T10:39:55.142811+0000","last_unstale":"2024-02-26T10:39:57.143925+0000","last_undegraded":"2024-02-26T10:39:57.143925+0000","last_fullsized":"2024-02-26T10:39:57.143925+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:26.522970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143592+0000","last_change":"2024-02-26T10:39:56.145300+0000","last_active":"2024-02-26T10:39:57.143592+0000","last_peered":"2024-02-26T10:39:57.143592+0000","last_clean":"2024-02-26T10:39:57.143592+0000","last_became_active":"2024-02-26T10:39:56.144515+0000","last_became_peered":"2024-02-26T10:39:56.144515+0000","last_unstale":"2024-02-26T10:39:57.143592+0000","last_undegraded":"2024-02-26T10:39:57.143592+0000","last_fullsized":"2024-02-26T10:39:57.143592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:44:34.878600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139805+0000","last_change":"2024-02-26T10:39:38.998681+0000","last_active":"2024-02-26T10:39:57.139805+0000","last_peered":"2024-02-26T10:39:57.139805+0000","last_clean":"2024-02-26T10:39:57.139805+0000","last_became_active":"2024-02-26T10:39:38.998355+0000","last_became_peered":"2024-02-26T10:39:38.998355+0000","last_unstale":"2024-02-26T10:39:57.139805+0000","last_undegraded":"2024-02-26T10:39:57.139805+0000","last_fullsized":"2024-02-26T10:39:57.139805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:54.191021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142732+0000","last_change":"2024-02-26T10:39:40.014474+0000","last_active":"2024-02-26T10:39:57.142732+0000","last_peered":"2024-02-26T10:39:57.142732+0000","last_clean":"2024-02-26T10:39:57.142732+0000","last_became_active":"2024-02-26T10:39:40.014129+0000","last_became_peered":"2024-02-26T10:39:40.014129+0000","last_unstale":"2024-02-26T10:39:57.142732+0000","last_undegraded":"2024-02-26T10:39:57.142732+0000","last_fullsized":"2024-02-26T10:39:57.142732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:48:55.255246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140095+0000","last_change":"2024-02-26T10:39:55.147338+0000","last_active":"2024-02-26T10:39:57.140095+0000","last_peered":"2024-02-26T10:39:57.140095+0000","last_clean":"2024-02-26T10:39:57.140095+0000","last_became_active":"2024-02-26T10:39:55.147106+0000","last_became_peered":"2024-02-26T10:39:55.147106+0000","last_unstale":"2024-02-26T10:39:57.140095+0000","last_undegraded":"2024-02-26T10:39:57.140095+0000","last_fullsized":"2024-02-26T10:39:57.140095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:19.064657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255083+0000","last_change":"2024-02-26T10:39:56.148427+0000","last_active":"2024-02-26T10:39:57.255083+0000","last_peered":"2024-02-26T10:39:57.255083+0000","last_clean":"2024-02-26T10:39:57.255083+0000","last_became_active":"2024-02-26T10:39:56.148146+0000","last_became_peered":"2024-02-26T10:39:56.148146+0000","last_unstale":"2024-02-26T10:39:57.255083+0000","last_undegraded":"2024-02-26T10:39:57.255083+0000","last_fullsized":"2024-02-26T10:39:57.255083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:26.929354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144317+0000","last_change":"2024-02-26T10:39:39.007073+0000","last_active":"2024-02-26T10:39:57.144317+0000","last_peered":"2024-02-26T10:39:57.144317+0000","last_clean":"2024-02-26T10:39:57.144317+0000","last_became_active":"2024-02-26T10:39:39.004616+0000","last_became_peered":"2024-02-26T10:39:39.004616+0000","last_unstale":"2024-02-26T10:39:57.144317+0000","last_undegraded":"2024-02-26T10:39:57.144317+0000","last_fullsized":"2024-02-26T10:39:57.144317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:33:07.037054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252940+0000","last_change":"2024-02-26T10:39:40.024744+0000","last_active":"2024-02-26T10:39:57.252940+0000","last_peered":"2024-02-26T10:39:57.252940+0000","last_clean":"2024-02-26T10:39:57.252940+0000","last_became_active":"2024-02-26T10:39:40.023928+0000","last_became_peered":"2024-02-26T10:39:40.023928+0000","last_unstale":"2024-02-26T10:39:57.252940+0000","last_undegraded":"2024-02-26T10:39:57.252940+0000","last_fullsized":"2024-02-26T10:39:57.252940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249140+0000","last_change":"2024-02-26T10:39:55.134727+0000","last_active":"2024-02-26T10:39:57.249140+0000","last_peered":"2024-02-26T10:39:57.249140+0000","last_clean":"2024-02-26T10:39:57.249140+0000","last_became_active":"2024-02-26T10:39:55.132818+0000","last_became_peered":"2024-02-26T10:39:55.132818+0000","last_unstale":"2024-02-26T10:39:57.249140+0000","last_undegraded":"2024-02-26T10:39:57.249140+0000","last_fullsized":"2024-02-26T10:39:57.249140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:22.197650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249126+0000","last_change":"2024-02-26T10:39:56.141946+0000","last_active":"2024-02-26T10:39:57.249126+0000","last_peered":"2024-02-26T10:39:57.249126+0000","last_clean":"2024-02-26T10:39:57.249126+0000","last_became_active":"2024-02-26T10:39:56.141311+0000","last_became_peered":"2024-02-26T10:39:56.141311+0000","last_unstale":"2024-02-26T10:39:57.249126+0000","last_undegraded":"2024-02-26T10:39:57.249126+0000","last_fullsized":"2024-02-26T10:39:57.249126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:50.487682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143993+0000","last_change":"2024-02-26T10:39:39.005253+0000","last_active":"2024-02-26T10:39:57.143993+0000","last_peered":"2024-02-26T10:39:57.143993+0000","last_clean":"2024-02-26T10:39:57.143993+0000","last_became_active":"2024-02-26T10:39:39.004947+0000","last_became_peered":"2024-02-26T10:39:39.004947+0000","last_unstale":"2024-02-26T10:39:57.143993+0000","last_undegraded":"2024-02-26T10:39:57.143993+0000","last_fullsized":"2024-02-26T10:39:57.143993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:19:13.361650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257547+0000","last_change":"2024-02-26T10:39:40.017365+0000","last_active":"2024-02-26T10:39:57.257547+0000","last_peered":"2024-02-26T10:39:57.257547+0000","last_clean":"2024-02-26T10:39:57.257547+0000","last_became_active":"2024-02-26T10:39:40.010336+0000","last_became_peered":"2024-02-26T10:39:40.010336+0000","last_unstale":"2024-02-26T10:39:57.257547+0000","last_undegraded":"2024-02-26T10:39:57.257547+0000","last_fullsized":"2024-02-26T10:39:57.257547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:24:26.186093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253111+0000","last_change":"2024-02-26T10:39:55.139886+0000","last_active":"2024-02-26T10:39:57.253111+0000","last_peered":"2024-02-26T10:39:57.253111+0000","last_clean":"2024-02-26T10:39:57.253111+0000","last_became_active":"2024-02-26T10:39:55.139576+0000","last_became_peered":"2024-02-26T10:39:55.139576+0000","last_unstale":"2024-02-26T10:39:57.253111+0000","last_undegraded":"2024-02-26T10:39:57.253111+0000","last_fullsized":"2024-02-26T10:39:57.253111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:01.886808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258390+0000","last_change":"2024-02-26T10:39:56.150239+0000","last_active":"2024-02-26T10:39:57.258390+0000","last_peered":"2024-02-26T10:39:57.258390+0000","last_clean":"2024-02-26T10:39:57.258390+0000","last_became_active":"2024-02-26T10:39:56.147771+0000","last_became_peered":"2024-02-26T10:39:56.147771+0000","last_unstale":"2024-02-26T10:39:57.258390+0000","last_undegraded":"2024-02-26T10:39:57.258390+0000","last_fullsized":"2024-02-26T10:39:57.258390+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:18.580748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145066+0000","last_change":"2024-02-26T10:39:55.140948+0000","last_active":"2024-02-26T10:39:57.145066+0000","last_peered":"2024-02-26T10:39:57.145066+0000","last_clean":"2024-02-26T10:39:57.145066+0000","last_became_active":"2024-02-26T10:39:55.140674+0000","last_became_peered":"2024-02-26T10:39:55.140674+0000","last_unstale":"2024-02-26T10:39:57.145066+0000","last_undegraded":"2024-02-26T10:39:57.145066+0000","last_fullsized":"2024-02-26T10:39:57.145066+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:13:36.808744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141400+0000","last_change":"2024-02-26T10:39:38.993592+0000","last_active":"2024-02-26T10:39:57.141400+0000","last_peered":"2024-02-26T10:39:57.141400+0000","last_clean":"2024-02-26T10:39:57.141400+0000","last_became_active":"2024-02-26T10:39:38.993042+0000","last_became_peered":"2024-02-26T10:39:38.993042+0000","last_unstale":"2024-02-26T10:39:57.141400+0000","last_undegraded":"2024-02-26T10:39:57.141400+0000","last_fullsized":"2024-02-26T10:39:57.141400+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:06:10.297346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251154+0000","last_change":"2024-02-26T10:39:40.008958+0000","last_active":"2024-02-26T10:39:57.251154+0000","last_peered":"2024-02-26T10:39:57.251154+0000","last_clean":"2024-02-26T10:39:57.251154+0000","last_became_active":"2024-02-26T10:39:40.008566+0000","last_became_peered":"2024-02-26T10:39:40.008566+0000","last_unstale":"2024-02-26T10:39:57.251154+0000","last_undegraded":"2024-02-26T10:39:57.251154+0000","last_fullsized":"2024-02-26T10:39:57.251154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251377+0000","last_change":"2024-02-26T10:39:56.139143+0000","last_active":"2024-02-26T10:39:57.251377+0000","last_peered":"2024-02-26T10:39:57.251377+0000","last_clean":"2024-02-26T10:39:57.251377+0000","last_became_active":"2024-02-26T10:39:56.138676+0000","last_became_peered":"2024-02-26T10:39:56.138676+0000","last_unstale":"2024-02-26T10:39:57.251377+0000","last_undegraded":"2024-02-26T10:39:57.251377+0000","last_fullsized":"2024-02-26T10:39:57.251377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:02.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142124+0000","last_change":"2024-02-26T10:39:55.151001+0000","last_active":"2024-02-26T10:39:57.142124+0000","last_peered":"2024-02-26T10:39:57.142124+0000","last_clean":"2024-02-26T10:39:57.142124+0000","last_became_active":"2024-02-26T10:39:55.150677+0000","last_became_peered":"2024-02-26T10:39:55.150677+0000","last_unstale":"2024-02-26T10:39:57.142124+0000","last_undegraded":"2024-02-26T10:39:57.142124+0000","last_fullsized":"2024-02-26T10:39:57.142124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:45.497453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141113+0000","last_change":"2024-02-26T10:39:39.000798+0000","last_active":"2024-02-26T10:39:57.141113+0000","last_peered":"2024-02-26T10:39:57.141113+0000","last_clean":"2024-02-26T10:39:57.141113+0000","last_became_active":"2024-02-26T10:39:38.998438+0000","last_became_peered":"2024-02-26T10:39:38.998438+0000","last_unstale":"2024-02-26T10:39:57.141113+0000","last_undegraded":"2024-02-26T10:39:57.141113+0000","last_fullsized":"2024-02-26T10:39:57.141113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:29:40.306221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144467+0000","last_change":"2024-02-26T10:39:39.992637+0000","last_active":"2024-02-26T10:39:57.144467+0000","last_peered":"2024-02-26T10:39:57.144467+0000","last_clean":"2024-02-26T10:39:57.144467+0000","last_became_active":"2024-02-26T10:39:39.991960+0000","last_became_peered":"2024-02-26T10:39:39.991960+0000","last_unstale":"2024-02-26T10:39:57.144467+0000","last_undegraded":"2024-02-26T10:39:57.144467+0000","last_fullsized":"2024-02-26T10:39:57.144467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251535+0000","last_change":"2024-02-26T10:39:56.144807+0000","last_active":"2024-02-26T10:39:57.251535+0000","last_peered":"2024-02-26T10:39:57.251535+0000","last_clean":"2024-02-26T10:39:57.251535+0000","last_became_active":"2024-02-26T10:39:56.144227+0000","last_became_peered":"2024-02-26T10:39:56.144227+0000","last_unstale":"2024-02-26T10:39:57.251535+0000","last_undegraded":"2024-02-26T10:39:57.251535+0000","last_fullsized":"2024-02-26T10:39:57.251535+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:01.709690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250666+0000","last_change":"2024-02-26T10:39:39.013171+0000","last_active":"2024-02-26T10:39:57.250666+0000","last_peered":"2024-02-26T10:39:57.250666+0000","last_clean":"2024-02-26T10:39:57.250666+0000","last_became_active":"2024-02-26T10:39:39.012785+0000","last_became_peered":"2024-02-26T10:39:39.012785+0000","last_unstale":"2024-02-26T10:39:57.250666+0000","last_undegraded":"2024-02-26T10:39:57.250666+0000","last_fullsized":"2024-02-26T10:39:57.250666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:04:56.455641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259712+0000","last_change":"2024-02-26T10:39:40.011479+0000","last_active":"2024-02-26T10:39:57.259712+0000","last_peered":"2024-02-26T10:39:57.259712+0000","last_clean":"2024-02-26T10:39:57.259712+0000","last_became_active":"2024-02-26T10:39:40.011123+0000","last_became_peered":"2024-02-26T10:39:40.011123+0000","last_unstale":"2024-02-26T10:39:57.259712+0000","last_undegraded":"2024-02-26T10:39:57.259712+0000","last_fullsized":"2024-02-26T10:39:57.259712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258691+0000","last_change":"2024-02-26T10:39:55.148876+0000","last_active":"2024-02-26T10:39:57.258691+0000","last_peered":"2024-02-26T10:39:57.258691+0000","last_clean":"2024-02-26T10:39:57.258691+0000","last_became_active":"2024-02-26T10:39:55.148547+0000","last_became_peered":"2024-02-26T10:39:55.148547+0000","last_unstale":"2024-02-26T10:39:57.258691+0000","last_undegraded":"2024-02-26T10:39:57.258691+0000","last_fullsized":"2024-02-26T10:39:57.258691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:42:01.886808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254068+0000","last_change":"2024-02-26T10:39:56.152573+0000","last_active":"2024-02-26T10:39:57.254068+0000","last_peered":"2024-02-26T10:39:57.254068+0000","last_clean":"2024-02-26T10:39:57.254068+0000","last_became_active":"2024-02-26T10:39:56.152297+0000","last_became_peered":"2024-02-26T10:39:56.152297+0000","last_unstale":"2024-02-26T10:39:57.254068+0000","last_undegraded":"2024-02-26T10:39:57.254068+0000","last_fullsized":"2024-02-26T10:39:57.254068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:29:29.247494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259949+0000","last_change":"2024-02-26T10:39:39.014072+0000","last_active":"2024-02-26T10:39:57.259949+0000","last_peered":"2024-02-26T10:39:57.259949+0000","last_clean":"2024-02-26T10:39:57.259949+0000","last_became_active":"2024-02-26T10:39:39.013810+0000","last_became_peered":"2024-02-26T10:39:39.013810+0000","last_unstale":"2024-02-26T10:39:57.259949+0000","last_undegraded":"2024-02-26T10:39:57.259949+0000","last_fullsized":"2024-02-26T10:39:57.259949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:37.941716+0000","last_clean_scrub_stamp":"2024-02-26T10:39:37.941716+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:12:03.804658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250528+0000","last_change":"2024-02-26T10:39:40.021369+0000","last_active":"2024-02-26T10:39:57.250528+0000","last_peered":"2024-02-26T10:39:57.250528+0000","last_clean":"2024-02-26T10:39:57.250528+0000","last_became_active":"2024-02-26T10:39:40.021061+0000","last_became_peered":"2024-02-26T10:39:40.021061+0000","last_unstale":"2024-02-26T10:39:57.250528+0000","last_undegraded":"2024-02-26T10:39:57.250528+0000","last_fullsized":"2024-02-26T10:39:57.250528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:13.801758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251509+0000","last_change":"2024-02-26T10:39:55.128982+0000","last_active":"2024-02-26T10:39:57.251509+0000","last_peered":"2024-02-26T10:39:57.251509+0000","last_clean":"2024-02-26T10:39:57.251509+0000","last_became_active":"2024-02-26T10:39:55.126448+0000","last_became_peered":"2024-02-26T10:39:55.126448+0000","last_unstale":"2024-02-26T10:39:57.251509+0000","last_undegraded":"2024-02-26T10:39:57.251509+0000","last_fullsized":"2024-02-26T10:39:57.251509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:54.073522+0000","last_clean_scrub_stamp":"2024-02-26T10:39:54.073522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:53.558326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143449+0000","last_change":"2024-02-26T10:39:56.149841+0000","last_active":"2024-02-26T10:39:57.143449+0000","last_peered":"2024-02-26T10:39:57.143449+0000","last_clean":"2024-02-26T10:39:57.143449+0000","last_became_active":"2024-02-26T10:39:56.148212+0000","last_became_peered":"2024-02-26T10:39:56.148212+0000","last_unstale":"2024-02-26T10:39:57.143449+0000","last_undegraded":"2024-02-26T10:39:57.143449+0000","last_fullsized":"2024-02-26T10:39:57.143449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:54.573144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255382+0000","last_change":"2024-02-26T10:39:40.013245+0000","last_active":"2024-02-26T10:39:57.255382+0000","last_peered":"2024-02-26T10:39:57.255382+0000","last_clean":"2024-02-26T10:39:57.255382+0000","last_became_active":"2024-02-26T10:39:40.008999+0000","last_became_peered":"2024-02-26T10:39:40.008999+0000","last_unstale":"2024-02-26T10:39:57.255382+0000","last_undegraded":"2024-02-26T10:39:57.255382+0000","last_fullsized":"2024-02-26T10:39:57.255382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:11.406843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256915+0000","last_change":"2024-02-26T10:39:56.140813+0000","last_active":"2024-02-26T10:39:57.256915+0000","last_peered":"2024-02-26T10:39:57.256915+0000","last_clean":"2024-02-26T10:39:57.256915+0000","last_became_active":"2024-02-26T10:39:56.140525+0000","last_became_peered":"2024-02-26T10:39:56.140525+0000","last_unstale":"2024-02-26T10:39:57.256915+0000","last_undegraded":"2024-02-26T10:39:57.256915+0000","last_fullsized":"2024-02-26T10:39:57.256915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:58:40.467226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255695+0000","last_change":"2024-02-26T10:39:40.008658+0000","last_active":"2024-02-26T10:39:57.255695+0000","last_peered":"2024-02-26T10:39:57.255695+0000","last_clean":"2024-02-26T10:39:57.255695+0000","last_became_active":"2024-02-26T10:39:40.008354+0000","last_became_peered":"2024-02-26T10:39:40.008354+0000","last_unstale":"2024-02-26T10:39:57.255695+0000","last_undegraded":"2024-02-26T10:39:57.255695+0000","last_fullsized":"2024-02-26T10:39:57.255695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140075+0000","last_change":"2024-02-26T10:39:56.132854+0000","last_active":"2024-02-26T10:39:57.140075+0000","last_peered":"2024-02-26T10:39:57.140075+0000","last_clean":"2024-02-26T10:39:57.140075+0000","last_became_active":"2024-02-26T10:39:56.130809+0000","last_became_peered":"2024-02-26T10:39:56.130809+0000","last_unstale":"2024-02-26T10:39:57.140075+0000","last_undegraded":"2024-02-26T10:39:57.140075+0000","last_fullsized":"2024-02-26T10:39:57.140075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:58:20.020460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257284+0000","last_change":"2024-02-26T10:39:40.010029+0000","last_active":"2024-02-26T10:39:57.257284+0000","last_peered":"2024-02-26T10:39:57.257284+0000","last_clean":"2024-02-26T10:39:57.257284+0000","last_became_active":"2024-02-26T10:39:40.009745+0000","last_became_peered":"2024-02-26T10:39:40.009745+0000","last_unstale":"2024-02-26T10:39:57.257284+0000","last_undegraded":"2024-02-26T10:39:57.257284+0000","last_fullsized":"2024-02-26T10:39:57.257284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138983+0000","last_change":"2024-02-26T10:39:56.140807+0000","last_active":"2024-02-26T10:39:57.138983+0000","last_peered":"2024-02-26T10:39:57.138983+0000","last_clean":"2024-02-26T10:39:57.138983+0000","last_became_active":"2024-02-26T10:39:56.140429+0000","last_became_peered":"2024-02-26T10:39:56.140429+0000","last_unstale":"2024-02-26T10:39:57.138983+0000","last_undegraded":"2024-02-26T10:39:57.138983+0000","last_fullsized":"2024-02-26T10:39:57.138983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:07:53.194566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143736+0000","last_change":"2024-02-26T10:39:39.994088+0000","last_active":"2024-02-26T10:39:57.143736+0000","last_peered":"2024-02-26T10:39:57.143736+0000","last_clean":"2024-02-26T10:39:57.143736+0000","last_became_active":"2024-02-26T10:39:39.993717+0000","last_became_peered":"2024-02-26T10:39:39.993717+0000","last_unstale":"2024-02-26T10:39:57.143736+0000","last_undegraded":"2024-02-26T10:39:57.143736+0000","last_fullsized":"2024-02-26T10:39:57.143736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140571+0000","last_change":"2024-02-26T10:39:56.133103+0000","last_active":"2024-02-26T10:39:57.140571+0000","last_peered":"2024-02-26T10:39:57.140571+0000","last_clean":"2024-02-26T10:39:57.140571+0000","last_became_active":"2024-02-26T10:39:56.132794+0000","last_became_peered":"2024-02-26T10:39:56.132794+0000","last_unstale":"2024-02-26T10:39:57.140571+0000","last_undegraded":"2024-02-26T10:39:57.140571+0000","last_fullsized":"2024-02-26T10:39:57.140571+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:06:13.230409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254546+0000","last_change":"2024-02-26T10:39:40.019372+0000","last_active":"2024-02-26T10:39:57.254546+0000","last_peered":"2024-02-26T10:39:57.254546+0000","last_clean":"2024-02-26T10:39:57.254546+0000","last_became_active":"2024-02-26T10:39:40.019025+0000","last_became_peered":"2024-02-26T10:39:40.019025+0000","last_unstale":"2024-02-26T10:39:57.254546+0000","last_undegraded":"2024-02-26T10:39:57.254546+0000","last_fullsized":"2024-02-26T10:39:57.254546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258943+0000","last_change":"2024-02-26T10:39:56.141645+0000","last_active":"2024-02-26T10:39:57.258943+0000","last_peered":"2024-02-26T10:39:57.258943+0000","last_clean":"2024-02-26T10:39:57.258943+0000","last_became_active":"2024-02-26T10:39:56.140872+0000","last_became_peered":"2024-02-26T10:39:56.140872+0000","last_unstale":"2024-02-26T10:39:57.258943+0000","last_undegraded":"2024-02-26T10:39:57.258943+0000","last_fullsized":"2024-02-26T10:39:57.258943+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:54.573144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251233+0000","last_change":"2024-02-26T10:39:40.008980+0000","last_active":"2024-02-26T10:39:57.251233+0000","last_peered":"2024-02-26T10:39:57.251233+0000","last_clean":"2024-02-26T10:39:57.251233+0000","last_became_active":"2024-02-26T10:39:40.008531+0000","last_became_peered":"2024-02-26T10:39:40.008531+0000","last_unstale":"2024-02-26T10:39:57.251233+0000","last_undegraded":"2024-02-26T10:39:57.251233+0000","last_fullsized":"2024-02-26T10:39:57.251233+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:11:31.399212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253631+0000","last_change":"2024-02-26T10:39:56.146232+0000","last_active":"2024-02-26T10:39:57.253631+0000","last_peered":"2024-02-26T10:39:57.253631+0000","last_clean":"2024-02-26T10:39:57.253631+0000","last_became_active":"2024-02-26T10:39:56.143256+0000","last_became_peered":"2024-02-26T10:39:56.143256+0000","last_unstale":"2024-02-26T10:39:57.253631+0000","last_undegraded":"2024-02-26T10:39:57.253631+0000","last_fullsized":"2024-02-26T10:39:57.253631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:21:53.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141785+0000","last_change":"2024-02-26T10:39:40.007272+0000","last_active":"2024-02-26T10:39:57.141785+0000","last_peered":"2024-02-26T10:39:57.141785+0000","last_clean":"2024-02-26T10:39:57.141785+0000","last_became_active":"2024-02-26T10:39:40.006923+0000","last_became_peered":"2024-02-26T10:39:40.006923+0000","last_unstale":"2024-02-26T10:39:57.141785+0000","last_undegraded":"2024-02-26T10:39:57.141785+0000","last_fullsized":"2024-02-26T10:39:57.141785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:21.270698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144250+0000","last_change":"2024-02-26T10:39:56.149837+0000","last_active":"2024-02-26T10:39:57.144250+0000","last_peered":"2024-02-26T10:39:57.144250+0000","last_clean":"2024-02-26T10:39:57.144250+0000","last_became_active":"2024-02-26T10:39:56.147660+0000","last_became_peered":"2024-02-26T10:39:56.147660+0000","last_unstale":"2024-02-26T10:39:57.144250+0000","last_undegraded":"2024-02-26T10:39:57.144250+0000","last_fullsized":"2024-02-26T10:39:57.144250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:46.652301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251424+0000","last_change":"2024-02-26T10:39:40.015014+0000","last_active":"2024-02-26T10:39:57.251424+0000","last_peered":"2024-02-26T10:39:57.251424+0000","last_clean":"2024-02-26T10:39:57.251424+0000","last_became_active":"2024-02-26T10:39:40.014611+0000","last_became_peered":"2024-02-26T10:39:40.014611+0000","last_unstale":"2024-02-26T10:39:57.251424+0000","last_undegraded":"2024-02-26T10:39:57.251424+0000","last_fullsized":"2024-02-26T10:39:57.251424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249191+0000","last_change":"2024-02-26T10:39:56.150318+0000","last_active":"2024-02-26T10:39:57.249191+0000","last_peered":"2024-02-26T10:39:57.249191+0000","last_clean":"2024-02-26T10:39:57.249191+0000","last_became_active":"2024-02-26T10:39:56.150020+0000","last_became_peered":"2024-02-26T10:39:56.150020+0000","last_unstale":"2024-02-26T10:39:57.249191+0000","last_undegraded":"2024-02-26T10:39:57.249191+0000","last_fullsized":"2024-02-26T10:39:57.249191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:52.852187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252380+0000","last_change":"2024-02-26T10:39:40.017086+0000","last_active":"2024-02-26T10:39:57.252380+0000","last_peered":"2024-02-26T10:39:57.252380+0000","last_clean":"2024-02-26T10:39:57.252380+0000","last_became_active":"2024-02-26T10:39:40.016779+0000","last_became_peered":"2024-02-26T10:39:40.016779+0000","last_unstale":"2024-02-26T10:39:57.252380+0000","last_undegraded":"2024-02-26T10:39:57.252380+0000","last_fullsized":"2024-02-26T10:39:57.252380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:07.081524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139808+0000","last_change":"2024-02-26T10:39:56.154203+0000","last_active":"2024-02-26T10:39:57.139808+0000","last_peered":"2024-02-26T10:39:57.139808+0000","last_clean":"2024-02-26T10:39:57.139808+0000","last_became_active":"2024-02-26T10:39:56.151496+0000","last_became_peered":"2024-02-26T10:39:56.151496+0000","last_unstale":"2024-02-26T10:39:57.139808+0000","last_undegraded":"2024-02-26T10:39:57.139808+0000","last_fullsized":"2024-02-26T10:39:57.139808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:46:37.087257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249183+0000","last_change":"2024-02-26T10:39:40.022805+0000","last_active":"2024-02-26T10:39:57.249183+0000","last_peered":"2024-02-26T10:39:57.249183+0000","last_clean":"2024-02-26T10:39:57.249183+0000","last_became_active":"2024-02-26T10:39:40.022510+0000","last_became_peered":"2024-02-26T10:39:40.022510+0000","last_unstale":"2024-02-26T10:39:57.249183+0000","last_undegraded":"2024-02-26T10:39:57.249183+0000","last_fullsized":"2024-02-26T10:39:57.249183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:14:55.206700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254107+0000","last_change":"2024-02-26T10:39:56.151748+0000","last_active":"2024-02-26T10:39:57.254107+0000","last_peered":"2024-02-26T10:39:57.254107+0000","last_clean":"2024-02-26T10:39:57.254107+0000","last_became_active":"2024-02-26T10:39:56.150645+0000","last_became_peered":"2024-02-26T10:39:56.150645+0000","last_unstale":"2024-02-26T10:39:57.254107+0000","last_undegraded":"2024-02-26T10:39:57.254107+0000","last_fullsized":"2024-02-26T10:39:57.254107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:52.489061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144063+0000","last_change":"2024-02-26T10:39:39.990872+0000","last_active":"2024-02-26T10:39:57.144063+0000","last_peered":"2024-02-26T10:39:57.144063+0000","last_clean":"2024-02-26T10:39:57.144063+0000","last_became_active":"2024-02-26T10:39:39.990134+0000","last_became_peered":"2024-02-26T10:39:39.990134+0000","last_unstale":"2024-02-26T10:39:57.144063+0000","last_undegraded":"2024-02-26T10:39:57.144063+0000","last_fullsized":"2024-02-26T10:39:57.144063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256202+0000","last_change":"2024-02-26T10:39:56.151370+0000","last_active":"2024-02-26T10:39:57.256202+0000","last_peered":"2024-02-26T10:39:57.256202+0000","last_clean":"2024-02-26T10:39:57.256202+0000","last_became_active":"2024-02-26T10:39:56.150807+0000","last_became_peered":"2024-02-26T10:39:56.150807+0000","last_unstale":"2024-02-26T10:39:57.256202+0000","last_undegraded":"2024-02-26T10:39:57.256202+0000","last_fullsized":"2024-02-26T10:39:57.256202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:18.352022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.143431+0000","last_change":"2024-02-26T10:39:40.003035+0000","last_active":"2024-02-26T10:39:57.143431+0000","last_peered":"2024-02-26T10:39:57.143431+0000","last_clean":"2024-02-26T10:39:57.143431+0000","last_became_active":"2024-02-26T10:39:40.002724+0000","last_became_peered":"2024-02-26T10:39:40.002724+0000","last_unstale":"2024-02-26T10:39:57.143431+0000","last_undegraded":"2024-02-26T10:39:57.143431+0000","last_fullsized":"2024-02-26T10:39:57.143431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:37.543329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144597+0000","last_change":"2024-02-26T10:39:56.163643+0000","last_active":"2024-02-26T10:39:57.144597+0000","last_peered":"2024-02-26T10:39:57.144597+0000","last_clean":"2024-02-26T10:39:57.144597+0000","last_became_active":"2024-02-26T10:39:56.163310+0000","last_became_peered":"2024-02-26T10:39:56.163310+0000","last_unstale":"2024-02-26T10:39:57.144597+0000","last_undegraded":"2024-02-26T10:39:57.144597+0000","last_fullsized":"2024-02-26T10:39:57.144597+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:51:53.159078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141323+0000","last_change":"2024-02-26T10:39:40.028123+0000","last_active":"2024-02-26T10:39:57.141323+0000","last_peered":"2024-02-26T10:39:57.141323+0000","last_clean":"2024-02-26T10:39:57.141323+0000","last_became_active":"2024-02-26T10:39:40.027299+0000","last_became_peered":"2024-02-26T10:39:40.027299+0000","last_unstale":"2024-02-26T10:39:57.141323+0000","last_undegraded":"2024-02-26T10:39:57.141323+0000","last_fullsized":"2024-02-26T10:39:57.141323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:54:29.531317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.145385+0000","last_change":"2024-02-26T10:39:56.150941+0000","last_active":"2024-02-26T10:39:57.145385+0000","last_peered":"2024-02-26T10:39:57.145385+0000","last_clean":"2024-02-26T10:39:57.145385+0000","last_became_active":"2024-02-26T10:39:56.148004+0000","last_became_peered":"2024-02-26T10:39:56.148004+0000","last_unstale":"2024-02-26T10:39:57.145385+0000","last_undegraded":"2024-02-26T10:39:57.145385+0000","last_fullsized":"2024-02-26T10:39:57.145385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:01.709690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140342+0000","last_change":"2024-02-26T10:39:39.996782+0000","last_active":"2024-02-26T10:39:57.140342+0000","last_peered":"2024-02-26T10:39:57.140342+0000","last_clean":"2024-02-26T10:39:57.140342+0000","last_became_active":"2024-02-26T10:39:39.996469+0000","last_became_peered":"2024-02-26T10:39:39.996469+0000","last_unstale":"2024-02-26T10:39:57.140342+0000","last_undegraded":"2024-02-26T10:39:57.140342+0000","last_fullsized":"2024-02-26T10:39:57.140342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.140977+0000","last_change":"2024-02-26T10:39:56.144562+0000","last_active":"2024-02-26T10:39:57.140977+0000","last_peered":"2024-02-26T10:39:57.140977+0000","last_clean":"2024-02-26T10:39:57.140977+0000","last_became_active":"2024-02-26T10:39:56.143770+0000","last_became_peered":"2024-02-26T10:39:56.143770+0000","last_unstale":"2024-02-26T10:39:57.140977+0000","last_undegraded":"2024-02-26T10:39:57.140977+0000","last_fullsized":"2024-02-26T10:39:57.140977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:25:09.826997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141686+0000","last_change":"2024-02-26T10:39:39.998011+0000","last_active":"2024-02-26T10:39:57.141686+0000","last_peered":"2024-02-26T10:39:57.141686+0000","last_clean":"2024-02-26T10:39:57.141686+0000","last_became_active":"2024-02-26T10:39:39.997725+0000","last_became_peered":"2024-02-26T10:39:39.997725+0000","last_unstale":"2024-02-26T10:39:57.141686+0000","last_undegraded":"2024-02-26T10:39:57.141686+0000","last_fullsized":"2024-02-26T10:39:57.141686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:35:02.449804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259034+0000","last_change":"2024-02-26T10:39:56.142150+0000","last_active":"2024-02-26T10:39:57.259034+0000","last_peered":"2024-02-26T10:39:57.259034+0000","last_clean":"2024-02-26T10:39:57.259034+0000","last_became_active":"2024-02-26T10:39:56.141273+0000","last_became_peered":"2024-02-26T10:39:56.141273+0000","last_unstale":"2024-02-26T10:39:57.259034+0000","last_undegraded":"2024-02-26T10:39:57.259034+0000","last_fullsized":"2024-02-26T10:39:57.259034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:46.652301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.260262+0000","last_change":"2024-02-26T10:39:40.017107+0000","last_active":"2024-02-26T10:39:57.260262+0000","last_peered":"2024-02-26T10:39:57.260262+0000","last_clean":"2024-02-26T10:39:57.260262+0000","last_became_active":"2024-02-26T10:39:40.016813+0000","last_became_peered":"2024-02-26T10:39:40.016813+0000","last_unstale":"2024-02-26T10:39:57.260262+0000","last_undegraded":"2024-02-26T10:39:57.260262+0000","last_fullsized":"2024-02-26T10:39:57.260262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249178+0000","last_change":"2024-02-26T10:39:56.134719+0000","last_active":"2024-02-26T10:39:57.249178+0000","last_peered":"2024-02-26T10:39:57.249178+0000","last_clean":"2024-02-26T10:39:57.249178+0000","last_became_active":"2024-02-26T10:39:56.134420+0000","last_became_peered":"2024-02-26T10:39:56.134420+0000","last_unstale":"2024-02-26T10:39:57.249178+0000","last_undegraded":"2024-02-26T10:39:57.249178+0000","last_fullsized":"2024-02-26T10:39:57.249178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:42:12.000649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256536+0000","last_change":"2024-02-26T10:39:40.011104+0000","last_active":"2024-02-26T10:39:57.256536+0000","last_peered":"2024-02-26T10:39:57.256536+0000","last_clean":"2024-02-26T10:39:57.256536+0000","last_became_active":"2024-02-26T10:39:40.010774+0000","last_became_peered":"2024-02-26T10:39:40.010774+0000","last_unstale":"2024-02-26T10:39:57.256536+0000","last_undegraded":"2024-02-26T10:39:57.256536+0000","last_fullsized":"2024-02-26T10:39:57.256536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:31:10.798410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142302+0000","last_change":"2024-02-26T10:39:56.131881+0000","last_active":"2024-02-26T10:39:57.142302+0000","last_peered":"2024-02-26T10:39:57.142302+0000","last_clean":"2024-02-26T10:39:57.142302+0000","last_became_active":"2024-02-26T10:39:56.131517+0000","last_became_peered":"2024-02-26T10:39:56.131517+0000","last_unstale":"2024-02-26T10:39:57.142302+0000","last_undegraded":"2024-02-26T10:39:57.142302+0000","last_fullsized":"2024-02-26T10:39:57.142302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:29:39.425379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.252741+0000","last_change":"2024-02-26T10:39:40.024788+0000","last_active":"2024-02-26T10:39:57.252741+0000","last_peered":"2024-02-26T10:39:57.252741+0000","last_clean":"2024-02-26T10:39:57.252741+0000","last_became_active":"2024-02-26T10:39:40.024454+0000","last_became_peered":"2024-02-26T10:39:40.024454+0000","last_unstale":"2024-02-26T10:39:57.252741+0000","last_undegraded":"2024-02-26T10:39:57.252741+0000","last_fullsized":"2024-02-26T10:39:57.252741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254552+0000","last_change":"2024-02-26T10:39:56.142843+0000","last_active":"2024-02-26T10:39:57.254552+0000","last_peered":"2024-02-26T10:39:57.254552+0000","last_clean":"2024-02-26T10:39:57.254552+0000","last_became_active":"2024-02-26T10:39:56.142576+0000","last_became_peered":"2024-02-26T10:39:56.142576+0000","last_unstale":"2024-02-26T10:39:57.254552+0000","last_undegraded":"2024-02-26T10:39:57.254552+0000","last_fullsized":"2024-02-26T10:39:57.254552+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:28.003726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139752+0000","last_change":"2024-02-26T10:39:39.999249+0000","last_active":"2024-02-26T10:39:57.139752+0000","last_peered":"2024-02-26T10:39:57.139752+0000","last_clean":"2024-02-26T10:39:57.139752+0000","last_became_active":"2024-02-26T10:39:39.998319+0000","last_became_peered":"2024-02-26T10:39:39.998319+0000","last_unstale":"2024-02-26T10:39:57.139752+0000","last_undegraded":"2024-02-26T10:39:57.139752+0000","last_fullsized":"2024-02-26T10:39:57.139752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:29.312334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138807+0000","last_change":"2024-02-26T10:39:56.142489+0000","last_active":"2024-02-26T10:39:57.138807+0000","last_peered":"2024-02-26T10:39:57.138807+0000","last_clean":"2024-02-26T10:39:57.138807+0000","last_became_active":"2024-02-26T10:39:56.141992+0000","last_became_peered":"2024-02-26T10:39:56.141992+0000","last_unstale":"2024-02-26T10:39:57.138807+0000","last_undegraded":"2024-02-26T10:39:57.138807+0000","last_fullsized":"2024-02-26T10:39:57.138807+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:17:37.709990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142357+0000","last_change":"2024-02-26T10:39:40.000902+0000","last_active":"2024-02-26T10:39:57.142357+0000","last_peered":"2024-02-26T10:39:57.142357+0000","last_clean":"2024-02-26T10:39:57.142357+0000","last_became_active":"2024-02-26T10:39:39.999441+0000","last_became_peered":"2024-02-26T10:39:39.999441+0000","last_unstale":"2024-02-26T10:39:57.142357+0000","last_undegraded":"2024-02-26T10:39:57.142357+0000","last_fullsized":"2024-02-26T10:39:57.142357+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:27.937806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250346+0000","last_change":"2024-02-26T10:39:56.155269+0000","last_active":"2024-02-26T10:39:57.250346+0000","last_peered":"2024-02-26T10:39:57.250346+0000","last_clean":"2024-02-26T10:39:57.250346+0000","last_became_active":"2024-02-26T10:39:56.149555+0000","last_became_peered":"2024-02-26T10:39:56.149555+0000","last_unstale":"2024-02-26T10:39:57.250346+0000","last_undegraded":"2024-02-26T10:39:57.250346+0000","last_fullsized":"2024-02-26T10:39:57.250346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:20.079475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254507+0000","last_change":"2024-02-26T10:39:40.011833+0000","last_active":"2024-02-26T10:39:57.254507+0000","last_peered":"2024-02-26T10:39:57.254507+0000","last_clean":"2024-02-26T10:39:57.254507+0000","last_became_active":"2024-02-26T10:39:40.011581+0000","last_became_peered":"2024-02-26T10:39:40.011581+0000","last_unstale":"2024-02-26T10:39:57.254507+0000","last_undegraded":"2024-02-26T10:39:57.254507+0000","last_fullsized":"2024-02-26T10:39:57.254507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250085+0000","last_change":"2024-02-26T10:39:56.150850+0000","last_active":"2024-02-26T10:39:57.250085+0000","last_peered":"2024-02-26T10:39:57.250085+0000","last_clean":"2024-02-26T10:39:57.250085+0000","last_became_active":"2024-02-26T10:39:56.150551+0000","last_became_peered":"2024-02-26T10:39:56.150551+0000","last_unstale":"2024-02-26T10:39:57.250085+0000","last_undegraded":"2024-02-26T10:39:57.250085+0000","last_fullsized":"2024-02-26T10:39:57.250085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:51.955346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.258388+0000","last_change":"2024-02-26T10:39:40.017115+0000","last_active":"2024-02-26T10:39:57.258388+0000","last_peered":"2024-02-26T10:39:57.258388+0000","last_clean":"2024-02-26T10:39:57.258388+0000","last_became_active":"2024-02-26T10:39:40.016876+0000","last_became_peered":"2024-02-26T10:39:40.016876+0000","last_unstale":"2024-02-26T10:39:57.258388+0000","last_undegraded":"2024-02-26T10:39:57.258388+0000","last_fullsized":"2024-02-26T10:39:57.258388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:36.358116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253180+0000","last_change":"2024-02-26T10:39:56.146988+0000","last_active":"2024-02-26T10:39:57.253180+0000","last_peered":"2024-02-26T10:39:57.253180+0000","last_clean":"2024-02-26T10:39:57.253180+0000","last_became_active":"2024-02-26T10:39:56.146609+0000","last_became_peered":"2024-02-26T10:39:56.146609+0000","last_unstale":"2024-02-26T10:39:57.253180+0000","last_undegraded":"2024-02-26T10:39:57.253180+0000","last_fullsized":"2024-02-26T10:39:57.253180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:50:20.079475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.251419+0000","last_change":"2024-02-26T10:39:39.995327+0000","last_active":"2024-02-26T10:39:57.251419+0000","last_peered":"2024-02-26T10:39:57.251419+0000","last_clean":"2024-02-26T10:39:57.251419+0000","last_became_active":"2024-02-26T10:39:39.994245+0000","last_became_peered":"2024-02-26T10:39:39.994245+0000","last_unstale":"2024-02-26T10:39:57.251419+0000","last_undegraded":"2024-02-26T10:39:57.251419+0000","last_fullsized":"2024-02-26T10:39:57.251419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:44:50.766180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249940+0000","last_change":"2024-02-26T10:39:56.136269+0000","last_active":"2024-02-26T10:39:57.249940+0000","last_peered":"2024-02-26T10:39:57.249940+0000","last_clean":"2024-02-26T10:39:57.249940+0000","last_became_active":"2024-02-26T10:39:56.134890+0000","last_became_peered":"2024-02-26T10:39:56.134890+0000","last_unstale":"2024-02-26T10:39:57.249940+0000","last_undegraded":"2024-02-26T10:39:57.249940+0000","last_fullsized":"2024-02-26T10:39:57.249940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:14.062824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.146252+0000","last_change":"2024-02-26T10:39:40.006430+0000","last_active":"2024-02-26T10:39:57.146252+0000","last_peered":"2024-02-26T10:39:57.146252+0000","last_clean":"2024-02-26T10:39:57.146252+0000","last_became_active":"2024-02-26T10:39:40.004846+0000","last_became_peered":"2024-02-26T10:39:40.004846+0000","last_unstale":"2024-02-26T10:39:57.146252+0000","last_undegraded":"2024-02-26T10:39:57.146252+0000","last_fullsized":"2024-02-26T10:39:57.146252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:37.671936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.138901+0000","last_change":"2024-02-26T10:39:56.155042+0000","last_active":"2024-02-26T10:39:57.138901+0000","last_peered":"2024-02-26T10:39:57.138901+0000","last_clean":"2024-02-26T10:39:57.138901+0000","last_became_active":"2024-02-26T10:39:56.152151+0000","last_became_peered":"2024-02-26T10:39:56.152151+0000","last_unstale":"2024-02-26T10:39:57.138901+0000","last_undegraded":"2024-02-26T10:39:57.138901+0000","last_fullsized":"2024-02-26T10:39:57.138901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:09:40.967204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255104+0000","last_change":"2024-02-26T10:39:40.005845+0000","last_active":"2024-02-26T10:39:57.255104+0000","last_peered":"2024-02-26T10:39:57.255104+0000","last_clean":"2024-02-26T10:39:57.255104+0000","last_became_active":"2024-02-26T10:39:40.005497+0000","last_became_peered":"2024-02-26T10:39:40.005497+0000","last_unstale":"2024-02-26T10:39:57.255104+0000","last_undegraded":"2024-02-26T10:39:57.255104+0000","last_fullsized":"2024-02-26T10:39:57.255104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.253128+0000","last_change":"2024-02-26T10:39:56.155017+0000","last_active":"2024-02-26T10:39:57.253128+0000","last_peered":"2024-02-26T10:39:57.253128+0000","last_clean":"2024-02-26T10:39:57.253128+0000","last_became_active":"2024-02-26T10:39:56.154611+0000","last_became_peered":"2024-02-26T10:39:56.154611+0000","last_unstale":"2024-02-26T10:39:57.253128+0000","last_undegraded":"2024-02-26T10:39:57.253128+0000","last_fullsized":"2024-02-26T10:39:57.253128+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:46.512272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.257358+0000","last_change":"2024-02-26T10:39:40.014885+0000","last_active":"2024-02-26T10:39:57.257358+0000","last_peered":"2024-02-26T10:39:57.257358+0000","last_clean":"2024-02-26T10:39:57.257358+0000","last_became_active":"2024-02-26T10:39:40.014675+0000","last_became_peered":"2024-02-26T10:39:40.014675+0000","last_unstale":"2024-02-26T10:39:57.257358+0000","last_undegraded":"2024-02-26T10:39:57.257358+0000","last_fullsized":"2024-02-26T10:39:57.257358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:26:04.318459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249016+0000","last_change":"2024-02-26T10:39:56.141979+0000","last_active":"2024-02-26T10:39:57.249016+0000","last_peered":"2024-02-26T10:39:57.249016+0000","last_clean":"2024-02-26T10:39:57.249016+0000","last_became_active":"2024-02-26T10:39:56.141657+0000","last_became_peered":"2024-02-26T10:39:56.141657+0000","last_unstale":"2024-02-26T10:39:57.249016+0000","last_undegraded":"2024-02-26T10:39:57.249016+0000","last_fullsized":"2024-02-26T10:39:57.249016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:38:01.709690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.139623+0000","last_change":"2024-02-26T10:39:40.025212+0000","last_active":"2024-02-26T10:39:57.139623+0000","last_peered":"2024-02-26T10:39:57.139623+0000","last_clean":"2024-02-26T10:39:57.139623+0000","last_became_active":"2024-02-26T10:39:40.024642+0000","last_became_peered":"2024-02-26T10:39:40.024642+0000","last_unstale":"2024-02-26T10:39:57.139623+0000","last_undegraded":"2024-02-26T10:39:57.139623+0000","last_fullsized":"2024-02-26T10:39:57.139623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:11.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.256056+0000","last_change":"2024-02-26T10:39:56.146889+0000","last_active":"2024-02-26T10:39:57.256056+0000","last_peered":"2024-02-26T10:39:57.256056+0000","last_clean":"2024-02-26T10:39:57.256056+0000","last_became_active":"2024-02-26T10:39:56.145742+0000","last_became_peered":"2024-02-26T10:39:56.145742+0000","last_unstale":"2024-02-26T10:39:57.256056+0000","last_undegraded":"2024-02-26T10:39:57.256056+0000","last_fullsized":"2024-02-26T10:39:57.256056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:55:46.512272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.142239+0000","last_change":"2024-02-26T10:39:39.996527+0000","last_active":"2024-02-26T10:39:57.142239+0000","last_peered":"2024-02-26T10:39:57.142239+0000","last_clean":"2024-02-26T10:39:57.142239+0000","last_became_active":"2024-02-26T10:39:39.995687+0000","last_became_peered":"2024-02-26T10:39:39.995687+0000","last_unstale":"2024-02-26T10:39:57.142239+0000","last_undegraded":"2024-02-26T10:39:57.142239+0000","last_fullsized":"2024-02-26T10:39:57.142239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:26:29.964072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.255484+0000","last_change":"2024-02-26T10:39:56.163773+0000","last_active":"2024-02-26T10:39:57.255484+0000","last_peered":"2024-02-26T10:39:57.255484+0000","last_clean":"2024-02-26T10:39:57.255484+0000","last_became_active":"2024-02-26T10:39:56.163436+0000","last_became_peered":"2024-02-26T10:39:56.163436+0000","last_unstale":"2024-02-26T10:39:57.255484+0000","last_undegraded":"2024-02-26T10:39:57.255484+0000","last_fullsized":"2024-02-26T10:39:57.255484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:09:37.102414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.250921+0000","last_change":"2024-02-26T10:39:40.009578+0000","last_active":"2024-02-26T10:39:57.250921+0000","last_peered":"2024-02-26T10:39:57.250921+0000","last_clean":"2024-02-26T10:39:57.250921+0000","last_became_active":"2024-02-26T10:39:40.008770+0000","last_became_peered":"2024-02-26T10:39:40.008770+0000","last_unstale":"2024-02-26T10:39:57.250921+0000","last_undegraded":"2024-02-26T10:39:57.250921+0000","last_fullsized":"2024-02-26T10:39:57.250921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:48:42.311486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259802+0000","last_change":"2024-02-26T10:39:56.148486+0000","last_active":"2024-02-26T10:39:57.259802+0000","last_peered":"2024-02-26T10:39:57.259802+0000","last_clean":"2024-02-26T10:39:57.259802+0000","last_became_active":"2024-02-26T10:39:56.147714+0000","last_became_peered":"2024-02-26T10:39:56.147714+0000","last_unstale":"2024-02-26T10:39:57.259802+0000","last_undegraded":"2024-02-26T10:39:57.259802+0000","last_fullsized":"2024-02-26T10:39:57.259802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:09:37.102414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.254189+0000","last_change":"2024-02-26T10:39:40.017208+0000","last_active":"2024-02-26T10:39:57.254189+0000","last_peered":"2024-02-26T10:39:57.254189+0000","last_clean":"2024-02-26T10:39:57.254189+0000","last_became_active":"2024-02-26T10:39:40.016350+0000","last_became_peered":"2024-02-26T10:39:40.016350+0000","last_unstale":"2024-02-26T10:39:57.254189+0000","last_undegraded":"2024-02-26T10:39:57.254189+0000","last_fullsized":"2024-02-26T10:39:57.254189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:58:55.980334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.249756+0000","last_change":"2024-02-26T10:39:56.142505+0000","last_active":"2024-02-26T10:39:57.249756+0000","last_peered":"2024-02-26T10:39:57.249756+0000","last_clean":"2024-02-26T10:39:57.249756+0000","last_became_active":"2024-02-26T10:39:56.142202+0000","last_became_peered":"2024-02-26T10:39:56.142202+0000","last_unstale":"2024-02-26T10:39:57.249756+0000","last_undegraded":"2024-02-26T10:39:57.249756+0000","last_fullsized":"2024-02-26T10:39:57.249756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:35:59.513549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.141660+0000","last_change":"2024-02-26T10:39:40.000691+0000","last_active":"2024-02-26T10:39:57.141660+0000","last_peered":"2024-02-26T10:39:57.141660+0000","last_clean":"2024-02-26T10:39:57.141660+0000","last_became_active":"2024-02-26T10:39:40.000419+0000","last_became_peered":"2024-02-26T10:39:40.000419+0000","last_unstale":"2024-02-26T10:39:57.141660+0000","last_undegraded":"2024-02-26T10:39:57.141660+0000","last_fullsized":"2024-02-26T10:39:57.141660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:02.221077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.144703+0000","last_change":"2024-02-26T10:39:56.144006+0000","last_active":"2024-02-26T10:39:57.144703+0000","last_peered":"2024-02-26T10:39:57.144703+0000","last_clean":"2024-02-26T10:39:57.144703+0000","last_became_active":"2024-02-26T10:39:56.143570+0000","last_became_peered":"2024-02-26T10:39:56.143570+0000","last_unstale":"2024-02-26T10:39:57.144703+0000","last_undegraded":"2024-02-26T10:39:57.144703+0000","last_fullsized":"2024-02-26T10:39:57.144703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:55.088340+0000","last_clean_scrub_stamp":"2024-02-26T10:39:55.088340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:15.643339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-26T10:39:57.259084+0000","last_change":"2024-02-26T10:39:40.026044+0000","last_active":"2024-02-26T10:39:57.259084+0000","last_peered":"2024-02-26T10:39:57.259084+0000","last_clean":"2024-02-26T10:39:57.259084+0000","last_became_active":"2024-02-26T10:39:40.025716+0000","last_became_peered":"2024-02-26T10:39:40.025716+0000","last_unstale":"2024-02-26T10:39:57.259084+0000","last_undegraded":"2024-02-26T10:39:57.259084+0000","last_fullsized":"2024-02-26T10:39:57.259084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:39:38.957395+0000","last_clean_scrub_stamp":"2024-02-26T10:39:38.957395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:11:19.722135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3913,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":769042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":179782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":179236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":180210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":182269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542165,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":180198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":181701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1028096,"data_stored":768896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:40:01.213 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:40:01.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-02-26T10:40:01.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.483+0000 7f3de5000640 1 Processor -- start 2024-02-26T10:40:01.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.484+0000 7f3de5000640 1 -- start start 2024-02-26T10:40:01.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.485+0000 7f3de5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 0x7f3de0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.485+0000 7f3de5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00c0a00 con 0x7f3de00ab460 2024-02-26T10:40:01.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.485+0000 7f3de5000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de01550a0 con 0x7f3de00bf820 2024-02-26T10:40:01.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.485+0000 7f3de5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00c0160 con 0x7f3de0155580 2024-02-26T10:40:01.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.486+0000 7f3ddf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3de00ab460 0x7f3de006e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52986/0 (socket says 172.21.15.67:52986) 2024-02-26T10:40:01.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.486+0000 7f3ddf400640 1 -- 172.21.15.67:0/4075050574 learned_addr learned my addr 172.21.15.67:0/4075050574 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:01.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.486+0000 7f3ddfe00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 0x7f3de0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.486+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151538323 0 0) 0x7f3de00c0a00 con 0x7f3de00ab460 2024-02-26T10:40:01.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.486+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dc4003520 con 0x7f3de00ab460 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 957261225 0 0) 0x7f3dc4003520 con 0x7f3de00ab460 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 msgr2=0x7f3de0155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 0x7f3de0155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 >> v1:172.21.15.82:6789/0 conn(0x7f3de00bf820 legacy=0x7f3de00c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00c0440 con 0x7f3de00ab460 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f3dd0001670 con 0x7f3de00ab460 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 >> v1:172.21.15.67:6789/0 conn(0x7f3de00ab460 legacy=0x7f3de006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 0x7f3dc4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4003f90 0x7f3dc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4004d40 0x7f3dc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00c0160 con 0x7f3dc4003f90 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc400bbd0 con 0x7f3dc4004d40 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc400bdd0 con 0x7f3de0155580 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3dd7e00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4003f90 0x7f3dc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.487+0000 7f3ddf400640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 0x7f3dc4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.488+0000 7f3dd7e00640 1 -- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 msgr2=0x7f3dc4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:01.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.488+0000 7f3dd7e00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3de0155580 0x7f3dc4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.488+0000 7f3dd7e00640 1 -- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4004d40 msgr2=0x7f3dc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:01.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.488+0000 7f3dd7e00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4004d40 0x7f3dc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.488+0000 7f3dd7e00640 1 -- 172.21.15.67:0/4075050574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4003520 con 0x7f3dc4003f90 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.489+0000 7f3dd7e00640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4003f90 0x7f3dc4004800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8001910 tx=0x7f3dc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16444a1d61e241a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.489+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80014e0 con 0x7f3dc4003f90 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.489+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8037070 con 0x7f3dc4003f90 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.489+0000 7f3ddea00640 1 -- 172.21.15.67:0/4075050574 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8032880 con 0x7f3dc4003f90 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.489+0000 7f3de5000640 1 -- 172.21.15.67:0/4075050574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4003f90 msgr2=0x7f3dc4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.490+0000 7f3de5000640 1 --2- 172.21.15.67:0/4075050574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4003f90 0x7f3dc4004800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8001910 tx=0x7f3dc4005830 comp rx=0 tx=0).stop 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.490+0000 7f3ddfe00640 1 -- 172.21.15.67:0/4075050574 reap_dead start 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.490+0000 7f3de5000640 1 -- 172.21.15.67:0/4075050574 shutdown_connections 2024-02-26T10:40:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.490+0000 7f3de5000640 1 -- 172.21.15.67:0/4075050574 >> 172.21.15.67:0/4075050574 conn(0x7f3de00a84a0 msgr2=0x7f3de00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.490+0000 7f3de5000640 1 -- 172.21.15.67:0/4075050574 shutdown_connections 2024-02-26T10:40:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.490+0000 7f3de5000640 1 -- 172.21.15.67:0/4075050574 wait complete. 2024-02-26T10:40:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.491+0000 7f3de5000640 1 Processor -- start 2024-02-26T10:40:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 -- start start 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3dc4003f90 0x7f3de0152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3de011f570 0x7f3de0070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 0x7f3de0140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00c1e70 con 0x7f3de0072a90 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00c1cf0 con 0x7f3de011f570 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.492+0000 7f3de5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00ab840 con 0x7f3dc4003f90 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 0x7f3de0140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 0x7f3de0140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54240/0 (socket says 172.21.15.67:54240) 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 -- 172.21.15.67:0/1849114905 learned_addr learned my addr 172.21.15.67:0/1849114905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:01.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddf400640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3dc4003f90 0x7f3de0152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3dd7e00640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3de011f570 0x7f3de0070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 -- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3dc4003f90 msgr2=0x7f3de0152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3dc4003f90 0x7f3de0152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 -- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3de011f570 msgr2=0x7f3de0070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3de011f570 0x7f3de0070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3ddfe00640 1 -- 172.21.15.67:0/1849114905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0071cf0 con 0x7f3de0072a90 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.493+0000 7f3dd7e00640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3de011f570 0x7f3de0070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:01.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.494+0000 7f3ddfe00640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 0x7f3de0140580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800ee50 tx=0x7f3dd800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.494+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8018020 con 0x7f3de0072a90 2024-02-26T10:40:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.494+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800bbf0 con 0x7f3de0072a90 2024-02-26T10:40:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.494+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8014400 con 0x7f3de0072a90 2024-02-26T10:40:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.494+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de01ee970 con 0x7f3de0072a90 2024-02-26T10:40:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.494+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de0072290 con 0x7f3de0072a90 2024-02-26T10:40:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.495+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de0074a30 con 0x7f3de0072a90 2024-02-26T10:40:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.496+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3dd8005d10 con 0x7f3de0072a90 2024-02-26T10:40:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.497+0000 7f3dd7400640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3dbc06b4d0 0x7f3dbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.497+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3dd808ab30 con 0x7f3de0072a90 2024-02-26T10:40:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.498+0000 7f3ddf400640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3dbc06b4d0 0x7f3dbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:01.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.499+0000 7f3ddf400640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3dbc06b4d0 0x7f3dbc06d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00058b0 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.501+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3dd8022030 con 0x7f3de0072a90 2024-02-26T10:40:01.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.767+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7f3de00a8da0 con 0x7f3de0072a90 2024-02-26T10:40:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.772+0000 7f3dd7400640 1 -- 172.21.15.67:0/1849114905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v10) v1 ==== 185+0+142 (secure 0 0 0) 0x7f3dd8059640 con 0x7f3de0072a90 2024-02-26T10:40:01.773 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-02-26T10:40:01.773 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw== 2024-02-26T10:40:01.773 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-02-26T10:40:01.773 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:40:01.773 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw pool=cephfs2_data" 2024-02-26T10:40:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.1 2024-02-26T10:40:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.774+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3dbc06b4d0 msgr2=0x7f3dbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.774+0000 7f3de5000640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3dbc06b4d0 0x7f3dbc06d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00058b0 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2024-02-26T10:40:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.774+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 msgr2=0x7f3de0140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.774+0000 7f3de5000640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 0x7f3de0140580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800ee50 tx=0x7f3dd800b480 comp rx=0 tx=0).stop 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 shutdown_connections 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3de0072a90 0x7f3de0140580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3dbc06b4d0 0x7f3dbc06d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3de011f570 0x7f3de0070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 --2- 172.21.15.67:0/1849114905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3dc4003f90 0x7f3de0152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 >> 172.21.15.67:0/1849114905 conn(0x7f3de00a84a0 msgr2=0x7f3de0074310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 shutdown_connections 2024-02-26T10:40:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:01.775+0000 7f3de5000640 1 -- 172.21.15.67:0/1849114905 wait complete. 2024-02-26T10:40:01.793 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:01.793 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:01.850 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:01.850 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:01.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.128+0000 7f62f1000640 1 Processor -- start 2024-02-26T10:40:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.128+0000 7f62f1000640 1 -- start start 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62f1000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f62ec06ac20 0x7f62ec06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62f1000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62ec0c2310 0x7f62ec1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62f1000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62f1000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec155f10 con 0x7f62ec0c2310 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62f1000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0c28d0 con 0x7f62ec06ac20 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62f1000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec155150 con 0x7f62ec152f20 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.129+0000 7f62e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62ec0c2310 0x7f62ec1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62ec0c2310 0x7f62ec1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54252/0 (socket says 172.21.15.67:54252) 2024-02-26T10:40:02.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 -- 172.21.15.67:0/4211625870 learned_addr learned my addr 172.21.15.67:0/4211625870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:02.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62eb400640 1 --2- 172.21.15.67:0/4211625870 >> v2:172.21.15.82:3300/0 conn(0x7f62ec06ac20 0x7f62ec06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62ebe00640 1 --2- 172.21.15.67:0/4211625870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 -- 172.21.15.67:0/4211625870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 msgr2=0x7f62ec153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 --2- 172.21.15.67:0/4211625870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 -- 172.21.15.67:0/4211625870 >> v2:172.21.15.82:3300/0 conn(0x7f62ec06ac20 msgr2=0x7f62ec06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 --2- 172.21.15.67:0/4211625870 >> v2:172.21.15.82:3300/0 conn(0x7f62ec06ac20 0x7f62ec06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62e3e00640 1 -- 172.21.15.67:0/4211625870 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec155430 con 0x7f62ec0c2310 2024-02-26T10:40:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.130+0000 7f62eb400640 1 --2- 172.21.15.67:0/4211625870 >> v2:172.21.15.82:3300/0 conn(0x7f62ec06ac20 0x7f62ec06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:02.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.131+0000 7f62e3e00640 1 --2- 172.21.15.67:0/4211625870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec1524b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f62ec06ee00 tx=0x7f62d402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a6995444a208e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62eaa00640 1 -- 172.21.15.67:0/4211625870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4038070 con 0x7f62ec0c2310 2024-02-26T10:40:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62eaa00640 1 -- 172.21.15.67:0/4211625870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4031740 con 0x7f62ec0c2310 2024-02-26T10:40:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62eaa00640 1 -- 172.21.15.67:0/4211625870 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d402f050 con 0x7f62ec0c2310 2024-02-26T10:40:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 -- 172.21.15.67:0/4211625870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 msgr2=0x7f62ec1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 --2- 172.21.15.67:0/4211625870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec1524b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f62ec06ee00 tx=0x7f62d402b7e0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 -- 172.21.15.67:0/4211625870 shutdown_connections 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 --2- 172.21.15.67:0/4211625870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 --2- 172.21.15.67:0/4211625870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec1524b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 --2- 172.21.15.67:0/4211625870 >> v2:172.21.15.82:3300/0 conn(0x7f62ec06ac20 0x7f62ec06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 -- 172.21.15.67:0/4211625870 >> 172.21.15.67:0/4211625870 conn(0x7f62ec0c63f0 msgr2=0x7f62ec0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 -- 172.21.15.67:0/4211625870 shutdown_connections 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.132+0000 7f62f1000640 1 -- 172.21.15.67:0/4211625870 wait complete. 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.134+0000 7f62f1000640 1 Processor -- start 2024-02-26T10:40:02.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.134+0000 7f62f1000640 1 -- start start 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62f1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ec06ac20 0x7f62ec072590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62f1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62f1000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62f1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec148850 con 0x7f62ec0c2310 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62f1000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1559f0 con 0x7f62ec06ac20 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62f1000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1489d0 con 0x7f62ec152f20 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec0ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54260/0 (socket says 172.21.15.67:54260) 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.135+0000 7f62e3e00640 1 -- 172.21.15.67:0/332062350 learned_addr learned my addr 172.21.15.67:0/332062350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62eb400640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ec06ac20 0x7f62ec072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3e00640 1 -- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 msgr2=0x7f62ec146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62ebe00640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3e00640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3e00640 1 -- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ec06ac20 msgr2=0x7f62ec072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3e00640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ec06ac20 0x7f62ec072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3e00640 1 -- 172.21.15.67:0/332062350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec1444d0 con 0x7f62ec0c2310 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62eb400640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ec06ac20 0x7f62ec072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62ebe00640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3e00640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec0ab7b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f62d4000c00 tx=0x7f62d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4042020 con 0x7f62ec0c2310 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4033a70 con 0x7f62ec0c2310 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4038040 con 0x7f62ec0c2310 2024-02-26T10:40:02.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.136+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec1447b0 con 0x7f62ec0c2310 2024-02-26T10:40:02.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.137+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec072ad0 con 0x7f62ec0c2310 2024-02-26T10:40:02.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.138+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec146a60 con 0x7f62ec0c2310 2024-02-26T10:40:02.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.139+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62d4045020 con 0x7f62ec0c2310 2024-02-26T10:40:02.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.139+0000 7f62e3400640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f62c806b500 0x7f62c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.140+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f62d40ac260 con 0x7f62ec0c2310 2024-02-26T10:40:02.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.140+0000 7f62eb400640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f62c806b500 0x7f62c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.141+0000 7f62eb400640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f62c806b500 0x7f62c806d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b840 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:02.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.144+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62ec146a60 con 0x7f62ec0c2310 2024-02-26T10:40:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.350+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62ec0a95e0 con 0x7f62ec0c2310 2024-02-26T10:40:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.352+0000 7f62e3400640 1 -- 172.21.15.67:0/332062350 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+20323 (secure 0 0 0) 0x7f62d403b360 con 0x7f62ec0c2310 2024-02-26T10:40:02.355 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:02.355 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:57.124601+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:02.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f62c806b500 msgr2=0x7f62c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f62c806b500 0x7f62c806d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b840 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2024-02-26T10:40:02.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 msgr2=0x7f62ec0ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec0ab7b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f62d4000c00 tx=0x7f62d40046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 shutdown_connections 2024-02-26T10:40:02.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f62ec152f20 0x7f62ec146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ec0c2310 0x7f62ec0ab7b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f62c806b500 0x7f62c806d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 --2- 172.21.15.67:0/332062350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ec06ac20 0x7f62ec072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 >> 172.21.15.67:0/332062350 conn(0x7f62ec0c63f0 msgr2=0x7f62ec1527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:02.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 shutdown_connections 2024-02-26T10:40:02.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.356+0000 7f62f1000640 1 -- 172.21.15.67:0/332062350 wait complete. 2024-02-26T10:40:02.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:39:37.371486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:39:38.475801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:39:53.146836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:39:54.623593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:02.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:02.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.594+0000 7fd1a6000640 1 Processor -- start 2024-02-26T10:40:02.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.595+0000 7fd1a6000640 1 -- start start 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1a01503a0 0x7fd1a00ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a00c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a6000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0150780 con 0x7fd1a00a9430 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a00a98c0 con 0x7fd1a01503a0 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00a9ba0 con 0x7fd1a006e2d0 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1a01503a0 0x7fd1a00ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1a01503a0 0x7fd1a00ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54280/0 (socket says 172.21.15.67:54280) 2024-02-26T10:40:02.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.596+0000 7fd1a4c00640 1 -- 172.21.15.67:0/1255018086 learned_addr learned my addr 172.21.15.67:0/1255018086 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:02.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd1a5600640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a00c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd1a4c00640 1 -- 172.21.15.67:0/1255018086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 msgr2=0x7fd1a00c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:02.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd1a4c00640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a00c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd1a4c00640 1 -- 172.21.15.67:0/1255018086 >> v1:172.21.15.67:6789/0 conn(0x7fd1a00a9430 legacy=0x7fd1a01535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd1a4c00640 1 -- 172.21.15.67:0/1255018086 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a00a9e80 con 0x7fd1a01503a0 2024-02-26T10:40:02.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd1a5600640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a00c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:02.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.597+0000 7fd19f400640 1 -- 172.21.15.67:0/1255018086 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299630293 0 0) 0x7fd1a0150780 con 0x7fd1a00a9430 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.598+0000 7fd1a4c00640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a00ab480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd194009870 tx=0x7fd19402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5568cb8351733c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd19f400640 1 -- 172.21.15.67:0/1255018086 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194034070 con 0x7fd1a01503a0 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd19f400640 1 -- 172.21.15.67:0/1255018086 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd194030840 con 0x7fd1a01503a0 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd19f400640 1 -- 172.21.15.67:0/1255018086 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194030c10 con 0x7fd1a01503a0 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 -- 172.21.15.67:0/1255018086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 msgr2=0x7fd1a00ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a00ab480 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd194009870 tx=0x7fd19402b980 comp rx=0 tx=0).stop 2024-02-26T10:40:02.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 -- 172.21.15.67:0/1255018086 shutdown_connections 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a00c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 --2- 172.21.15.67:0/1255018086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a00ab480 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 -- 172.21.15.67:0/1255018086 >> 172.21.15.67:0/1255018086 conn(0x7fd1a00c6380 msgr2=0x7fd1a00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 -- 172.21.15.67:0/1255018086 shutdown_connections 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.599+0000 7fd1a6000640 1 -- 172.21.15.67:0/1255018086 wait complete. 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.601+0000 7fd1a6000640 1 Processor -- start 2024-02-26T10:40:02.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.601+0000 7fd1a6000640 1 -- start start 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd1a6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a01ec970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd1a6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a01e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd1a6000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a00c0890 0x7fd1a006ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd1a6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00c0f80 con 0x7fd1a01503a0 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd1a6000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a006ddb0 con 0x7fd1a00c0890 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd1a6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a006e740 con 0x7fd1a006e2d0 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a01e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd19fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a01e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54282/0 (socket says 172.21.15.67:54282) 2024-02-26T10:40:02.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.602+0000 7fd19fe00640 1 -- 172.21.15.67:0/4125241734 learned_addr learned my addr 172.21.15.67:0/4125241734 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd1a4c00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a01ec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd1a5600640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a00c0890 0x7fd1a006ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19fe00640 1 -- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 msgr2=0x7fd1a01ec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19fe00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a01ec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19fe00640 1 -- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a00c0890 msgr2=0x7fd1a006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19fe00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a00c0890 0x7fd1a006ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19fe00640 1 -- 172.21.15.67:0/4125241734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a01482d0 con 0x7fd1a01503a0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd1a4c00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a01ec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19fe00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a01e8880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c550 tx=0x7fd19000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190017070 con 0x7fd1a01503a0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.603+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1900035f0 con 0x7fd1a01503a0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.604+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a014baa0 con 0x7fd1a01503a0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.604+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900039e0 con 0x7fd1a01503a0 2024-02-26T10:40:02.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.604+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a01ec450 con 0x7fd1a01503a0 2024-02-26T10:40:02.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.605+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a00678e0 con 0x7fd1a01503a0 2024-02-26T10:40:02.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.606+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd19001a070 con 0x7fd1a01503a0 2024-02-26T10:40:02.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.607+0000 7fd19cc00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b4d0 0x7fd18406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:02.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.607+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd1900898d0 con 0x7fd1a01503a0 2024-02-26T10:40:02.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.608+0000 7fd1a4c00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b4d0 0x7fd18406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:02.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.609+0000 7fd1a4c00640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b4d0 0x7fd18406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd194036000 tx=0x7fd19403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:02.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.611+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1a014baa0 con 0x7fd1a01503a0 2024-02-26T10:40:02.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.811+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1a0074a30 con 0x7fd1a01503a0 2024-02-26T10:40:02.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.812+0000 7fd19cc00640 1 -- 172.21.15.67:0/4125241734 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd190058460 con 0x7fd1a01503a0 2024-02-26T10:40:02.813 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:02.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b4d0 msgr2=0x7fd18406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b4d0 0x7fd18406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd194036000 tx=0x7fd19403b040 comp rx=0 tx=0).stop 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 msgr2=0x7fd1a01e8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a01e8880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c550 tx=0x7fd19000ca10 comp rx=0 tx=0).stop 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 shutdown_connections 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b4d0 0x7fd18406d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a00c0890 0x7fd1a006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1a01503a0 0x7fd1a01e8880 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 --2- 172.21.15.67:0/4125241734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a006e2d0 0x7fd1a01ec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 >> 172.21.15.67:0/4125241734 conn(0x7fd1a00c6380 msgr2=0x7fd1a014ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 shutdown_connections 2024-02-26T10:40:02.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:02.815+0000 7fd1a6000640 1 -- 172.21.15.67:0/4125241734 wait complete. 2024-02-26T10:40:02.833 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:40:03.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.055+0000 7ffa3aa00640 1 Processor -- start 2024-02-26T10:40:03.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.055+0000 7ffa3aa00640 1 -- start start 2024-02-26T10:40:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa3aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa340c25a0 0x7ffa34155880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa3aa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa34151d70 con 0x7ffa3406e860 2024-02-26T10:40:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa3aa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa34152030 con 0x7ffa340a6150 2024-02-26T10:40:03.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa3aa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa34152b10 con 0x7ffa340c25a0 2024-02-26T10:40:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa2b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffa3406e860 0x7ffa3415a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53008/0 (socket says 172.21.15.67:53008) 2024-02-26T10:40:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa2b400640 1 -- 172.21.15.67:0/605840512 learned_addr learned my addr 172.21.15.67:0/605840512 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa38a00640 1 --2- 172.21.15.67:0/605840512 >> v2:172.21.15.82:3300/0 conn(0x7ffa340c25a0 0x7ffa34155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816015517 0 0) 0x7ffa34151d70 con 0x7ffa3406e860 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.057+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa18003520 con 0x7ffa3406e860 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1419486644 0 0) 0x7ffa18003520 con 0x7ffa3406e860 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 >> v2:172.21.15.82:3300/0 conn(0x7ffa340c25a0 msgr2=0x7ffa34155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 --2- 172.21.15.67:0/605840512 >> v2:172.21.15.82:3300/0 conn(0x7ffa340c25a0 0x7ffa34155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 >> v1:172.21.15.82:6789/0 conn(0x7ffa340a6150 legacy=0x7ffa34155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34152310 con 0x7ffa3406e860 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ffa1c001670 con 0x7ffa3406e860 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 >> v1:172.21.15.67:6789/0 conn(0x7ffa3406e860 legacy=0x7ffa3415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18004800 0x7ffa18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa180051c0 0x7ffa1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34152b10 con 0x7ffa18003a00 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1800c0c0 con 0x7ffa18004800 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1800c2c0 con 0x7ffa180051c0 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.058+0000 7ffa33e00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa38a00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa180051c0 0x7ffa1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa33e00640 1 -- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa180051c0 msgr2=0x7ffa1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa33e00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa180051c0 0x7ffa1800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa33e00640 1 -- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18004800 msgr2=0x7ffa18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa33e00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18004800 0x7ffa18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa33e00640 1 -- 172.21.15.67:0/605840512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa18003520 con 0x7ffa18003a00 2024-02-26T10:40:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.059+0000 7ffa38a00640 1 -- 172.21.15.67:0/605840512 reap_dead start 2024-02-26T10:40:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.061+0000 7ffa33e00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa18003e20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa24001a30 tx=0x7ffa18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e1227b95e349e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.061+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24001a70 con 0x7ffa18003a00 2024-02-26T10:40:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.061+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24003b70 con 0x7ffa18003a00 2024-02-26T10:40:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.061+0000 7ffa33400640 1 -- 172.21.15.67:0/605840512 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa240050d0 con 0x7ffa18003a00 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 -- 172.21.15.67:0/605840512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 msgr2=0x7ffa18003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa18003e20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa24001a30 tx=0x7ffa18005830 comp rx=0 tx=0).stop 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 -- 172.21.15.67:0/605840512 shutdown_connections 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/605840512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa18003e20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 -- 172.21.15.67:0/605840512 >> 172.21.15.67:0/605840512 conn(0x7ffa340c6da0 msgr2=0x7ffa340c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 -- 172.21.15.67:0/605840512 shutdown_connections 2024-02-26T10:40:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.062+0000 7ffa3aa00640 1 -- 172.21.15.67:0/605840512 wait complete. 2024-02-26T10:40:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.064+0000 7ffa3aa00640 1 Processor -- start 2024-02-26T10:40:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.064+0000 7ffa3aa00640 1 -- start start 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa3aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa341e6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa3aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa3414aa00 0x7ffa340c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa3aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa340c4b40 0x7ffa341e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa3aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3414f3a0 con 0x7ffa18003a00 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa3aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34152680 con 0x7ffa3414aa00 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa3aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3414f520 con 0x7ffa340c4b40 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa341e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa341e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54312/0 (socket says 172.21.15.67:54312) 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 -- 172.21.15.67:0/1481955680 learned_addr learned my addr 172.21.15.67:0/1481955680 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa2b400640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa3414aa00 0x7ffa340c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 -- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa340c4b40 msgr2=0x7ffa341e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa340c4b40 0x7ffa341e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 -- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa3414aa00 msgr2=0x7ffa340c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa3414aa00 0x7ffa340c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa33e00640 1 -- 172.21.15.67:0/1481955680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa340c4fc0 con 0x7ffa18003a00 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.065+0000 7ffa2b400640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa3414aa00 0x7ffa340c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.066+0000 7ffa33e00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa341e6870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa24001f40 tx=0x7ffa240014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.066+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24043070 con 0x7ffa18003a00 2024-02-26T10:40:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.066+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24003970 con 0x7ffa18003a00 2024-02-26T10:40:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.066+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24001a70 con 0x7ffa18003a00 2024-02-26T10:40:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.066+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa341e6e50 con 0x7ffa18003a00 2024-02-26T10:40:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.066+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa341e7410 con 0x7ffa18003a00 2024-02-26T10:40:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.067+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa34074a30 con 0x7ffa18003a00 2024-02-26T10:40:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.068+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa2402e5e0 con 0x7ffa18003a00 2024-02-26T10:40:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.070+0000 7ffa30c00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffa1006b500 0x7ffa1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.070+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ffa240ab420 con 0x7ffa18003a00 2024-02-26T10:40:03.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.070+0000 7ffa2b400640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffa1006b500 0x7ffa1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.071+0000 7ffa2b400640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffa1006b500 0x7ffa1006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa340c6410 tx=0x7ffa1c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.075+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa24003b40 con 0x7ffa18003a00 2024-02-26T10:40:03.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.277+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa340c0560 con 0x7ffa18003a00 2024-02-26T10:40:03.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.278+0000 7ffa30c00640 1 -- 172.21.15.67:0/1481955680 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa24079e10 con 0x7ffa18003a00 2024-02-26T10:40:03.279 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffa1006b500 msgr2=0x7ffa1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffa1006b500 0x7ffa1006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa340c6410 tx=0x7ffa1c009040 comp rx=0 tx=0).stop 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 msgr2=0x7ffa341e6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa341e6870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa24001f40 tx=0x7ffa240014a0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 shutdown_connections 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa340c4b40 0x7ffa341e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffa1006b500 0x7ffa1006d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa3414aa00 0x7ffa340c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 --2- 172.21.15.67:0/1481955680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18003a00 0x7ffa341e6870 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 >> 172.21.15.67:0/1481955680 conn(0x7ffa340c6da0 msgr2=0x7ffa34072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:03.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 shutdown_connections 2024-02-26T10:40:03.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.281+0000 7ffa3aa00640 1 -- 172.21.15.67:0/1481955680 wait complete. 2024-02-26T10:40:03.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:40:03.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.522+0000 7fcd2e400640 1 Processor -- start 2024-02-26T10:40:03.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.522+0000 7fcd2e400640 1 -- start start 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2e400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd28071710 0x7fcd280c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2e400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd28156320 0x7fcd28156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2e400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd28156cd0 con 0x7fcd2806e3d0 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2e400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd280c2a80 con 0x7fcd28071710 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2e400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2815d0b0 con 0x7fcd28156320 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd28071710 0x7fcd280c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd28071710 0x7fcd280c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54318/0 (socket says 172.21.15.67:54318) 2024-02-26T10:40:03.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.524+0000 7fcd2d000640 1 -- 172.21.15.67:0/1981795374 learned_addr learned my addr 172.21.15.67:0/1981795374 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:03.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.525+0000 7fcd2d000640 1 -- 172.21.15.67:0/1981795374 >> v2:172.21.15.82:3300/0 conn(0x7fcd28156320 msgr2=0x7fcd28156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.525+0000 7fcd2d000640 1 --2- 172.21.15.67:0/1981795374 >> v2:172.21.15.82:3300/0 conn(0x7fcd28156320 0x7fcd28156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.525+0000 7fcd2d000640 1 -- 172.21.15.67:0/1981795374 >> v1:172.21.15.82:6789/0 conn(0x7fcd2806e3d0 legacy=0x7fcd2815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.525+0000 7fcd2d000640 1 -- 172.21.15.67:0/1981795374 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2815d390 con 0x7fcd28071710 2024-02-26T10:40:03.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.526+0000 7fcd2d000640 1 --2- 172.21.15.67:0/1981795374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28071710 0x7fcd280c2540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcd280c7320 tx=0x7fcd1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cc93efc6d6ec6a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.526+0000 7fcd27400640 1 -- 172.21.15.67:0/1981795374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c02e070 con 0x7fcd28071710 2024-02-26T10:40:03.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.526+0000 7fcd27400640 1 -- 172.21.15.67:0/1981795374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c037070 con 0x7fcd28071710 2024-02-26T10:40:03.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.526+0000 7fcd27400640 1 -- 172.21.15.67:0/1981795374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c031a00 con 0x7fcd28071710 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 -- 172.21.15.67:0/1981795374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28071710 msgr2=0x7fcd280c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 --2- 172.21.15.67:0/1981795374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28071710 0x7fcd280c2540 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcd280c7320 tx=0x7fcd1c02b880 comp rx=0 tx=0).stop 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 -- 172.21.15.67:0/1981795374 shutdown_connections 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 --2- 172.21.15.67:0/1981795374 >> v2:172.21.15.82:3300/0 conn(0x7fcd28156320 0x7fcd28156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 --2- 172.21.15.67:0/1981795374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28071710 0x7fcd280c2540 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 -- 172.21.15.67:0/1981795374 >> 172.21.15.67:0/1981795374 conn(0x7fcd280c6380 msgr2=0x7fcd280c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 -- 172.21.15.67:0/1981795374 shutdown_connections 2024-02-26T10:40:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.527+0000 7fcd2e400640 1 -- 172.21.15.67:0/1981795374 wait complete. 2024-02-26T10:40:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.529+0000 7fcd2e400640 1 Processor -- start 2024-02-26T10:40:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.529+0000 7fcd2e400640 1 -- start start 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2e400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd2806e3d0 0x7fcd2814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2e400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28071710 0x7fcd2815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 0x7fcd281e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2815cd70 con 0x7fcd28156320 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2e400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2806deb0 con 0x7fcd28071710 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2e400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd28159290 con 0x7fcd2806e3d0 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 0x7fcd281e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 0x7fcd281e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54326/0 (socket says 172.21.15.67:54326) 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2da00640 1 -- 172.21.15.67:0/682620123 learned_addr learned my addr 172.21.15.67:0/682620123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd27e00640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28071710 0x7fcd2815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.530+0000 7fcd2d000640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd2806e3d0 0x7fcd2814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2da00640 1 -- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd2806e3d0 msgr2=0x7fcd2814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2da00640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd2806e3d0 0x7fcd2814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2da00640 1 -- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28071710 msgr2=0x7fcd2815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2da00640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28071710 0x7fcd2815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2da00640 1 -- 172.21.15.67:0/682620123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd280aa3c0 con 0x7fcd28156320 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2da00640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 0x7fcd281e3b60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000b9a0 tx=0x7fcd2000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20019070 con 0x7fcd28156320 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2000cbc0 con 0x7fcd28156320 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2814f280 con 0x7fcd28156320 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.531+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20014440 con 0x7fcd28156320 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.532+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd280aa0e0 con 0x7fcd28156320 2024-02-26T10:40:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.533+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd280678e0 con 0x7fcd28156320 2024-02-26T10:40:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.534+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd2001c070 con 0x7fcd28156320 2024-02-26T10:40:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.535+0000 7fcd24c00640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd0806b500 0x7fcd0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.535+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fcd2008a1c0 con 0x7fcd28156320 2024-02-26T10:40:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.535+0000 7fcd2d000640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd0806b500 0x7fcd0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.537+0000 7fcd2d000640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd0806b500 0x7fcd0806d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c036000 tx=0x7fcd1c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.539+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd20058690 con 0x7fcd28156320 2024-02-26T10:40:03.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.740+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd28074a30 con 0x7fcd28156320 2024-02-26T10:40:03.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.741+0000 7fcd24c00640 1 -- 172.21.15.67:0/682620123 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd2005c730 con 0x7fcd28156320 2024-02-26T10:40:03.741 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.743+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd0806b500 msgr2=0x7fcd0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.743+0000 7fcd2e400640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd0806b500 0x7fcd0806d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c036000 tx=0x7fcd1c03d040 comp rx=0 tx=0).stop 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.743+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 msgr2=0x7fcd281e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.743+0000 7fcd2e400640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 0x7fcd281e3b60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000b9a0 tx=0x7fcd2000be60 comp rx=0 tx=0).stop 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 shutdown_connections 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd28156320 0x7fcd281e3b60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd0806b500 0x7fcd0806d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28071710 0x7fcd2815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 --2- 172.21.15.67:0/682620123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd2806e3d0 0x7fcd2814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 >> 172.21.15.67:0/682620123 conn(0x7fcd280c6380 msgr2=0x7fcd280c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 shutdown_connections 2024-02-26T10:40:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.744+0000 7fcd2e400640 1 -- 172.21.15.67:0/682620123 wait complete. 2024-02-26T10:40:03.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:40:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.983+0000 7fc583e00640 1 Processor -- start 2024-02-26T10:40:03.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.984+0000 7fc583e00640 1 -- start start 2024-02-26T10:40:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.985+0000 7fc583e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5840a9430 0x7fc5840ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.985+0000 7fc583e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc58406e2d0 0x7fc5840c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.985+0000 7fc583e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc584150780 con 0x7fc5841503a0 2024-02-26T10:40:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.985+0000 7fc583e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5840a98c0 con 0x7fc5840a9430 2024-02-26T10:40:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.985+0000 7fc583e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5840a9ba0 con 0x7fc58406e2d0 2024-02-26T10:40:03.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5840a9430 0x7fc5840ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5840a9430 0x7fc5840ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54340/0 (socket says 172.21.15.67:54340) 2024-02-26T10:40:03.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 -- 172.21.15.67:0/2698460188 learned_addr learned my addr 172.21.15.67:0/2698460188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:03.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc583400640 1 --2- 172.21.15.67:0/2698460188 >> v2:172.21.15.82:3300/0 conn(0x7fc58406e2d0 0x7fc5840c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 -- 172.21.15.67:0/2698460188 >> v2:172.21.15.82:3300/0 conn(0x7fc58406e2d0 msgr2=0x7fc5840c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 --2- 172.21.15.67:0/2698460188 >> v2:172.21.15.82:3300/0 conn(0x7fc58406e2d0 0x7fc5840c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 -- 172.21.15.67:0/2698460188 >> v1:172.21.15.82:6789/0 conn(0x7fc5841503a0 legacy=0x7fc5841535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:03.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc57be00640 1 -- 172.21.15.67:0/2698460188 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840a9e80 con 0x7fc5840a9430 2024-02-26T10:40:03.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.986+0000 7fc583400640 1 --2- 172.21.15.67:0/2698460188 >> v2:172.21.15.82:3300/0 conn(0x7fc58406e2d0 0x7fc5840c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.988+0000 7fc57be00640 1 --2- 172.21.15.67:0/2698460188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5840a9430 0x7fc5840ab480 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc5840c3f40 tx=0x7fc56c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8179127264db29 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.988+0000 7fc582000640 1 -- 172.21.15.67:0/2698460188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c035020 con 0x7fc5840a9430 2024-02-26T10:40:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.988+0000 7fc582000640 1 -- 172.21.15.67:0/2698460188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c031870 con 0x7fc5840a9430 2024-02-26T10:40:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.988+0000 7fc582000640 1 -- 172.21.15.67:0/2698460188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c031c40 con 0x7fc5840a9430 2024-02-26T10:40:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.988+0000 7fc583e00640 1 -- 172.21.15.67:0/2698460188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5840a9430 msgr2=0x7fc5840ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.988+0000 7fc583e00640 1 --2- 172.21.15.67:0/2698460188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5840a9430 0x7fc5840ab480 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc5840c3f40 tx=0x7fc56c02b910 comp rx=0 tx=0).stop 2024-02-26T10:40:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.989+0000 7fc583e00640 1 -- 172.21.15.67:0/2698460188 shutdown_connections 2024-02-26T10:40:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.989+0000 7fc583e00640 1 --2- 172.21.15.67:0/2698460188 >> v2:172.21.15.82:3300/0 conn(0x7fc58406e2d0 0x7fc5840c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.989+0000 7fc583e00640 1 --2- 172.21.15.67:0/2698460188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5840a9430 0x7fc5840ab480 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.989+0000 7fc583e00640 1 -- 172.21.15.67:0/2698460188 >> 172.21.15.67:0/2698460188 conn(0x7fc5840c6380 msgr2=0x7fc5840c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.989+0000 7fc583e00640 1 -- 172.21.15.67:0/2698460188 shutdown_connections 2024-02-26T10:40:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.989+0000 7fc583e00640 1 -- 172.21.15.67:0/2698460188 wait complete. 2024-02-26T10:40:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.991+0000 7fc583e00640 1 Processor -- start 2024-02-26T10:40:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.991+0000 7fc583e00640 1 -- start start 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58406e2d0 0x7fc584147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5841503a0 0x7fc5841569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 0x7fc58414a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5840c0f80 con 0x7fc584149c40 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58406ddb0 con 0x7fc58406e2d0 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58406e720 con 0x7fc5841503a0 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 0x7fc58414a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 0x7fc58414a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54344/0 (socket says 172.21.15.67:54344) 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc583400640 1 -- 172.21.15.67:0/3210429396 learned_addr learned my addr 172.21.15.67:0/3210429396 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc57be00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5841503a0 0x7fc5841569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.992+0000 7fc582a00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58406e2d0 0x7fc584147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583400640 1 -- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5841503a0 msgr2=0x7fc5841569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583400640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5841503a0 0x7fc5841569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583400640 1 -- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58406e2d0 msgr2=0x7fc584147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583400640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58406e2d0 0x7fc584147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583400640 1 -- 172.21.15.67:0/3210429396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5841485f0 con 0x7fc584149c40 2024-02-26T10:40:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc57be00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5841503a0 0x7fc5841569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583400640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 0x7fc58414a0d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00cef0 tx=0x7fc57c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c01a070 con 0x7fc584149c40 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c00d860 con 0x7fc584149c40 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5841e87a0 con 0x7fc584149c40 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00dc50 con 0x7fc584149c40 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.993+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5841e8a20 con 0x7fc584149c40 2024-02-26T10:40:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.995+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5840678e0 con 0x7fc584149c40 2024-02-26T10:40:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.996+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc57c01d070 con 0x7fc584149c40 2024-02-26T10:40:03.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.997+0000 7fc57aa00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc56006b4d0 0x7fc56006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.997+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc57c08adc0 con 0x7fc584149c40 2024-02-26T10:40:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.998+0000 7fc582a00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc56006b4d0 0x7fc56006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:03.999+0000 7fc582a00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc56006b4d0 0x7fc56006d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc574006fd0 tx=0x7fc574008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.002+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5841e87a0 con 0x7fc584149c40 2024-02-26T10:40:04.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.217+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5840c29a0 con 0x7fc584149c40 2024-02-26T10:40:04.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.218+0000 7fc57aa00640 1 -- 172.21.15.67:0/3210429396 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v38) v1 ==== 105+0+11 (secure 0 0 0) 0x7fc5841485f0 con 0x7fc584149c40 2024-02-26T10:40:04.218 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.220+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc56006b4d0 msgr2=0x7fc56006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc56006b4d0 0x7fc56006d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc574006fd0 tx=0x7fc574008040 comp rx=0 tx=0).stop 2024-02-26T10:40:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 msgr2=0x7fc58414a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 0x7fc58414a0d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00cef0 tx=0x7fc57c00a710 comp rx=0 tx=0).stop 2024-02-26T10:40:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 shutdown_connections 2024-02-26T10:40:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc584149c40 0x7fc58414a0d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5841503a0 0x7fc5841569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc56006b4d0 0x7fc56006d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 --2- 172.21.15.67:0/3210429396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc58406e2d0 0x7fc584147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 >> 172.21.15.67:0/3210429396 conn(0x7fc5840c6380 msgr2=0x7fc5840c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 shutdown_connections 2024-02-26T10:40:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.221+0000 7fc583e00640 1 -- 172.21.15.67:0/3210429396 wait complete. 2024-02-26T10:40:04.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:40:04.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.471+0000 7fca9ea00640 1 Processor -- start 2024-02-26T10:40:04.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.471+0000 7fca9ea00640 1 -- start start 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca980c5290 0x7fca981562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca9806ce30 0x7fca980af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9806c110 con 0x7fca980c5290 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9ea00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca980c72c0 con 0x7fca9806ce30 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca980c75a0 con 0x7fca98155880 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca980c5290 0x7fca981562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fca980c5290 0x7fca981562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54358/0 (socket says 172.21.15.67:54358) 2024-02-26T10:40:04.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.473+0000 7fca9d600640 1 -- 172.21.15.67:0/4276208482 learned_addr learned my addr 172.21.15.67:0/4276208482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9e000640 1 --2- 172.21.15.67:0/4276208482 >> v2:172.21.15.82:3300/0 conn(0x7fca9806ce30 0x7fca980af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9d600640 1 -- 172.21.15.67:0/4276208482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 msgr2=0x7fca9806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9d600640 1 --2- 172.21.15.67:0/4276208482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9d600640 1 -- 172.21.15.67:0/4276208482 >> v2:172.21.15.82:3300/0 conn(0x7fca9806ce30 msgr2=0x7fca980af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9d600640 1 --2- 172.21.15.67:0/4276208482 >> v2:172.21.15.82:3300/0 conn(0x7fca9806ce30 0x7fca980af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9d600640 1 -- 172.21.15.67:0/4276208482 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca980c7880 con 0x7fca980c5290 2024-02-26T10:40:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.474+0000 7fca9e000640 1 --2- 172.21.15.67:0/4276208482 >> v2:172.21.15.82:3300/0 conn(0x7fca9806ce30 0x7fca980af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.475+0000 7fca9d600640 1 --2- 172.21.15.67:0/4276208482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca981562f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fca980c64d0 tx=0x7fca8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d329599332cd86a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.475+0000 7fca97e00640 1 -- 172.21.15.67:0/4276208482 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c02e070 con 0x7fca980c5290 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.475+0000 7fca97e00640 1 -- 172.21.15.67:0/4276208482 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c037070 con 0x7fca980c5290 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.475+0000 7fca97e00640 1 -- 172.21.15.67:0/4276208482 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c031a00 con 0x7fca980c5290 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 -- 172.21.15.67:0/4276208482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 msgr2=0x7fca981562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 --2- 172.21.15.67:0/4276208482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca981562f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fca980c64d0 tx=0x7fca8c02b880 comp rx=0 tx=0).stop 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 -- 172.21.15.67:0/4276208482 shutdown_connections 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 --2- 172.21.15.67:0/4276208482 >> v2:172.21.15.82:3300/0 conn(0x7fca9806ce30 0x7fca980af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 --2- 172.21.15.67:0/4276208482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 --2- 172.21.15.67:0/4276208482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca981562f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 -- 172.21.15.67:0/4276208482 >> 172.21.15.67:0/4276208482 conn(0x7fca980ac230 msgr2=0x7fca980ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 -- 172.21.15.67:0/4276208482 shutdown_connections 2024-02-26T10:40:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.476+0000 7fca9ea00640 1 -- 172.21.15.67:0/4276208482 wait complete. 2024-02-26T10:40:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.478+0000 7fca9ea00640 1 Processor -- start 2024-02-26T10:40:04.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.478+0000 7fca9ea00640 1 -- start start 2024-02-26T10:40:04.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca980c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9814e9e0 0x7fca98153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca980c5670 con 0x7fca980c5290 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca98155cc0 con 0x7fca9814e9e0 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca980c57f0 con 0x7fca98155880 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca980c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca980c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54374/0 (socket says 172.21.15.67:54374) 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.479+0000 7fca9d600640 1 -- 172.21.15.67:0/1032218334 learned_addr learned my addr 172.21.15.67:0/1032218334 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9e000640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9814e9e0 0x7fca98153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9cc00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9d600640 1 -- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 msgr2=0x7fca9814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9d600640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9814e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9d600640 1 -- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9814e9e0 msgr2=0x7fca98153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9d600640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9814e9e0 0x7fca98153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9d600640 1 -- 172.21.15.67:0/1032218334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca981eb330 con 0x7fca980c5290 2024-02-26T10:40:04.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca9d600640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca980c9530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fca8c000c00 tx=0x7fca8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.480+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c032880 con 0x7fca980c5290 2024-02-26T10:40:04.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.481+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c032a20 con 0x7fca980c5290 2024-02-26T10:40:04.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.481+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c032e10 con 0x7fca980c5290 2024-02-26T10:40:04.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.481+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca98152da0 con 0x7fca980c5290 2024-02-26T10:40:04.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.481+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca980c9a70 con 0x7fca980c5290 2024-02-26T10:40:04.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.482+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca98074a30 con 0x7fca980c5290 2024-02-26T10:40:04.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.483+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca8c04c060 con 0x7fca980c5290 2024-02-26T10:40:04.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.484+0000 7fca95600640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca7806b4d0 0x7fca7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.484+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fca8c0ac1f0 con 0x7fca980c5290 2024-02-26T10:40:04.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.484+0000 7fca9cc00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca7806b4d0 0x7fca7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.485+0000 7fca9cc00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca7806b4d0 0x7fca7806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca98155400 tx=0x7fca84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.488+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca8c040070 con 0x7fca980c5290 2024-02-26T10:40:04.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.687+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fca9806d950 con 0x7fca980c5290 2024-02-26T10:40:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.687+0000 7fca95600640 1 -- 172.21.15.67:0/1032218334 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v38) v1 ==== 101+0+11 (secure 0 0 0) 0x7fca8c07ad00 con 0x7fca980c5290 2024-02-26T10:40:04.689 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca7806b4d0 msgr2=0x7fca7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca7806b4d0 0x7fca7806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca98155400 tx=0x7fca84008040 comp rx=0 tx=0).stop 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 msgr2=0x7fca980c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca980c9530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fca8c000c00 tx=0x7fca8c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 shutdown_connections 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fca7806b4d0 0x7fca7806d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9814e9e0 0x7fca98153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca98155880 0x7fca9814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 --2- 172.21.15.67:0/1032218334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fca980c5290 0x7fca980c9530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 >> 172.21.15.67:0/1032218334 conn(0x7fca980ac230 msgr2=0x7fca980aa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 shutdown_connections 2024-02-26T10:40:04.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.690+0000 7fca9ea00640 1 -- 172.21.15.67:0/1032218334 wait complete. 2024-02-26T10:40:04.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:40:04.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.928+0000 7f3e3ea00640 1 Processor -- start 2024-02-26T10:40:04.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.929+0000 7f3e3ea00640 1 -- start start 2024-02-26T10:40:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.930+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 0x7f3e380aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.930+0000 7f3e3ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e38152e50 con 0x7f3e38150300 2024-02-26T10:40:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.930+0000 7f3e3ea00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e38153130 con 0x7f3e38156230 2024-02-26T10:40:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.930+0000 7f3e3ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381506e0 con 0x7f3e3806e410 2024-02-26T10:40:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.930+0000 7f3e37e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3e38150300 0x7f3e380c2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53018/0 (socket says 172.21.15.67:53018) 2024-02-26T10:40:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.930+0000 7f3e37e00640 1 -- 172.21.15.67:0/252140552 learned_addr learned my addr 172.21.15.67:0/252140552 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:04.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600219045 0 0) 0x7f3e38152e50 con 0x7f3e38150300 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e1c003520 con 0x7f3e38150300 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 0x7f3e380aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4292096063 0 0) 0x7f3e1c003520 con 0x7f3e38150300 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 msgr2=0x7f3e380aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 0x7f3e380aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 >> v1:172.21.15.82:6789/0 conn(0x7f3e38156230 legacy=0x7f3e380ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38153410 con 0x7f3e38150300 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.931+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f3e28001670 con 0x7f3e38150300 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 >> v1:172.21.15.67:6789/0 conn(0x7f3e38150300 legacy=0x7f3e380c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 0x7f3e1c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1c004d40 0x7f3e1c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381506e0 con 0x7f3e1c003fb0 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c00bc50 con 0x7f3e1c004d40 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c00be50 con 0x7f3e3806e410 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e37e00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 0x7f3e1c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e3ca00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1c004d40 0x7f3e1c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e2fe00640 1 -- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 msgr2=0x7f3e1c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3806e410 0x7f3e1c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.932+0000 7f3e2fe00640 1 -- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1c004d40 msgr2=0x7f3e1c0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.933+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1c004d40 0x7f3e1c0051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.933+0000 7f3e2fe00640 1 -- 172.21.15.67:0/252140552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c003520 con 0x7f3e1c003fb0 2024-02-26T10:40:04.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.933+0000 7f3e3ca00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1c004d40 0x7f3e1c0051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:04.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.934+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e1c004800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009a90 tx=0x7f3e1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116aa2d29a251bdd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.934+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20039070 con 0x7f3e1c003fb0 2024-02-26T10:40:04.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.934+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2002c680 con 0x7f3e1c003fb0 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.934+0000 7f3e37400640 1 -- 172.21.15.67:0/252140552 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20009690 con 0x7f3e1c003fb0 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ea00640 1 -- 172.21.15.67:0/252140552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 msgr2=0x7f3e1c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/252140552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e1c004800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009a90 tx=0x7f3e1c005830 comp rx=0 tx=0).stop 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ca00640 1 -- 172.21.15.67:0/252140552 reap_dead start 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ea00640 1 -- 172.21.15.67:0/252140552 shutdown_connections 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ea00640 1 -- 172.21.15.67:0/252140552 >> 172.21.15.67:0/252140552 conn(0x7f3e380a84a0 msgr2=0x7f3e380a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ea00640 1 -- 172.21.15.67:0/252140552 shutdown_connections 2024-02-26T10:40:04.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.935+0000 7f3e3ea00640 1 -- 172.21.15.67:0/252140552 wait complete. 2024-02-26T10:40:04.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.937+0000 7f3e3ea00640 1 Processor -- start 2024-02-26T10:40:04.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.937+0000 7f3e3ea00640 1 -- start start 2024-02-26T10:40:04.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e38151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e38152820 0x7f3e380c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e38144fe0 0x7f3e38145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380ab7f0 con 0x7f3e1c003fb0 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38153a60 con 0x7f3e38144fe0 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3806dd80 con 0x7f3e38152820 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e38151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e38151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54384/0 (socket says 172.21.15.67:54384) 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 -- 172.21.15.67:0/2645583109 learned_addr learned my addr 172.21.15.67:0/2645583109 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e38152820 0x7f3e380c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e3ca00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e38144fe0 0x7f3e38145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 -- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e38152820 msgr2=0x7f3e380c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e38152820 0x7f3e380c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 -- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e38144fe0 msgr2=0x7f3e38145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e38144fe0 0x7f3e38145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.938+0000 7f3e37e00640 1 -- 172.21.15.67:0/2645583109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e380c4bd0 con 0x7f3e1c003fb0 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.939+0000 7f3e37e00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e38151d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800e7e0 tx=0x7f3e2800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.939+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800ee90 con 0x7f3e1c003fb0 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.939+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e28018070 con 0x7f3e1c003fb0 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.939+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2800ca90 con 0x7f3e1c003fb0 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.939+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e380c4e50 con 0x7f3e1c003fb0 2024-02-26T10:40:04.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.939+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e381499d0 con 0x7f3e1c003fb0 2024-02-26T10:40:04.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.940+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e380c0bd0 con 0x7f3e1c003fb0 2024-02-26T10:40:04.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.941+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e2800cc30 con 0x7f3e1c003fb0 2024-02-26T10:40:04.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.942+0000 7f3e34c00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3e1406b670 0x7f3e1406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:04.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.942+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3e2808a340 con 0x7f3e1c003fb0 2024-02-26T10:40:04.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.943+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3e1406b670 0x7f3e1406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:04.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.944+0000 7f3e2fe00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3e1406b670 0x7f3e1406db30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3e20036000 tx=0x7f3e20030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:04.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:04.947+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e380c0bd0 con 0x7f3e1c003fb0 2024-02-26T10:40:05.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.161+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3e38074a30 con 0x7f3e1c003fb0 2024-02-26T10:40:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.162+0000 7f3e34c00640 1 -- 172.21.15.67:0/2645583109 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3e380c4bd0 con 0x7f3e1c003fb0 2024-02-26T10:40:05.163 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3e1406b670 msgr2=0x7f3e1406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3e1406b670 0x7f3e1406db30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3e20036000 tx=0x7f3e20030000 comp rx=0 tx=0).stop 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 msgr2=0x7f3e38151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e38151d80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800e7e0 tx=0x7f3e2800eca0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 shutdown_connections 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3e1406b670 0x7f3e1406db30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e38144fe0 0x7f3e38145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e38152820 0x7f3e380c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 --2- 172.21.15.67:0/2645583109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3e1c003fb0 0x7f3e38151d80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.165+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 >> 172.21.15.67:0/2645583109 conn(0x7f3e380a84a0 msgr2=0x7f3e3806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:05.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.166+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 shutdown_connections 2024-02-26T10:40:05.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.166+0000 7f3e3ea00640 1 -- 172.21.15.67:0/2645583109 wait complete. 2024-02-26T10:40:05.184 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:40:05.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:05.184 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:05.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:40:05.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:05.292 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:40:05.343 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:05.343 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:05.390 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:40:05.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:40:05.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.619+0000 7f8550e00640 1 Processor -- start 2024-02-26T10:40:05.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.619+0000 7f8550e00640 1 -- start start 2024-02-26T10:40:05.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.620+0000 7f8550e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f854c06b000 0x7f854c06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:05.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.620+0000 7f8550e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c0a9a80 con 0x7f854c06ac20 2024-02-26T10:40:05.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.620+0000 7f8550e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854c0a9d60 con 0x7f854c0a94a0 2024-02-26T10:40:05.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.620+0000 7f8550e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854c06b470 con 0x7f854c06b000 2024-02-26T10:40:05.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f854c06b000 0x7f854c06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:05.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f854c06b000 0x7f854c06db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54398/0 (socket says 172.21.15.67:54398) 2024-02-26T10:40:05.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854be00640 1 -- 172.21.15.67:0/3570106722 learned_addr learned my addr 172.21.15.67:0/3570106722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:05.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854b400640 1 --1- 172.21.15.67:0/3570106722 >> v1:172.21.15.67:6789/0 conn(0x7f854c06ac20 0x7f854c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53032/0 (socket says 172.21.15.67:53032) 2024-02-26T10:40:05.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854be00640 1 -- 172.21.15.67:0/3570106722 >> v1:172.21.15.82:6789/0 conn(0x7f854c0a94a0 legacy=0x7f854c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:05.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854be00640 1 -- 172.21.15.67:0/3570106722 >> v1:172.21.15.67:6789/0 conn(0x7f854c06ac20 legacy=0x7f854c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854be00640 1 -- 172.21.15.67:0/3570106722 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0ab750 con 0x7f854c06b000 2024-02-26T10:40:05.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.621+0000 7f854a000640 1 -- 172.21.15.67:0/3570106722 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741600153 0 0) 0x7f854c0a9a80 con 0x7f854c06ac20 2024-02-26T10:40:05.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.622+0000 7f854be00640 1 --2- 172.21.15.67:0/3570106722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06b000 0x7f854c06db00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8544009e50 tx=0x7f854402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441ca240d74f349b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:05.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f854a000640 1 -- 172.21.15.67:0/3570106722 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854402cb20 con 0x7f854c06b000 2024-02-26T10:40:05.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f854a000640 1 -- 172.21.15.67:0/3570106722 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854402ccc0 con 0x7f854c06b000 2024-02-26T10:40:05.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f854a000640 1 -- 172.21.15.67:0/3570106722 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544034610 con 0x7f854c06b000 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f8550e00640 1 -- 172.21.15.67:0/3570106722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06b000 msgr2=0x7f854c06db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f8550e00640 1 --2- 172.21.15.67:0/3570106722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06b000 0x7f854c06db00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8544009e50 tx=0x7f854402bf80 comp rx=0 tx=0).stop 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f8550e00640 1 -- 172.21.15.67:0/3570106722 shutdown_connections 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f8550e00640 1 --2- 172.21.15.67:0/3570106722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06b000 0x7f854c06db00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.623+0000 7f8550e00640 1 -- 172.21.15.67:0/3570106722 >> 172.21.15.67:0/3570106722 conn(0x7f854c0c63f0 msgr2=0x7f854c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.624+0000 7f8550e00640 1 -- 172.21.15.67:0/3570106722 shutdown_connections 2024-02-26T10:40:05.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.624+0000 7f8550e00640 1 -- 172.21.15.67:0/3570106722 wait complete. 2024-02-26T10:40:05.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.625+0000 7f8550e00640 1 Processor -- start 2024-02-26T10:40:05.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.625+0000 7f8550e00640 1 -- start start 2024-02-26T10:40:05.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f8550e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 0x7f854c0c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:05.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f8550e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854c06b000 0x7f854c1eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:05.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f8550e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854c0a94a0 0x7f854c1eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:05.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f8550e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c150590 con 0x7f854c06ac20 2024-02-26T10:40:05.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f8550e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c150410 con 0x7f854c0a94a0 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f8550e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854c15e520 con 0x7f854c06b000 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f854b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 0x7f854c0c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.626+0000 7f854b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 0x7f854c0c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54408/0 (socket says 172.21.15.67:54408) 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 -- 172.21.15.67:0/3609164003 learned_addr learned my addr 172.21.15.67:0/3609164003 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 -- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854c06b000 msgr2=0x7f854c1eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854c06b000 0x7f854c1eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 -- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854c0a94a0 msgr2=0x7f854c1eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854c0a94a0 0x7f854c1eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 -- 172.21.15.67:0/3609164003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c152960 con 0x7f854c06ac20 2024-02-26T10:40:05.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f854b400640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 0x7f854c0c1530 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c500 tx=0x7f853c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:05.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c018070 con 0x7f854c06ac20 2024-02-26T10:40:05.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c007560 con 0x7f854c06ac20 2024-02-26T10:40:05.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.627+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c004a20 con 0x7f854c06ac20 2024-02-26T10:40:05.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.628+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854c1523a0 con 0x7f854c06ac20 2024-02-26T10:40:05.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.628+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854c0c1a70 con 0x7f854c06ac20 2024-02-26T10:40:05.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.629+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854c074a30 con 0x7f854c06ac20 2024-02-26T10:40:05.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.630+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f853c01b070 con 0x7f854c06ac20 2024-02-26T10:40:05.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.631+0000 7f8543400640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852c06b4d0 0x7f852c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:05.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.631+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f853c089890 con 0x7f854c06ac20 2024-02-26T10:40:05.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.632+0000 7f854aa00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852c06b4d0 0x7f852c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:05.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.633+0000 7f854aa00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852c06b4d0 0x7f852c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8538006fd0 tx=0x7f8538006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:05.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.638+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f853c0583a0 con 0x7f854c06ac20 2024-02-26T10:40:05.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.875+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f854c15d990 con 0x7f854c06ac20 2024-02-26T10:40:05.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.875+0000 7f8543400640 1 -- 172.21.15.67:0/3609164003 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+149 (secure 0 0 0) 0x7f853c057ca0 con 0x7f854c06ac20 2024-02-26T10:40:05.876 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:40:05.876 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852c06b4d0 msgr2=0x7f852c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852c06b4d0 0x7f852c06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8538006fd0 tx=0x7f8538006bc0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 msgr2=0x7f854c0c1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 0x7f854c0c1530 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c500 tx=0x7f853c00c9c0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 shutdown_connections 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f852c06b4d0 0x7f852c06d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854c0a94a0 0x7f854c1eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854c06b000 0x7f854c1eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 --2- 172.21.15.67:0/3609164003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f854c06ac20 0x7f854c0c1530 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:05.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.878+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 >> 172.21.15.67:0/3609164003 conn(0x7f854c0c63f0 msgr2=0x7f854c0c2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:05.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.879+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 shutdown_connections 2024-02-26T10:40:05.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:05.879+0000 7f8550e00640 1 -- 172.21.15.67:0/3609164003 wait complete. 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:40:05.899 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:40:05.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:05.900 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:40:05.957 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:40:05.957 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:40:05.957 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:40:05.958 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42273sec preferred_lft 42273sec 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:40:05.959 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:05.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:05.960 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:40:06.049 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:40:06.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.054 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:40:06.105 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:40:06.105 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:40:06.106 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:40:06.106 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:40:06.106 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:40:06.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.106 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:40:06.106 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:40:06.106 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:40:06.106 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:40:06.106 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:40:06.106 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:40:06.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.287 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:40:06.334 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:40:06.335 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:40:06.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:40:06.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:06.419 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:06.464 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:40:06.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:06.464 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:06.558 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs2=/ 2024-02-26T10:40:06.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.558 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:06.691 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:06.691 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:06.726 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:06.726 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:06.726 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:40:06.726 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs2=/ 2024-02-26T10:40:06.726 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:40:06.728 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:40:06.728 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:40:06.728 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:40:06.728 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:40:06.728 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:40:06.732 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:40:06.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.732 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:06.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:06.817 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:40:06.885 INFO:teuthology.orchestra.run.smithi082.stdout:client4954 2024-02-26T10:40:06.885 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:06.885 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:06.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs authorize cephfs client.testuser / rw 2024-02-26T10:40:07.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.170+0000 7ff870e00640 1 Processor -- start 2024-02-26T10:40:07.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.170+0000 7ff870e00640 1 -- start start 2024-02-26T10:40:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.172+0000 7ff870e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff86c072770 0x7ff86c0c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.172+0000 7ff870e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c071d00 con 0x7ff86c0a89f0 2024-02-26T10:40:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.172+0000 7ff870e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c0c21c0 con 0x7ff86c0c0830 2024-02-26T10:40:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.172+0000 7ff870e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86c072c00 con 0x7ff86c072770 2024-02-26T10:40:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.172+0000 7ff86be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff86c0a89f0 0x7ff86c15f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53034/0 (socket says 172.21.15.67:53034) 2024-02-26T10:40:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.172+0000 7ff86be00640 1 -- 172.21.15.67:0/3851191635 learned_addr learned my addr 172.21.15.67:0/3851191635 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff862a00640 1 --2- 172.21.15.67:0/3851191635 >> v2:172.21.15.82:3300/0 conn(0x7ff86c072770 0x7ff86c0c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521637281 0 0) 0x7ff86c071d00 con 0x7ff86c0a89f0 2024-02-26T10:40:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff850003520 con 0x7ff86c0a89f0 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635390822 0 0) 0x7ff850003520 con 0x7ff86c0a89f0 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 >> v2:172.21.15.82:3300/0 conn(0x7ff86c072770 msgr2=0x7ff86c0c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 --2- 172.21.15.67:0/3851191635 >> v2:172.21.15.82:3300/0 conn(0x7ff86c072770 0x7ff86c0c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 >> v1:172.21.15.82:6789/0 conn(0x7ff86c0c0830 legacy=0x7ff86c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c071f70 con 0x7ff86c0a89f0 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ff8640019f0 con 0x7ff86c0a89f0 2024-02-26T10:40:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 >> v1:172.21.15.67:6789/0 conn(0x7ff86c0a89f0 legacy=0x7ff86c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff850003a00 0x7ff850003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.173+0000 7ff86aa00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86aa00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff850005210 0x7ff85000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000c130 con 0x7ff850004800 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000c2e0 con 0x7ff850005210 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000c4e0 con 0x7ff850003a00 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff862a00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff862a00640 1 -- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff850003a00 msgr2=0x7ff850003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86b400640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff850003a00 0x7ff850003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86be00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff850005210 0x7ff85000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff862a00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff850003a00 0x7ff850003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff862a00640 1 -- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff850005210 msgr2=0x7ff85000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff862a00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff850005210 0x7ff85000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff862a00640 1 -- 172.21.15.67:0/3851191635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850003520 con 0x7ff850004800 2024-02-26T10:40:07.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86b400640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff850003a00 0x7ff850003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:07.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86be00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff850005210 0x7ff85000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:07.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.174+0000 7ff86be00640 1 -- 172.21.15.67:0/3851191635 reap_dead start 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.176+0000 7ff862a00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff850004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff854009a90 tx=0x7ff850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd964261316aeaf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.176+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854039070 con 0x7ff850004800 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.176+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85402c680 con 0x7ff850004800 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.176+0000 7ff86aa00640 1 -- 172.21.15.67:0/3851191635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854009690 con 0x7ff850004800 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.176+0000 7ff870e00640 1 -- 172.21.15.67:0/3851191635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 msgr2=0x7ff850004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.176+0000 7ff870e00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff850004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff854009a90 tx=0x7ff850005830 comp rx=0 tx=0).stop 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.177+0000 7ff870e00640 1 -- 172.21.15.67:0/3851191635 shutdown_connections 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.177+0000 7ff870e00640 1 --2- 172.21.15.67:0/3851191635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff850004c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.177+0000 7ff870e00640 1 -- 172.21.15.67:0/3851191635 >> 172.21.15.67:0/3851191635 conn(0x7ff86c0c3f00 msgr2=0x7ff86c0c4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.177+0000 7ff870e00640 1 -- 172.21.15.67:0/3851191635 shutdown_connections 2024-02-26T10:40:07.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.177+0000 7ff870e00640 1 -- 172.21.15.67:0/3851191635 wait complete. 2024-02-26T10:40:07.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.178+0000 7ff870e00640 1 Processor -- start 2024-02-26T10:40:07.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.178+0000 7ff870e00640 1 -- start start 2024-02-26T10:40:07.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.179+0000 7ff870e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff86c1eb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.179+0000 7ff870e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff86c158810 0x7ff86c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.179+0000 7ff870e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff86c06ac20 0x7ff86c1edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.179+0000 7ff870e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c13e460 con 0x7ff850004800 2024-02-26T10:40:07.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.179+0000 7ff870e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c0c1750 con 0x7ff86c158810 2024-02-26T10:40:07.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.179+0000 7ff870e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c13e5e0 con 0x7ff86c06ac20 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff86c1eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff86c1eb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54428/0 (socket says 172.21.15.67:54428) 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 -- 172.21.15.67:0/234734248 learned_addr learned my addr 172.21.15.67:0/234734248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 -- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff86c06ac20 msgr2=0x7ff86c1edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff86c06ac20 0x7ff86c1edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 -- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff86c158810 msgr2=0x7ff86c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff86c158810 0x7ff86c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 -- 172.21.15.67:0/234734248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c1eaa60 con 0x7ff850004800 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.180+0000 7ff86b400640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff86c1eb520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00d4b0 tx=0x7ff85c00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.181+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c005480 con 0x7ff850004800 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.181+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c017070 con 0x7ff850004800 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.181+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c003850 con 0x7ff850004800 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.181+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c1dec20 con 0x7ff850004800 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.181+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c1df1e0 con 0x7ff850004800 2024-02-26T10:40:07.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.182+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c074a30 con 0x7ff850004800 2024-02-26T10:40:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.183+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff85c0039f0 con 0x7ff850004800 2024-02-26T10:40:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.184+0000 7ff863e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff84806b4d0 0x7ff84806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.184+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff85c08a240 con 0x7ff850004800 2024-02-26T10:40:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.184+0000 7ff862a00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff84806b4d0 0x7ff84806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.186+0000 7ff862a00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff84806b4d0 0x7ff84806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff854036000 tx=0x7ff85402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:07.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.188+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff85c058e10 con 0x7ff850004800 2024-02-26T10:40:07.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.425+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.testuser", "caps": ["/", "rw"]} v 0) v1 -- 0x7ff86c1526f0 con 0x7ff850004800 2024-02-26T10:40:07.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.430+0000 7ff863e00640 1 -- 172.21.15.67:0/234734248 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.testuser", "caps": ["/", "rw"]}]=0 v11) v1 ==== 134+0+185 (secure 0 0 0) 0x7ff85c058710 con 0x7ff850004800 2024-02-26T10:40:07.430 INFO:teuthology.orchestra.run.smithi067.stdout:[client.testuser] 2024-02-26T10:40:07.431 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQCHatxl8b54GRAAIFSuHYyYQln561YAeiB2Hg== 2024-02-26T10:40:07.431 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow rw fsname=cephfs" 2024-02-26T10:40:07.431 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r fsname=cephfs" 2024-02-26T10:40:07.431 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-02-26T10:40:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff84806b4d0 msgr2=0x7ff84806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff84806b4d0 0x7ff84806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff854036000 tx=0x7ff85402e000 comp rx=0 tx=0).stop 2024-02-26T10:40:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 msgr2=0x7ff86c1eb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff86c1eb520 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00d4b0 tx=0x7ff85c00d970 comp rx=0 tx=0).stop 2024-02-26T10:40:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 shutdown_connections 2024-02-26T10:40:07.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff86c06ac20 0x7ff86c1edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff84806b4d0 0x7ff84806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.432+0000 7ff870e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff86c158810 0x7ff86c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.433+0000 7ff870e00640 1 --2- 172.21.15.67:0/234734248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff850004800 0x7ff86c1eb520 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.433+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 >> 172.21.15.67:0/234734248 conn(0x7ff86c0c3f00 msgr2=0x7ff86c0c0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.433+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 shutdown_connections 2024-02-26T10:40:07.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.433+0000 7ff870e00640 1 -- 172.21.15.67:0/234734248 wait complete. 2024-02-26T10:40:07.452 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:40:07.506 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.t74yb7Egfh 2024-02-26T10:40:07.507 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:07.507 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/tmp/tmp.t74yb7Egfh 2024-02-26T10:40:07.561 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:07.561 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:07.661 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:07.661 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:07.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:07.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.942+0000 7f45f3e00640 1 Processor -- start 2024-02-26T10:40:07.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.942+0000 7f45f3e00640 1 -- start start 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.944+0000 7f45f3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f406b000 0x7f45f406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.944+0000 7f45f3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40a9a80 con 0x7f45f406ac20 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.944+0000 7f45f3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40a9d60 con 0x7f45f40a94a0 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.944+0000 7f45f3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f406b490 con 0x7f45f406b000 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.944+0000 7f45f2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f45f406ac20 0x7f45f4153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53040/0 (socket says 172.21.15.67:53040) 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.944+0000 7f45f2a00640 1 -- 172.21.15.67:0/1857489747 learned_addr learned my addr 172.21.15.67:0/1857489747 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:07.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f3400640 1 --2- 172.21.15.67:0/1857489747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f406b000 0x7f45f406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f1600640 1 -- 172.21.15.67:0/1857489747 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246832293 0 0) 0x7f45f40a9a80 con 0x7f45f406ac20 2024-02-26T10:40:07.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f1600640 1 -- 172.21.15.67:0/1857489747 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d8003520 con 0x7f45f406ac20 2024-02-26T10:40:07.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f3400640 1 -- 172.21.15.67:0/1857489747 >> v1:172.21.15.82:6789/0 conn(0x7f45f40a94a0 legacy=0x7f45f40abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f1600640 1 -- 172.21.15.67:0/1857489747 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477288620 0 0) 0x7f45f40a9d60 con 0x7f45f40a94a0 2024-02-26T10:40:07.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f3400640 1 -- 172.21.15.67:0/1857489747 >> v1:172.21.15.67:6789/0 conn(0x7f45f406ac20 legacy=0x7f45f4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.945+0000 7f45f3400640 1 -- 172.21.15.67:0/1857489747 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40ab750 con 0x7f45f406b000 2024-02-26T10:40:07.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.947+0000 7f45f3400640 1 --2- 172.21.15.67:0/1857489747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f406b000 0x7f45f406db00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009dc0 tx=0x7f45ec02bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d091cc40e5cc7b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:07.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.947+0000 7f45f1600640 1 -- 172.21.15.67:0/1857489747 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec039070 con 0x7f45f406b000 2024-02-26T10:40:07.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.947+0000 7f45f1600640 1 -- 172.21.15.67:0/1857489747 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec02c840 con 0x7f45f406b000 2024-02-26T10:40:07.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.947+0000 7f45f1600640 1 -- 172.21.15.67:0/1857489747 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4001670 con 0x7f45f406b000 2024-02-26T10:40:07.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 -- 172.21.15.67:0/1857489747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f406b000 msgr2=0x7f45f406db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 --2- 172.21.15.67:0/1857489747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f406b000 0x7f45f406db00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009dc0 tx=0x7f45ec02bd10 comp rx=0 tx=0).stop 2024-02-26T10:40:07.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 -- 172.21.15.67:0/1857489747 shutdown_connections 2024-02-26T10:40:07.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 --2- 172.21.15.67:0/1857489747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f406b000 0x7f45f406db00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 -- 172.21.15.67:0/1857489747 >> 172.21.15.67:0/1857489747 conn(0x7f45f40c63f0 msgr2=0x7f45f40c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:07.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 -- 172.21.15.67:0/1857489747 shutdown_connections 2024-02-26T10:40:07.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.948+0000 7f45f3e00640 1 -- 172.21.15.67:0/1857489747 wait complete. 2024-02-26T10:40:07.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.950+0000 7f45f3e00640 1 Processor -- start 2024-02-26T10:40:07.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.950+0000 7f45f3e00640 1 -- start start 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f406ac20 0x7f45f4072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 0x7f45f41442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40a94a0 0x7f45f4070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4150590 con 0x7f45f406b000 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4150410 con 0x7f45f406ac20 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f415e520 con 0x7f45f40a94a0 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 0x7f45f41442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 0x7f45f41442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54438/0 (socket says 172.21.15.67:54438) 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.951+0000 7f45f2000640 1 -- 172.21.15.67:0/381030706 learned_addr learned my addr 172.21.15.67:0/381030706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:07.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2a00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f406ac20 0x7f45f4072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f3400640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40a94a0 0x7f45f4070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2000640 1 -- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40a94a0 msgr2=0x7f45f4070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2000640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40a94a0 0x7f45f4070c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2000640 1 -- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f406ac20 msgr2=0x7f45f4072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2000640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f406ac20 0x7f45f4072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2000640 1 -- 172.21.15.67:0/381030706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40c5d80 con 0x7f45f406b000 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2a00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f406ac20 0x7f45f4072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f2000640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 0x7f45f41442e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c410 tx=0x7f45e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0017070 con 0x7f45f406b000 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e00035f0 con 0x7f45f406b000 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e00039e0 con 0x7f45f406b000 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.952+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f41512f0 con 0x7f45f406b000 2024-02-26T10:40:07.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.953+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4144dd0 con 0x7f45f406b000 2024-02-26T10:40:07.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.954+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f41510e0 con 0x7f45f406b000 2024-02-26T10:40:07.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.955+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45e001a070 con 0x7f45f406b000 2024-02-26T10:40:07.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.956+0000 7f45eaa00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:07.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.956+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f45e00899c0 con 0x7f45f406b000 2024-02-26T10:40:07.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.957+0000 7f45f2a00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:07.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.958+0000 7f45f2a00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f45d406b4d0 0x7f45d406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000c00 tx=0x7f45e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:07.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:07.961+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45e00584d0 con 0x7f45f406b000 2024-02-26T10:40:08.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.159+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f4074a30 con 0x7f45f406b000 2024-02-26T10:40:08.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.163+0000 7f45eaa00640 1 -- 172.21.15.67:0/381030706 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+20323 (secure 0 0 0) 0x7f45e0057dd0 con 0x7f45f406b000 2024-02-26T10:40:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:08.164 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:57.124601+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:08.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f45d406b4d0 msgr2=0x7f45d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f45d406b4d0 0x7f45d406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000c00 tx=0x7f45e4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:08.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 msgr2=0x7f45f41442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 0x7f45f41442e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c410 tx=0x7f45e000c8d0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 shutdown_connections 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40a94a0 0x7f45f4070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45f406b000 0x7f45f41442e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 --2- 172.21.15.67:0/381030706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f406ac20 0x7f45f4072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 >> 172.21.15.67:0/381030706 conn(0x7f45f40c63f0 msgr2=0x7f45f414f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 shutdown_connections 2024-02-26T10:40:08.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.166+0000 7f45f3e00640 1 -- 172.21.15.67:0/381030706 wait complete. 2024-02-26T10:40:08.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:39:37.371486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:39:38.475801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:39:53.146836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:39:54.623593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:08.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:08.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.404+0000 7fa5a3200640 1 Processor -- start 2024-02-26T10:40:08.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.404+0000 7fa5a3200640 1 -- start start 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.405+0000 7fa5a3200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5a4156500 0x7fa5a4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.405+0000 7fa5a3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 0x7fa5a4154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.405+0000 7fa5a3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a4157210 con 0x7fa5a41568e0 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.405+0000 7fa5a3200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a40c4550 con 0x7fa5a4156500 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.405+0000 7fa5a3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c4830 con 0x7fa5a4156d40 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.405+0000 7fa5a1400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa5a41568e0 0x7fa5a414ffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53052/0 (socket says 172.21.15.67:53052) 2024-02-26T10:40:08.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a1400640 1 -- 172.21.15.67:0/3141650297 learned_addr learned my addr 172.21.15.67:0/3141650297 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a2800640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 0x7fa5a4154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3141650297 >> v2:172.21.15.82:3300/0 conn(0x7fa5a4156500 0x7fa5a4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697746070 0 0) 0x7fa5a4157210 con 0x7fa5a41568e0 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa584003520 con 0x7fa5a41568e0 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678202805 0 0) 0x7fa584003520 con 0x7fa5a41568e0 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 msgr2=0x7fa5a4154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.406+0000 7fa5a0a00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 0x7fa5a4154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 >> v2:172.21.15.82:3300/0 conn(0x7fa5a4156500 msgr2=0x7fa5a4070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 --2- 172.21.15.67:0/3141650297 >> v2:172.21.15.82:3300/0 conn(0x7fa5a4156500 0x7fa5a4070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a406ee30 con 0x7fa5a41568e0 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fa58c0019f0 con 0x7fa5a41568e0 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 >> v1:172.21.15.67:6789/0 conn(0x7fa5a41568e0 legacy=0x7fa5a414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5840039b0 0x7fa584003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 0x7fa584004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584004d40 0x7fa5840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c4550 con 0x7fa5840039b0 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c4830 con 0x7fa584004d40 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58400b9f0 con 0x7fa5a4156d40 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5840039b0 0x7fa584003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a2800640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584004d40 0x7fa5840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a1e00640 1 -- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 msgr2=0x7fa584004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4156d40 0x7fa584004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a1e00640 1 -- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584004d40 msgr2=0x7fa5840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa584004d40 0x7fa5840051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.407+0000 7fa5a1e00640 1 -- 172.21.15.67:0/3141650297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584003520 con 0x7fa5840039b0 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5840039b0 0x7fa584003dd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa594009ac0 tx=0x7fa584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab4ccf5bd2891f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594039070 con 0x7fa5840039b0 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59402c710 con 0x7fa5840039b0 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a0a00640 1 -- 172.21.15.67:0/3141650297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594009690 con 0x7fa5840039b0 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a3200640 1 -- 172.21.15.67:0/3141650297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5840039b0 msgr2=0x7fa584003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3141650297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5840039b0 0x7fa584003dd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa594009ac0 tx=0x7fa584005830 comp rx=0 tx=0).stop 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.409+0000 7fa5a2800640 1 -- 172.21.15.67:0/3141650297 reap_dead start 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.410+0000 7fa5a3200640 1 -- 172.21.15.67:0/3141650297 shutdown_connections 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.410+0000 7fa5a3200640 1 -- 172.21.15.67:0/3141650297 >> 172.21.15.67:0/3141650297 conn(0x7fa5a40c84a0 msgr2=0x7fa5a40c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.410+0000 7fa5a3200640 1 -- 172.21.15.67:0/3141650297 shutdown_connections 2024-02-26T10:40:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.410+0000 7fa5a3200640 1 -- 172.21.15.67:0/3141650297 wait complete. 2024-02-26T10:40:08.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.412+0000 7fa5a3200640 1 Processor -- start 2024-02-26T10:40:08.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.412+0000 7fa5a3200640 1 -- start start 2024-02-26T10:40:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5840039b0 0x7fa5a406d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 0x7fa5a41ea210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4147980 0x7fa5a41ea750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a406f4d0 con 0x7fa5a4147450 2024-02-26T10:40:08.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a406f350 con 0x7fa5840039b0 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a41506b0 con 0x7fa5a4147980 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 0x7fa5a41ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 0x7fa5a41ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54446/0 (socket says 172.21.15.67:54446) 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.413+0000 7fa5a1400640 1 -- 172.21.15.67:0/3838350714 learned_addr learned my addr 172.21.15.67:0/3838350714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5840039b0 0x7fa5a406d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a2800640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4147980 0x7fa5a41ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1400640 1 -- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4147980 msgr2=0x7fa5a41ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1400640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4147980 0x7fa5a41ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1400640 1 -- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5840039b0 msgr2=0x7fa5a406d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1400640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5840039b0 0x7fa5a406d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1400640 1 -- 172.21.15.67:0/3838350714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40c76c0 con 0x7fa5a4147450 2024-02-26T10:40:08.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa5a1400640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 0x7fa5a41ea210 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00caa0 tx=0x7fa58c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c018070 con 0x7fa5a4147450 2024-02-26T10:40:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00a730 con 0x7fa5a4147450 2024-02-26T10:40:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.414+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00ab20 con 0x7fa5a4147450 2024-02-26T10:40:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.415+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a40c8110 con 0x7fa5a4147450 2024-02-26T10:40:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.415+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a406d660 con 0x7fa5a4147450 2024-02-26T10:40:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.416+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a4075020 con 0x7fa5a4147450 2024-02-26T10:40:08.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.417+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa58c00acc0 con 0x7fa5a4147450 2024-02-26T10:40:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.418+0000 7fa59a000640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa58006b4d0 0x7fa58006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.418+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa58c089e00 con 0x7fa5a4147450 2024-02-26T10:40:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.418+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa58006b4d0 0x7fa58006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.419+0000 7fa5a1e00640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa58006b4d0 0x7fa58006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa594036000 tx=0x7fa594030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.422+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5a4075020 con 0x7fa5a4147450 2024-02-26T10:40:08.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.621+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5a40c6f50 con 0x7fa5a4147450 2024-02-26T10:40:08.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.622+0000 7fa59a000640 1 -- 172.21.15.67:0/3838350714 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa58c058910 con 0x7fa5a4147450 2024-02-26T10:40:08.622 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.624+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa58006b4d0 msgr2=0x7fa58006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa58006b4d0 0x7fa58006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa594036000 tx=0x7fa594030000 comp rx=0 tx=0).stop 2024-02-26T10:40:08.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 msgr2=0x7fa5a41ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 0x7fa5a41ea210 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00caa0 tx=0x7fa58c00cf60 comp rx=0 tx=0).stop 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 shutdown_connections 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5a4147980 0x7fa5a41ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a4147450 0x7fa5a41ea210 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa58006b4d0 0x7fa58006d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 --2- 172.21.15.67:0/3838350714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5840039b0 0x7fa5a406d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 >> 172.21.15.67:0/3838350714 conn(0x7fa5a40c84a0 msgr2=0x7fa5a40c2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 shutdown_connections 2024-02-26T10:40:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.625+0000 7fa5a3200640 1 -- 172.21.15.67:0/3838350714 wait complete. 2024-02-26T10:40:08.643 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:40:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.862+0000 7f67a2200640 1 Processor -- start 2024-02-26T10:40:08.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.862+0000 7f67a2200640 1 -- start start 2024-02-26T10:40:08.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.863+0000 7f67a2200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f679c06cfe0 0x7f679c0c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.863+0000 7f67a2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c155c00 0x7f679c0ac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.863+0000 7f67a2200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c0c58f0 con 0x7f679c14f250 2024-02-26T10:40:08.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.863+0000 7f67a2200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679c14f630 con 0x7f679c06cfe0 2024-02-26T10:40:08.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.863+0000 7f67a2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0c8e50 con 0x7f679c155c00 2024-02-26T10:40:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.864+0000 7f679aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f679c06cfe0 0x7f679c0c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.864+0000 7f679be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c155c00 0x7f679c0ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.864+0000 7f679aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f679c06cfe0 0x7f679c0c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40110/0 (socket says 172.21.15.67:40110) 2024-02-26T10:40:08.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.864+0000 7f679aa00640 1 -- 172.21.15.67:0/669072791 learned_addr learned my addr 172.21.15.67:0/669072791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:08.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.865+0000 7f679aa00640 1 -- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c155c00 msgr2=0x7f679c0ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:08.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.865+0000 7f679aa00640 1 --2- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c155c00 0x7f679c0ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.865+0000 7f679aa00640 1 -- 172.21.15.67:0/669072791 >> v1:172.21.15.82:6789/0 conn(0x7f679c14f250 legacy=0x7f679c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.865+0000 7f679aa00640 1 -- 172.21.15.67:0/669072791 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c06c500 con 0x7f679c06cfe0 2024-02-26T10:40:08.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.866+0000 7f679aa00640 1 --2- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c06cfe0 0x7f679c0c51c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6784009870 tx=0x7f678402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=982601ab6dd9a5af server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:08.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.867+0000 7f679a000640 1 -- 172.21.15.67:0/669072791 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840317c0 con 0x7f679c06cfe0 2024-02-26T10:40:08.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.867+0000 7f679a000640 1 -- 172.21.15.67:0/669072791 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6784037070 con 0x7f679c06cfe0 2024-02-26T10:40:08.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.867+0000 7f679a000640 1 -- 172.21.15.67:0/669072791 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784031b80 con 0x7f679c06cfe0 2024-02-26T10:40:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 -- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c06cfe0 msgr2=0x7f679c0c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 --2- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c06cfe0 0x7f679c0c51c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6784009870 tx=0x7f678402b860 comp rx=0 tx=0).stop 2024-02-26T10:40:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 -- 172.21.15.67:0/669072791 shutdown_connections 2024-02-26T10:40:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 --2- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c155c00 0x7f679c0ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 --2- 172.21.15.67:0/669072791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c06cfe0 0x7f679c0c51c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 -- 172.21.15.67:0/669072791 >> 172.21.15.67:0/669072791 conn(0x7f679c072b40 msgr2=0x7f679c0ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 -- 172.21.15.67:0/669072791 shutdown_connections 2024-02-26T10:40:08.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.868+0000 7f67a2200640 1 -- 172.21.15.67:0/669072791 wait complete. 2024-02-26T10:40:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.870+0000 7f67a2200640 1 Processor -- start 2024-02-26T10:40:08.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.870+0000 7f67a2200640 1 -- start start 2024-02-26T10:40:08.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f67a2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c06cfe0 0x7f679c1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f67a2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c14f250 0x7f679c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f67a2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 0x7f679c149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f67a2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0c5bd0 con 0x7f679c155c00 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f67a2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c06cac0 con 0x7f679c14f250 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f67a2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c156000 con 0x7f679c06cfe0 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f679be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 0x7f679c149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f679be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 0x7f679c149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54450/0 (socket says 172.21.15.67:54450) 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f679be00640 1 -- 172.21.15.67:0/2493330108 learned_addr learned my addr 172.21.15.67:0/2493330108 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f679aa00640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c14f250 0x7f679c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.871+0000 7f679b400640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c06cfe0 0x7f679c1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f679be00640 1 -- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c06cfe0 msgr2=0x7f679c1521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f679be00640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c06cfe0 0x7f679c1521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f679be00640 1 -- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c14f250 msgr2=0x7f679c150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f679be00640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c14f250 0x7f679c150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:08.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f679be00640 1 -- 172.21.15.67:0/2493330108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c152830 con 0x7f679c155c00 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f679be00640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 0x7f679c149eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6794009d30 tx=0x7f679400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794018020 con 0x7f679c155c00 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6794014480 con 0x7f679c155c00 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794014870 con 0x7f679c155c00 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.872+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c0c3a70 con 0x7f679c155c00 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.873+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c0c4030 con 0x7f679c155c00 2024-02-26T10:40:08.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.874+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c074a30 con 0x7f679c155c00 2024-02-26T10:40:08.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.875+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6794005d10 con 0x7f679c155c00 2024-02-26T10:40:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.876+0000 7f6793400640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f677806b4d0 0x7f677806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.876+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f679408aff0 con 0x7f679c155c00 2024-02-26T10:40:08.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.876+0000 7f679b400640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f677806b4d0 0x7f677806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:08.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.877+0000 7f679b400640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f677806b4d0 0x7f677806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f679c0aca10 tx=0x7f678c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:08.880+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6794022070 con 0x7f679c155c00 2024-02-26T10:40:09.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.074+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f679c06db10 con 0x7f679c155c00 2024-02-26T10:40:09.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.075+0000 7f6793400640 1 -- 172.21.15.67:0/2493330108 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6794010040 con 0x7f679c155c00 2024-02-26T10:40:09.075 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:09.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.077+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f677806b4d0 msgr2=0x7f677806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.077+0000 7f67a2200640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f677806b4d0 0x7f677806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f679c0aca10 tx=0x7f678c009040 comp rx=0 tx=0).stop 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.077+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 msgr2=0x7f679c149eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.077+0000 7f67a2200640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 0x7f679c149eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6794009d30 tx=0x7f679400c490 comp rx=0 tx=0).stop 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 shutdown_connections 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f679c155c00 0x7f679c149eb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f677806b4d0 0x7f677806d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f679c14f250 0x7f679c150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 --2- 172.21.15.67:0/2493330108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f679c06cfe0 0x7f679c1521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 >> 172.21.15.67:0/2493330108 conn(0x7f679c072b40 msgr2=0x7f679c14bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 shutdown_connections 2024-02-26T10:40:09.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.078+0000 7f67a2200640 1 -- 172.21.15.67:0/2493330108 wait complete. 2024-02-26T10:40:09.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:40:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.308+0000 7fe8a3200640 1 Processor -- start 2024-02-26T10:40:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.308+0000 7fe8a3200640 1 -- start start 2024-02-26T10:40:09.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40a9430 0x7fe8a40ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8a406e2d0 0x7fe8a40c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a4150780 con 0x7fe8a41503a0 2024-02-26T10:40:09.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40a98c0 con 0x7fe8a406e2d0 2024-02-26T10:40:09.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40a9ba0 con 0x7fe8a40a9430 2024-02-26T10:40:09.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8a406e2d0 0x7fe8a40c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a1e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe8a41503a0 0x7fe8a41535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53058/0 (socket says 172.21.15.67:53058) 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a1e00640 1 -- 172.21.15.67:0/4013396689 learned_addr learned my addr 172.21.15.67:0/4013396689 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.310+0000 7fe8a2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8a406e2d0 0x7fe8a40c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54456/0 (socket says 172.21.15.67:54456) 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.311+0000 7fe899400640 1 --2- 172.21.15.67:0/4013396689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40a9430 0x7fe8a40ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.311+0000 7fe8a2800640 1 -- 172.21.15.67:0/4013396689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40a9430 msgr2=0x7fe8a40ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.311+0000 7fe8a2800640 1 --2- 172.21.15.67:0/4013396689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40a9430 0x7fe8a40ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.311+0000 7fe8a2800640 1 -- 172.21.15.67:0/4013396689 >> v1:172.21.15.67:6789/0 conn(0x7fe8a41503a0 legacy=0x7fe8a41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.311+0000 7fe8a2800640 1 -- 172.21.15.67:0/4013396689 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40a9e80 con 0x7fe8a406e2d0 2024-02-26T10:40:09.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.311+0000 7fe8a1400640 1 -- 172.21.15.67:0/4013396689 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466788071 0 0) 0x7fe8a4150780 con 0x7fe8a41503a0 2024-02-26T10:40:09.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.312+0000 7fe8a2800640 1 --2- 172.21.15.67:0/4013396689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a406e2d0 0x7fe8a40c2e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0097a0 tx=0x7fe89c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabb6d07e2fc6fa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:09.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.312+0000 7fe8a1400640 1 -- 172.21.15.67:0/4013396689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c035020 con 0x7fe8a406e2d0 2024-02-26T10:40:09.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.312+0000 7fe8a1400640 1 -- 172.21.15.67:0/4013396689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c031830 con 0x7fe8a406e2d0 2024-02-26T10:40:09.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.312+0000 7fe8a1400640 1 -- 172.21.15.67:0/4013396689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c031c00 con 0x7fe8a406e2d0 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 -- 172.21.15.67:0/4013396689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a406e2d0 msgr2=0x7fe8a40c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 --2- 172.21.15.67:0/4013396689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a406e2d0 0x7fe8a40c2e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0097a0 tx=0x7fe89c02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 -- 172.21.15.67:0/4013396689 shutdown_connections 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 --2- 172.21.15.67:0/4013396689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a406e2d0 0x7fe8a40c2e10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 --2- 172.21.15.67:0/4013396689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40a9430 0x7fe8a40ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 -- 172.21.15.67:0/4013396689 >> 172.21.15.67:0/4013396689 conn(0x7fe8a40c6380 msgr2=0x7fe8a40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 -- 172.21.15.67:0/4013396689 shutdown_connections 2024-02-26T10:40:09.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.313+0000 7fe8a3200640 1 -- 172.21.15.67:0/4013396689 wait complete. 2024-02-26T10:40:09.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.314+0000 7fe8a3200640 1 Processor -- start 2024-02-26T10:40:09.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.315+0000 7fe8a3200640 1 -- start start 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.315+0000 7fe8a3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a406e2d0 0x7fe8a40c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.315+0000 7fe8a3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 0x7fe8a4071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe8a3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40c40c0 0x7fe8a41dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe8a3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40c0f80 con 0x7fe8a41503a0 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe8a3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a406ddb0 con 0x7fe8a406e2d0 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe8a3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a406e740 con 0x7fe8a40c40c0 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 0x7fe8a4071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 0x7fe8a4071560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54464/0 (socket says 172.21.15.67:54464) 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 -- 172.21.15.67:0/474724240 learned_addr learned my addr 172.21.15.67:0/474724240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 -- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40c40c0 msgr2=0x7fe8a41dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40c40c0 0x7fe8a41dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 -- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a406e2d0 msgr2=0x7fe8a40c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a406e2d0 0x7fe8a40c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.316+0000 7fe899400640 1 -- 172.21.15.67:0/474724240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a41de900 con 0x7fe8a41503a0 2024-02-26T10:40:09.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.317+0000 7fe899400640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 0x7fe8a4071560 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00d800 tx=0x7fe88c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:09.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.317+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00b6c0 con 0x7fe8a41503a0 2024-02-26T10:40:09.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.317+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c019070 con 0x7fe8a41503a0 2024-02-26T10:40:09.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.317+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a41debe0 con 0x7fe8a41503a0 2024-02-26T10:40:09.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.317+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c00ba80 con 0x7fe8a41503a0 2024-02-26T10:40:09.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.317+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a4072730 con 0x7fe8a41503a0 2024-02-26T10:40:09.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.318+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a40678e0 con 0x7fe8a41503a0 2024-02-26T10:40:09.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.319+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe88c00bc20 con 0x7fe8a41503a0 2024-02-26T10:40:09.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.321+0000 7fe89aa00640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe88006b500 0x7fe88006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.321+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fe88c08a960 con 0x7fe8a41503a0 2024-02-26T10:40:09.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.321+0000 7fe8a1e00640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe88006b500 0x7fe88006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.322+0000 7fe8a1e00640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe88006b500 0x7fe88006d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe894007fd0 tx=0x7fe894007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:09.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.325+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe88c059510 con 0x7fe8a41503a0 2024-02-26T10:40:09.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.523+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8a4074a30 con 0x7fe8a41503a0 2024-02-26T10:40:09.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.524+0000 7fe89aa00640 1 -- 172.21.15.67:0/474724240 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe88c058e10 con 0x7fe8a41503a0 2024-02-26T10:40:09.524 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe88006b500 msgr2=0x7fe88006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe88006b500 0x7fe88006d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe894007fd0 tx=0x7fe894007ce0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 msgr2=0x7fe8a4071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 0x7fe8a4071560 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00d800 tx=0x7fe88c00dcc0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 shutdown_connections 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8a40c40c0 0x7fe8a41dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8a41503a0 0x7fe8a4071560 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe88006b500 0x7fe88006d9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 --2- 172.21.15.67:0/474724240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a406e2d0 0x7fe8a40c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 >> 172.21.15.67:0/474724240 conn(0x7fe8a40c6380 msgr2=0x7fe8a40c1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:09.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 shutdown_connections 2024-02-26T10:40:09.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.527+0000 7fe8a3200640 1 -- 172.21.15.67:0/474724240 wait complete. 2024-02-26T10:40:09.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:40:09.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.775+0000 7f0eb1400640 1 Processor -- start 2024-02-26T10:40:09.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.776+0000 7f0eb1400640 1 -- start start 2024-02-26T10:40:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.777+0000 7f0eb1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0eac1503d0 0x7f0eac06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.777+0000 7f0eb1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0eac06e390 0x7f0eac1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.777+0000 7f0eb1400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac0a6eb0 con 0x7f0eac0c2950 2024-02-26T10:40:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.777+0000 7f0eb1400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac0a7190 con 0x7f0eac06e390 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.777+0000 7f0eb1400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac0a7470 con 0x7f0eac1503d0 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0eac06e390 0x7f0eac1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0eac06e390 0x7f0eac1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36202/0 (socket says 172.21.15.67:36202) 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 -- 172.21.15.67:0/2888670180 learned_addr learned my addr 172.21.15.67:0/2888670180 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0eaa800640 1 --2- 172.21.15.67:0/2888670180 >> v2:172.21.15.82:3300/0 conn(0x7f0eac1503d0 0x7f0eac06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 -- 172.21.15.67:0/2888670180 >> v2:172.21.15.82:3300/0 conn(0x7f0eac1503d0 msgr2=0x7f0eac06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 --2- 172.21.15.67:0/2888670180 >> v2:172.21.15.82:3300/0 conn(0x7f0eac1503d0 0x7f0eac06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 -- 172.21.15.67:0/2888670180 >> v1:172.21.15.82:6789/0 conn(0x7f0eac0c2950 legacy=0x7f0eac156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.778+0000 7f0ea9e00640 1 -- 172.21.15.67:0/2888670180 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0a7750 con 0x7f0eac06e390 2024-02-26T10:40:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.779+0000 7f0ea9e00640 1 --2- 172.21.15.67:0/2888670180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1533e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0eac06f1d0 tx=0x7f0e9402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a55bedd5d81619 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0ea9400640 1 -- 172.21.15.67:0/2888670180 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94038070 con 0x7f0eac06e390 2024-02-26T10:40:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0ea9400640 1 -- 172.21.15.67:0/2888670180 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9402f050 con 0x7f0eac06e390 2024-02-26T10:40:09.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0ea9400640 1 -- 172.21.15.67:0/2888670180 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94031970 con 0x7f0eac06e390 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0eb1400640 1 -- 172.21.15.67:0/2888670180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 msgr2=0x7f0eac1533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2888670180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1533e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0eac06f1d0 tx=0x7f0e9402b7f0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0eb1400640 1 -- 172.21.15.67:0/2888670180 shutdown_connections 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2888670180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1533e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2888670180 >> v2:172.21.15.82:3300/0 conn(0x7f0eac1503d0 0x7f0eac06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.780+0000 7f0eb1400640 1 -- 172.21.15.67:0/2888670180 >> 172.21.15.67:0/2888670180 conn(0x7f0eac0c6da0 msgr2=0x7f0eac0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.781+0000 7f0eb1400640 1 -- 172.21.15.67:0/2888670180 shutdown_connections 2024-02-26T10:40:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.781+0000 7f0eb1400640 1 -- 172.21.15.67:0/2888670180 wait complete. 2024-02-26T10:40:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.782+0000 7f0eb1400640 1 Processor -- start 2024-02-26T10:40:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.782+0000 7f0eb1400640 1 -- start start 2024-02-26T10:40:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eb1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eb1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac0c2950 0x7f0eac14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eb1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0eac1503d0 0x7f0eac0c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eb1400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1507b0 con 0x7f0eac06e390 2024-02-26T10:40:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eb1400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac06ef00 con 0x7f0eac0c2950 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eb1400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0c0ef0 con 0x7f0eac1503d0 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eaa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eaa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36214/0 (socket says 172.21.15.67:36214) 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.783+0000 7f0eaa800640 1 -- 172.21.15.67:0/2210280478 learned_addr learned my addr 172.21.15.67:0/2210280478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eaa800640 1 -- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0eac1503d0 msgr2=0x7f0eac0c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eaa800640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0eac1503d0 0x7f0eac0c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eaa800640 1 -- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac0c2950 msgr2=0x7f0eac14c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eaa800640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac0c2950 0x7f0eac14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eaa800640 1 -- 172.21.15.67:0/2210280478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac156d30 con 0x7f0eac06e390 2024-02-26T10:40:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eaa800640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1459e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00d5f0 tx=0x7f0e9c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0125f0 con 0x7f0eac06e390 2024-02-26T10:40:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c019070 con 0x7f0eac06e390 2024-02-26T10:40:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0129d0 con 0x7f0eac06e390 2024-02-26T10:40:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.784+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac145700 con 0x7f0eac06e390 2024-02-26T10:40:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.785+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac1e91f0 con 0x7f0eac06e390 2024-02-26T10:40:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.786+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac074a30 con 0x7f0eac06e390 2024-02-26T10:40:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.786+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e9c012b70 con 0x7f0eac06e390 2024-02-26T10:40:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.787+0000 7f0ea2a00640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0e8806b500 0x7f0e8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.787+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f0e9c08a930 con 0x7f0eac06e390 2024-02-26T10:40:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.788+0000 7f0ea9e00640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0e8806b500 0x7f0e8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.789+0000 7f0ea9e00640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0e8806b500 0x7f0e8806d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e94035000 tx=0x7f0e94030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:09.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.792+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e9c059440 con 0x7f0eac06e390 2024-02-26T10:40:09.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.992+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0eac06ec50 con 0x7f0eac06e390 2024-02-26T10:40:09.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.994+0000 7f0ea2a00640 1 -- 172.21.15.67:0/2210280478 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v38) v1 ==== 105+0+11 (secure 0 0 0) 0x7f0e9c058d40 con 0x7f0eac06e390 2024-02-26T10:40:09.994 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0e8806b500 msgr2=0x7f0e8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0e8806b500 0x7f0e8806d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0e94035000 tx=0x7f0e94030000 comp rx=0 tx=0).stop 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 msgr2=0x7f0eac1459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1459e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00d5f0 tx=0x7f0e9c00dab0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 shutdown_connections 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0eac1503d0 0x7f0eac0c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0e8806b500 0x7f0e8806d9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac0c2950 0x7f0eac14c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 --2- 172.21.15.67:0/2210280478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0eac06e390 0x7f0eac1459e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 >> 172.21.15.67:0/2210280478 conn(0x7f0eac0c6da0 msgr2=0x7f0eac150f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 shutdown_connections 2024-02-26T10:40:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:09.996+0000 7f0eb1400640 1 -- 172.21.15.67:0/2210280478 wait complete. 2024-02-26T10:40:10.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:40:10.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.234+0000 7f9702600640 1 Processor -- start 2024-02-26T10:40:10.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.234+0000 7f9702600640 1 -- start start 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f9702600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96fc14f250 0x7f96fc06cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f9702600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc0c51c0 0x7f96fc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f9702600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96fc0c5670 con 0x7f96fc0c8740 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f9702600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96fc14f630 con 0x7f96fc14f250 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f9702600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0c58f0 con 0x7f96fc0c51c0 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f96fb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96fc14f250 0x7f96fc06cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f96fb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96fc14f250 0x7f96fc06cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36228/0 (socket says 172.21.15.67:36228) 2024-02-26T10:40:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.235+0000 7f96fb400640 1 -- 172.21.15.67:0/3509700775 learned_addr learned my addr 172.21.15.67:0/3509700775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.236+0000 7f96fb400640 1 -- 172.21.15.67:0/3509700775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc0c51c0 msgr2=0x7f96fc155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.236+0000 7f96fb400640 1 --2- 172.21.15.67:0/3509700775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc0c51c0 0x7f96fc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.236+0000 7f96fb400640 1 -- 172.21.15.67:0/3509700775 >> v1:172.21.15.82:6789/0 conn(0x7f96fc0c8740 legacy=0x7f96fc0ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.236+0000 7f96fb400640 1 -- 172.21.15.67:0/3509700775 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc06c500 con 0x7f96fc14f250 2024-02-26T10:40:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.237+0000 7f96fb400640 1 --2- 172.21.15.67:0/3509700775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc14f250 0x7f96fc06cfe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009870 tx=0x7f96ec02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8552be6be185dfc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.237+0000 7f96fa000640 1 -- 172.21.15.67:0/3509700775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec038070 con 0x7f96fc14f250 2024-02-26T10:40:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.237+0000 7f96fa000640 1 -- 172.21.15.67:0/3509700775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec031740 con 0x7f96fc14f250 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.237+0000 7f96fa000640 1 -- 172.21.15.67:0/3509700775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec02f050 con 0x7f96fc14f250 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 -- 172.21.15.67:0/3509700775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc14f250 msgr2=0x7f96fc06cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 --2- 172.21.15.67:0/3509700775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc14f250 0x7f96fc06cfe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009870 tx=0x7f96ec02b7e0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 -- 172.21.15.67:0/3509700775 shutdown_connections 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 --2- 172.21.15.67:0/3509700775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc0c51c0 0x7f96fc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 --2- 172.21.15.67:0/3509700775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc14f250 0x7f96fc06cfe0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 -- 172.21.15.67:0/3509700775 >> 172.21.15.67:0/3509700775 conn(0x7f96fc072b40 msgr2=0x7f96fc0ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 -- 172.21.15.67:0/3509700775 shutdown_connections 2024-02-26T10:40:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.238+0000 7f9702600640 1 -- 172.21.15.67:0/3509700775 wait complete. 2024-02-26T10:40:10.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.240+0000 7f9702600640 1 Processor -- start 2024-02-26T10:40:10.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.240+0000 7f9702600640 1 -- start start 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f9702600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96fc0c51c0 0x7f96fc14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f9702600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 0x7f96fc1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f9702600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc14f250 0x7f96fc1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f9702600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0c5bd0 con 0x7f96fc0c8740 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f9702600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc06cac0 con 0x7f96fc0c51c0 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f9702600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0aca50 con 0x7f96fc14f250 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f96faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 0x7f96fc1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f96faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 0x7f96fc1515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36238/0 (socket says 172.21.15.67:36238) 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f96faa00640 1 -- 172.21.15.67:0/3988285676 learned_addr learned my addr 172.21.15.67:0/3988285676 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f96fb400640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96fc0c51c0 0x7f96fc14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.241+0000 7f96fbe00640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc14f250 0x7f96fc1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96faa00640 1 -- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc14f250 msgr2=0x7f96fc1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96faa00640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc14f250 0x7f96fc1568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96faa00640 1 -- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96fc0c51c0 msgr2=0x7f96fc14ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96faa00640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96fc0c51c0 0x7f96fc14ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96faa00640 1 -- 172.21.15.67:0/3988285676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc1eef90 con 0x7f96fc0c8740 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96fbe00640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc14f250 0x7f96fc1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96faa00640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 0x7f96fc1515c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b4a0 tx=0x7f96e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8014070 con 0x7f96fc0c8740 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8010610 con 0x7f96fc0c8740 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8010a00 con 0x7f96fc0c8740 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc0c4150 con 0x7f96fc0c8740 2024-02-26T10:40:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.242+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc14aa30 con 0x7f96fc0c8740 2024-02-26T10:40:10.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.243+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96fc074a30 con 0x7f96fc0c8740 2024-02-26T10:40:10.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.244+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96e8017070 con 0x7f96fc0c8740 2024-02-26T10:40:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.245+0000 7f96f3400640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96dc06b4d0 0x7f96dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.245+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f96e8089ed0 con 0x7f96fc0c8740 2024-02-26T10:40:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.246+0000 7f96fb400640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96dc06b4d0 0x7f96dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.247+0000 7f96fb400640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96dc06b4d0 0x7f96dc06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96ec035000 tx=0x7f96ec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:10.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.251+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96e8058a80 con 0x7f96fc0c8740 2024-02-26T10:40:10.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.451+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f96fc0c4320 con 0x7f96fc0c8740 2024-02-26T10:40:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.451+0000 7f96f3400640 1 -- 172.21.15.67:0/3988285676 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v38) v1 ==== 101+0+11 (secure 0 0 0) 0x7f96e8058380 con 0x7f96fc0c8740 2024-02-26T10:40:10.452 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96dc06b4d0 msgr2=0x7f96dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96dc06b4d0 0x7f96dc06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96ec035000 tx=0x7f96ec008030 comp rx=0 tx=0).stop 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 msgr2=0x7f96fc1515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 0x7f96fc1515c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b4a0 tx=0x7f96e800b960 comp rx=0 tx=0).stop 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 shutdown_connections 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96fc14f250 0x7f96fc1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96fc0c8740 0x7f96fc1515c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96dc06b4d0 0x7f96dc06d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 --2- 172.21.15.67:0/3988285676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96fc0c51c0 0x7f96fc14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 >> 172.21.15.67:0/3988285676 conn(0x7f96fc072b40 msgr2=0x7f96fc0c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:10.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.454+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 shutdown_connections 2024-02-26T10:40:10.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.455+0000 7f9702600640 1 -- 172.21.15.67:0/3988285676 wait complete. 2024-02-26T10:40:10.475 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:40:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.692+0000 7faf2b200640 1 Processor -- start 2024-02-26T10:40:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.693+0000 7faf2b200640 1 -- start start 2024-02-26T10:40:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf2b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf2b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf2c0c0850 0x7faf2c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf2b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c155cf0 con 0x7faf2c06b160 2024-02-26T10:40:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf2b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c06dca0 con 0x7faf2c0c0850 2024-02-26T10:40:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf2b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0c2a70 con 0x7faf2c0c1c10 2024-02-26T10:40:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf29400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf2c0c0850 0x7faf2c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf29400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf2c0c0850 0x7faf2c06ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36254/0 (socket says 172.21.15.67:36254) 2024-02-26T10:40:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf29400640 1 -- 172.21.15.67:0/903149348 learned_addr learned my addr 172.21.15.67:0/903149348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.694+0000 7faf29e00640 1 --2- 172.21.15.67:0/903149348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.695+0000 7faf29400640 1 -- 172.21.15.67:0/903149348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 msgr2=0x7faf2c1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.695+0000 7faf29400640 1 --2- 172.21.15.67:0/903149348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.695+0000 7faf29400640 1 -- 172.21.15.67:0/903149348 >> v1:172.21.15.82:6789/0 conn(0x7faf2c06b160 legacy=0x7faf2c06e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.695+0000 7faf29400640 1 -- 172.21.15.67:0/903149348 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0c21d0 con 0x7faf2c0c0850 2024-02-26T10:40:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.696+0000 7faf29400640 1 --2- 172.21.15.67:0/903149348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c0c0850 0x7faf2c06ac20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf18009870 tx=0x7faf1802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3641ad82cac6ebd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf28a00640 1 -- 172.21.15.67:0/903149348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180317c0 con 0x7faf2c0c0850 2024-02-26T10:40:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf28a00640 1 -- 172.21.15.67:0/903149348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf18037070 con 0x7faf2c0c0850 2024-02-26T10:40:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf28a00640 1 -- 172.21.15.67:0/903149348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18031b80 con 0x7faf2c0c0850 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf2b200640 1 -- 172.21.15.67:0/903149348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c0c0850 msgr2=0x7faf2c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf2b200640 1 --2- 172.21.15.67:0/903149348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c0c0850 0x7faf2c06ac20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf18009870 tx=0x7faf1802b860 comp rx=0 tx=0).stop 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf2b200640 1 -- 172.21.15.67:0/903149348 shutdown_connections 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf2b200640 1 --2- 172.21.15.67:0/903149348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c0c0850 0x7faf2c06ac20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.697+0000 7faf2b200640 1 --2- 172.21.15.67:0/903149348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.698+0000 7faf2b200640 1 -- 172.21.15.67:0/903149348 >> 172.21.15.67:0/903149348 conn(0x7faf2c0c63f0 msgr2=0x7faf2c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.698+0000 7faf2b200640 1 -- 172.21.15.67:0/903149348 shutdown_connections 2024-02-26T10:40:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.698+0000 7faf2b200640 1 -- 172.21.15.67:0/903149348 wait complete. 2024-02-26T10:40:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.699+0000 7faf2b200640 1 Processor -- start 2024-02-26T10:40:10.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 -- start start 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 0x7faf2c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf2c0c0850 0x7faf2c14b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1edf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0c0df0 con 0x7faf2c06b160 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0c2790 con 0x7faf2c0c0850 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.700+0000 7faf2b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0c0f70 con 0x7faf2c0c1c10 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 0x7faf2c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 0x7faf2c1514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36264/0 (socket says 172.21.15.67:36264) 2024-02-26T10:40:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 -- 172.21.15.67:0/1949825861 learned_addr learned my addr 172.21.15.67:0/1949825861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 -- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 msgr2=0x7faf2c1edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 -- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf2c0c0850 msgr2=0x7faf2c14b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf2c0c0850 0x7faf2c14b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.701+0000 7faf29e00640 1 -- 172.21.15.67:0/1949825861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c1523e0 con 0x7faf2c06b160 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.702+0000 7faf29e00640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 0x7faf2c1514c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00e870 tx=0x7faf1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.702+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c017070 con 0x7faf2c06b160 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.702+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c00c940 con 0x7faf2c06b160 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.702+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00cd30 con 0x7faf2c06b160 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.702+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c0ac340 con 0x7faf2c06b160 2024-02-26T10:40:10.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.702+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c0aab30 con 0x7faf2c06b160 2024-02-26T10:40:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.703+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c074a30 con 0x7faf2c06b160 2024-02-26T10:40:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.704+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf1c01a070 con 0x7faf2c06b160 2024-02-26T10:40:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.705+0000 7faf22000640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faf0c06b500 0x7faf0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.705+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7faf1c08a470 con 0x7faf2c06b160 2024-02-26T10:40:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.706+0000 7faf29400640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faf0c06b500 0x7faf0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:10.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.707+0000 7faf29400640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faf0c06b500 0x7faf0c06d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf18036000 tx=0x7faf18030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.711+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf1c058ef0 con 0x7faf2c06b160 2024-02-26T10:40:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.922+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faf2c14f180 con 0x7faf2c06b160 2024-02-26T10:40:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.923+0000 7faf22000640 1 -- 172.21.15.67:0/1949825861 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7faf1c0587f0 con 0x7faf2c06b160 2024-02-26T10:40:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faf0c06b500 msgr2=0x7faf0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faf0c06b500 0x7faf0c06d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faf18036000 tx=0x7faf18030000 comp rx=0 tx=0).stop 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 msgr2=0x7faf2c1514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 0x7faf2c1514c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00e870 tx=0x7faf1c00ed30 comp rx=0 tx=0).stop 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 shutdown_connections 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf2c0c1c10 0x7faf2c1edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faf0c06b500 0x7faf0c06d9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf2c0c0850 0x7faf2c14b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 --2- 172.21.15.67:0/1949825861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf2c06b160 0x7faf2c1514c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 >> 172.21.15.67:0/1949825861 conn(0x7faf2c0c63f0 msgr2=0x7faf2c072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 shutdown_connections 2024-02-26T10:40:10.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:10.926+0000 7faf2b200640 1 -- 172.21.15.67:0/1949825861 wait complete. 2024-02-26T10:40:10.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T10:40:10.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:10.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:11.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:40:11.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:11.051 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T10:40:11.104 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:11.104 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:11.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:40:11.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:40:11.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.368+0000 7f5cbf200640 1 Processor -- start 2024-02-26T10:40:11.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.368+0000 7f5cbf200640 1 -- start start 2024-02-26T10:40:11.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbf200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5cc0153630 0x7f5cc006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:11.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbf200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc0155f10 con 0x7f5cc006ac20 2024-02-26T10:40:11.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbf200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc0153a60 con 0x7f5cc00c2310 2024-02-26T10:40:11.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbf200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00c28d0 con 0x7f5cc0153630 2024-02-26T10:40:11.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbde00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5cc006ac20 0x7f5cc0152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34290/0 (socket says 172.21.15.67:34290) 2024-02-26T10:40:11.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbde00640 1 -- 172.21.15.67:0/3536041532 learned_addr learned my addr 172.21.15.67:0/3536041532 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:11.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.370+0000 7f5cbe800640 1 --2- 172.21.15.67:0/3536041532 >> v2:172.21.15.67:3300/0 conn(0x7f5cc0153630 0x7f5cc006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.371+0000 7f5cbca00640 1 -- 172.21.15.67:0/3536041532 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157293774 0 0) 0x7f5cc0155f10 con 0x7f5cc006ac20 2024-02-26T10:40:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.371+0000 7f5cbca00640 1 -- 172.21.15.67:0/3536041532 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca4005730 con 0x7f5cc006ac20 2024-02-26T10:40:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.371+0000 7f5cbe800640 1 -- 172.21.15.67:0/3536041532 >> v1:172.21.15.82:6789/0 conn(0x7f5cc00c2310 legacy=0x7f5cc015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.371+0000 7f5cbe800640 1 -- 172.21.15.67:0/3536041532 >> v1:172.21.15.67:6789/0 conn(0x7f5cc006ac20 legacy=0x7f5cc0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.371+0000 7f5cbe800640 1 -- 172.21.15.67:0/3536041532 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0155150 con 0x7f5cc0153630 2024-02-26T10:40:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.372+0000 7f5cbe800640 1 --2- 172.21.15.67:0/3536041532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0153630 0x7f5cc006b000 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80097a0 tx=0x7f5cb802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fbfbc3daad6c53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:11.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.372+0000 7f5cbca00640 1 -- 172.21.15.67:0/3536041532 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80315a0 con 0x7f5cc0153630 2024-02-26T10:40:11.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.372+0000 7f5cbca00640 1 -- 172.21.15.67:0/3536041532 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8037070 con 0x7f5cc0153630 2024-02-26T10:40:11.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.372+0000 7f5cbca00640 1 -- 172.21.15.67:0/3536041532 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8031960 con 0x7f5cc0153630 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 -- 172.21.15.67:0/3536041532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0153630 msgr2=0x7f5cc006b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 --2- 172.21.15.67:0/3536041532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0153630 0x7f5cc006b000 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80097a0 tx=0x7f5cb802b840 comp rx=0 tx=0).stop 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 -- 172.21.15.67:0/3536041532 shutdown_connections 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 --2- 172.21.15.67:0/3536041532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc0153630 0x7f5cc006b000 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 -- 172.21.15.67:0/3536041532 >> 172.21.15.67:0/3536041532 conn(0x7f5cc00c63f0 msgr2=0x7f5cc00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 -- 172.21.15.67:0/3536041532 shutdown_connections 2024-02-26T10:40:11.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.373+0000 7f5cbf200640 1 -- 172.21.15.67:0/3536041532 wait complete. 2024-02-26T10:40:11.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.375+0000 7f5cbf200640 1 Processor -- start 2024-02-26T10:40:11.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.375+0000 7f5cbf200640 1 -- start start 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbf200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cc006ac20 0x7f5cc0143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 0x7f5cc0070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbf200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc0153630 0x7f5cc01edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc01559f0 con 0x7f5cc00c2310 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbf200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0155710 con 0x7f5cc0153630 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbf200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0148850 con 0x7f5cc006ac20 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 0x7f5cc0070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 0x7f5cc0070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36290/0 (socket says 172.21.15.67:36290) 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.376+0000 7f5cbd400640 1 -- 172.21.15.67:0/1129202040 learned_addr learned my addr 172.21.15.67:0/1129202040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbde00640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cc006ac20 0x7f5cc0143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbe800640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc0153630 0x7f5cc01edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:11.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbd400640 1 -- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cc006ac20 msgr2=0x7f5cc0143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbd400640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cc006ac20 0x7f5cc0143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbd400640 1 -- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc0153630 msgr2=0x7f5cc01edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbd400640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc0153630 0x7f5cc01edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbd400640 1 -- 172.21.15.67:0/1129202040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc013ee50 con 0x7f5cc00c2310 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbe800640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc0153630 0x7f5cc01edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbde00640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cc006ac20 0x7f5cc0143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cbd400640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 0x7f5cc0070c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c430 tx=0x7f5cac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac017070 con 0x7f5cc00c2310 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac0052d0 con 0x7f5cc00c2310 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.377+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003830 con 0x7f5cc00c2310 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.378+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc00ac090 con 0x7f5cc00c2310 2024-02-26T10:40:11.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.378+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc013f7c0 con 0x7f5cc00c2310 2024-02-26T10:40:11.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.379+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc00abe80 con 0x7f5cc00c2310 2024-02-26T10:40:11.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.380+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cac01a070 con 0x7f5cc00c2310 2024-02-26T10:40:11.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.381+0000 7f5cb6000640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ca006b4d0 0x7f5ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:11.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.381+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f5cac089900 con 0x7f5cc00c2310 2024-02-26T10:40:11.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.381+0000 7f5cbde00640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ca006b4d0 0x7f5ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:11.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.382+0000 7f5cbde00640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ca006b4d0 0x7f5ca006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0001560 tx=0x7f5cb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:11.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.385+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cac058410 con 0x7f5cc00c2310 2024-02-26T10:40:11.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.628+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5cc0074a30 con 0x7f5cc00c2310 2024-02-26T10:40:11.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.629+0000 7f5cb6000640 1 -- 172.21.15.67:0/1129202040 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+149 (secure 0 0 0) 0x7f5cac057d10 con 0x7f5cc00c2310 2024-02-26T10:40:11.629 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:40:11.629 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ca006b4d0 msgr2=0x7f5ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ca006b4d0 0x7f5ca006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0001560 tx=0x7f5cb0009040 comp rx=0 tx=0).stop 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 msgr2=0x7f5cc0070c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 0x7f5cc0070c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c430 tx=0x7f5cac00c8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 shutdown_connections 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5ca006b4d0 0x7f5ca006d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc0153630 0x7f5cc01edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cc00c2310 0x7f5cc0070c30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.631+0000 7f5cbf200640 1 --2- 172.21.15.67:0/1129202040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cc006ac20 0x7f5cc0143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.632+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 >> 172.21.15.67:0/1129202040 conn(0x7f5cc00c63f0 msgr2=0x7f5cc0152e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.632+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 shutdown_connections 2024-02-26T10:40:11.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:11.632+0000 7f5cbf200640 1 -- 172.21.15.67:0/1129202040 wait complete. 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:40:11.652 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.t74yb7Egfh 2024-02-26T10:40:11.653 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:40:11.703 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.o1zUEAZlhq 2024-02-26T10:40:11.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /tmp/tmp.t74yb7Egfh /tmp/tmp.o1zUEAZlhq 2024-02-26T10:40:11.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.o1zUEAZlhq 2024-02-26T10:40:11.953 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.o1zUEAZlhq is 185B 2024-02-26T10:40:12.020 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.o1zUEAZlhq 2024-02-26T10:40:12.074 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQCHatxl8b54GRAAIFSuHYyYQln561YAeiB2Hg== caps mds = "allow rw fsname=cephfs" caps mon = "allow r fsname=cephfs" caps osd = "allow rw tag cephfs data=cephfs" 2024-02-26T10:40:12.074 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:40:12.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.074 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:40:12.127 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:40:12.127 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:40:12.127 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:40:12.127 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:12.127 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:40:12.128 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42438sec preferred_lft 42438sec 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::288e:93ff:fe9f:f95d/64 scope link 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::3cfc:46ff:fe71:84e3/64 scope link 2024-02-26T10:40:12.129 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:12.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.130 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:40:12.223 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:40:12.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.228 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:40:12.279 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:40:12.279 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:40:12.279 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:40:12.279 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:40:12.279 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:40:12.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.280 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:40:12.280 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:40:12.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:40:12.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:40:12.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:40:12.280 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:40:12.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.475 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:40:12.527 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:40:12.528 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:40:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.528 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:40:12.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:12.623 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:12.678 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:40:12.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:12.679 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:12.734 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:12.735 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/tmp/tmp.t74yb7Egfh of=/dev/stdout 2024-02-26T10:40:12.788 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs2=/ 2024-02-26T10:40:12.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:12.789 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs2=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQCHatxl8b54GRAAIFSuHYyYQln561YAeiB2Hg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:12.924 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,secret=AQCHatxl8b54GRAAIFSuHYyYQln561YAeiB2Hg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:12.924 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:12.924 INFO:teuthology.orchestra.run.smithi067.stdout:keyring.get_secret failed 2024-02-26T10:40:12.958 INFO:teuthology.orchestra.run.smithi067.stderr:mount.ceph: kernel BUG! 2024-02-26T10:40:12.958 INFO:teuthology.orchestra.run.smithi067.stderr:mount error: no mds (Metadata Server) is up. The cluster might be laggy, or you may not be authorized 2024-02-26T10:40:12.959 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,secret=AQCHatxl8b54GRAAIFSuHYyYQln561YAeiB2Hg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:12.959 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:12.959 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:40:12.959 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: testuser@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs2=/ 2024-02-26T10:40:12.959 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:40:12.965 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-26T10:40:12.965 INFO:tasks.cephfs.kernel_mount:mount command failed 2024-02-26T10:40:12.966 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-26T10:40:12.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:40:13.079 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:40:13.202 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-26T10:40:13.202 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:13.202 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:13.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:40:13.263 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:13.263 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:13.317 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:13.318 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:13.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.579+0000 7fb6c1600640 1 Processor -- start 2024-02-26T10:40:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.579+0000 7fb6c1600640 1 -- start start 2024-02-26T10:40:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.581+0000 7fb6c1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6bc0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.581+0000 7fb6c1600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc152760 con 0x7fb6bc152380 2024-02-26T10:40:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.581+0000 7fb6c1600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc06d9a0 con 0x7fb6bc0c7020 2024-02-26T10:40:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.581+0000 7fb6c1600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0c7550 con 0x7fb6bc0c8510 2024-02-26T10:40:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.581+0000 7fb6bbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb6bc152380 0x7fb6bc155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34298/0 (socket says 172.21.15.67:34298) 2024-02-26T10:40:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.581+0000 7fb6bbe00640 1 -- 172.21.15.67:0/3271466845 learned_addr learned my addr 172.21.15.67:0/3271466845 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6c0c00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6bc0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239738285 0 0) 0x7fb6bc152760 con 0x7fb6bc152380 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a0003520 con 0x7fb6bc152380 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196040158 0 0) 0x7fb6a0003520 con 0x7fb6bc152380 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 msgr2=0x7fb6bc0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6bc0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 >> v1:172.21.15.82:6789/0 conn(0x7fb6bc0c7020 legacy=0x7fb6bc06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0c7830 con 0x7fb6bc152380 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fb6a4001670 con 0x7fb6bc152380 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 >> v1:172.21.15.67:6789/0 conn(0x7fb6bc152380 legacy=0x7fb6bc155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a0003fb0 0x7fb6a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a0004d40 0x7fb6a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0c7550 con 0x7fb6a0004d40 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a000bbb0 con 0x7fb6a0003fb0 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.582+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a000bdb0 con 0x7fb6bc0c8510 2024-02-26T10:40:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6c0c00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a0004d40 0x7fb6a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6bb400640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a0003fb0 0x7fb6a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6c0c00640 1 -- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 msgr2=0x7fb6a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6c0c00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6c0c00640 1 -- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a0003fb0 msgr2=0x7fb6a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6c0c00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a0003fb0 0x7fb6a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6c0c00640 1 -- 172.21.15.67:0/3271466845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0003520 con 0x7fb6a0004d40 2024-02-26T10:40:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.583+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6bc0c8510 0x7fb6a00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.584+0000 7fb6c0c00640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a0004d40 0x7fb6a00051c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009ac0 tx=0x7fb6a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1781bda6301964 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4039070 con 0x7fb6a0004d40 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b402c6b0 con 0x7fb6a0004d40 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6baa00640 1 -- 172.21.15.67:0/3271466845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4009690 con 0x7fb6a0004d40 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6c1600640 1 -- 172.21.15.67:0/3271466845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a0004d40 msgr2=0x7fb6a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6c1600640 1 --2- 172.21.15.67:0/3271466845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a0004d40 0x7fb6a00051c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009ac0 tx=0x7fb6a0005830 comp rx=0 tx=0).stop 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6c0c00640 1 -- 172.21.15.67:0/3271466845 reap_dead start 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6c1600640 1 -- 172.21.15.67:0/3271466845 shutdown_connections 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.585+0000 7fb6c1600640 1 -- 172.21.15.67:0/3271466845 >> 172.21.15.67:0/3271466845 conn(0x7fb6bc0ac1a0 msgr2=0x7fb6bc0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.586+0000 7fb6c1600640 1 -- 172.21.15.67:0/3271466845 shutdown_connections 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.586+0000 7fb6c1600640 1 -- 172.21.15.67:0/3271466845 wait complete. 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.587+0000 7fb6c1600640 1 Processor -- start 2024-02-26T10:40:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.587+0000 7fb6c1600640 1 -- start start 2024-02-26T10:40:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.588+0000 7fb6c1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6a0004d40 0x7fb6bc0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.588+0000 7fb6c1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 0x7fb6bc156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.588+0000 7fb6c1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6bc1557b0 0x7fb6bc1eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.588+0000 7fb6c1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0c4f90 con 0x7fb6bc0afc20 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.588+0000 7fb6c1600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc06cd70 con 0x7fb6bc1557b0 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.588+0000 7fb6c1600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0c5110 con 0x7fb6a0004d40 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 0x7fb6bc156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 0x7fb6bc156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36312/0 (socket says 172.21.15.67:36312) 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 -- 172.21.15.67:0/905703911 learned_addr learned my addr 172.21.15.67:0/905703911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 -- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6a0004d40 msgr2=0x7fb6bc0ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6a0004d40 0x7fb6bc0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 -- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6bc1557b0 msgr2=0x7fb6bc1eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6bc1557b0 0x7fb6bc1eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 -- 172.21.15.67:0/905703911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0ade40 con 0x7fb6bc0afc20 2024-02-26T10:40:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6bb400640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 0x7fb6bc156240 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00e8f0 tx=0x7fb6ac00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.589+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac003b80 con 0x7fb6bc0afc20 2024-02-26T10:40:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.590+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac018070 con 0x7fb6bc0afc20 2024-02-26T10:40:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.590+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0050d0 con 0x7fb6bc0afc20 2024-02-26T10:40:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.590+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc158800 con 0x7fb6bc0afc20 2024-02-26T10:40:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.590+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc156780 con 0x7fb6bc0afc20 2024-02-26T10:40:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.591+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc074a30 con 0x7fb6bc0afc20 2024-02-26T10:40:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.592+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6ac005270 con 0x7fb6bc0afc20 2024-02-26T10:40:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.593+0000 7fb6b3e00640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb69806b500 0x7fb69806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.593+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb6ac08a420 con 0x7fb6bc0afc20 2024-02-26T10:40:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.593+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb69806b500 0x7fb69806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.594+0000 7fb6bbe00640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb69806b500 0x7fb69806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc151110 tx=0x7fb6a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.597+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6ac01a310 con 0x7fb6bc0afc20 2024-02-26T10:40:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.802+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6bc0c7c50 con 0x7fb6bc0afc20 2024-02-26T10:40:13.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.805+0000 7fb6b3e00640 1 -- 172.21.15.67:0/905703911 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+20323 (secure 0 0 0) 0x7fb6ac058fd0 con 0x7fb6bc0afc20 2024-02-26T10:40:13.805 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:13.806 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:57.124601+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.807+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb69806b500 msgr2=0x7fb69806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.807+0000 7fb6c1600640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb69806b500 0x7fb69806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc151110 tx=0x7fb6a4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.807+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 msgr2=0x7fb6bc156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.807+0000 7fb6c1600640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 0x7fb6bc156240 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00e8f0 tx=0x7fb6ac00edb0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 shutdown_connections 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb69806b500 0x7fb69806d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6bc1557b0 0x7fb6bc1eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6bc0afc20 0x7fb6bc156240 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 --2- 172.21.15.67:0/905703911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6a0004d40 0x7fb6bc0ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:13.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 >> 172.21.15.67:0/905703911 conn(0x7fb6bc0ac1a0 msgr2=0x7fb6bc0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:13.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 shutdown_connections 2024-02-26T10:40:13.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:13.808+0000 7fb6c1600640 1 -- 172.21.15.67:0/905703911 wait complete. 2024-02-26T10:40:13.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:39:37.371486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:39:38.475801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:39:53.146836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:39:54.623593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:13.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.048+0000 7f2427e00640 1 Processor -- start 2024-02-26T10:40:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.048+0000 7f2427e00640 1 -- start start 2024-02-26T10:40:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.050+0000 7f2427e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24280c2540 0x7f242815c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.050+0000 7f2427e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f242815cba0 0x7f2428156320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.050+0000 7f2427e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242815d0b0 con 0x7f2428071710 2024-02-26T10:40:14.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.050+0000 7f2427e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242815d390 con 0x7f242815cba0 2024-02-26T10:40:14.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.050+0000 7f2427e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242806dbd0 con 0x7f24280c2540 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f242815cba0 0x7f2428156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f242815cba0 0x7f2428156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36324/0 (socket says 172.21.15.67:36324) 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 -- 172.21.15.67:0/3289513997 learned_addr learned my addr 172.21.15.67:0/3289513997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2426000640 1 --2- 172.21.15.67:0/3289513997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24280c2540 0x7f242815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 -- 172.21.15.67:0/3289513997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24280c2540 msgr2=0x7f242815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 --2- 172.21.15.67:0/3289513997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24280c2540 0x7f242815c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 -- 172.21.15.67:0/3289513997 >> v1:172.21.15.82:6789/0 conn(0x7f2428071710 legacy=0x7f242806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.051+0000 7f2427400640 1 -- 172.21.15.67:0/3289513997 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242806deb0 con 0x7f242815cba0 2024-02-26T10:40:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.052+0000 7f2427400640 1 --2- 172.21.15.67:0/3289513997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242815cba0 0x7f2428156320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2420009870 tx=0x7f242002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89422b810965eea0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2425600640 1 -- 172.21.15.67:0/3289513997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420035020 con 0x7f242815cba0 2024-02-26T10:40:14.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2425600640 1 -- 172.21.15.67:0/3289513997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2420031640 con 0x7f242815cba0 2024-02-26T10:40:14.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2425600640 1 -- 172.21.15.67:0/3289513997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242002d040 con 0x7f242815cba0 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2427e00640 1 -- 172.21.15.67:0/3289513997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242815cba0 msgr2=0x7f2428156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2427e00640 1 --2- 172.21.15.67:0/3289513997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242815cba0 0x7f2428156320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2420009870 tx=0x7f242002b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2427e00640 1 -- 172.21.15.67:0/3289513997 shutdown_connections 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2427e00640 1 --2- 172.21.15.67:0/3289513997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242815cba0 0x7f2428156320 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2427e00640 1 --2- 172.21.15.67:0/3289513997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24280c2540 0x7f242815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.053+0000 7f2427e00640 1 -- 172.21.15.67:0/3289513997 >> 172.21.15.67:0/3289513997 conn(0x7f24280c6380 msgr2=0x7f24280c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:14.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.054+0000 7f2427e00640 1 -- 172.21.15.67:0/3289513997 shutdown_connections 2024-02-26T10:40:14.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.054+0000 7f2427e00640 1 -- 172.21.15.67:0/3289513997 wait complete. 2024-02-26T10:40:14.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.055+0000 7f2427e00640 1 Processor -- start 2024-02-26T10:40:14.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.055+0000 7f2427e00640 1 -- start start 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.056+0000 7f2427e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2428071710 0x7f2428153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.056+0000 7f2427e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 0x7f24280a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.056+0000 7f2427e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f242815cba0 0x7f24281f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.056+0000 7f2427e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428159410 con 0x7f24280c2540 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.056+0000 7f2427e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428159290 con 0x7f2428071710 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.056+0000 7f2427e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428071af0 con 0x7f242815cba0 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 0x7f24280a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 0x7f24280a9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36336/0 (socket says 172.21.15.67:36336) 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 -- 172.21.15.67:0/4273024426 learned_addr learned my addr 172.21.15.67:0/4273024426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426a00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2428071710 0x7f2428153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2427400640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f242815cba0 0x7f24281f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 -- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f242815cba0 msgr2=0x7f24281f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f242815cba0 0x7f24281f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 -- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2428071710 msgr2=0x7f2428153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2428071710 0x7f2428153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426000640 1 -- 172.21.15.67:0/4273024426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428154140 con 0x7f24280c2540 2024-02-26T10:40:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.057+0000 7f2426a00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2428071710 0x7f2428153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.058+0000 7f2426000640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 0x7f24280a9e60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2414007e70 tx=0x7f241400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.058+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414014420 con 0x7f24280c2540 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.058+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2414019070 con 0x7f24280c2540 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.058+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280aa3a0 con 0x7f24280c2540 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.058+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414014800 con 0x7f24280c2540 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.058+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280aa670 con 0x7f24280c2540 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.059+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24280678e0 con 0x7f24280c2540 2024-02-26T10:40:14.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.060+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24140149a0 con 0x7f24280c2540 2024-02-26T10:40:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.061+0000 7f241f400640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f240806b4d0 0x7f240806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.061+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f241408ad40 con 0x7f24280c2540 2024-02-26T10:40:14.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.062+0000 7f2426a00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f240806b4d0 0x7f240806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.063+0000 7f2426a00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f240806b4d0 0x7f240806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24180015e0 tx=0x7f2418009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.066+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2414059850 con 0x7f24280c2540 2024-02-26T10:40:14.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.266+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2428074a30 con 0x7f24280c2540 2024-02-26T10:40:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.267+0000 7f241f400640 1 -- 172.21.15.67:0/4273024426 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2414059150 con 0x7f24280c2540 2024-02-26T10:40:14.267 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f240806b4d0 msgr2=0x7f240806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f240806b4d0 0x7f240806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24180015e0 tx=0x7f2418009040 comp rx=0 tx=0).stop 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 msgr2=0x7f24280a9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 0x7f24280a9e60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2414007e70 tx=0x7f241400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 shutdown_connections 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f242815cba0 0x7f24281f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24280c2540 0x7f24280a9e60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f240806b4d0 0x7f240806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 --2- 172.21.15.67:0/4273024426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2428071710 0x7f2428153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.269+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 >> 172.21.15.67:0/4273024426 conn(0x7f24280c6380 msgr2=0x7f24280c0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.270+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 shutdown_connections 2024-02-26T10:40:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.270+0000 7f2427e00640 1 -- 172.21.15.67:0/4273024426 wait complete. 2024-02-26T10:40:14.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:40:14.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.514+0000 7f692be00640 1 Processor -- start 2024-02-26T10:40:14.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.514+0000 7f692be00640 1 -- start start 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.516+0000 7f692be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c1503a0 0x7f692c0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.516+0000 7f692be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f692c06e2d0 0x7f692c0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.516+0000 7f692be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c150780 con 0x7f692c0a9430 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.516+0000 7f692be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c0a98c0 con 0x7f692c06e2d0 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.516+0000 7f692be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0a9ba0 con 0x7f692c1503a0 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.517+0000 7f6923e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f692c0a9430 0x7f692c1535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34302/0 (socket says 172.21.15.67:34302) 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.517+0000 7f6923e00640 1 -- 172.21.15.67:0/3670491689 learned_addr learned my addr 172.21.15.67:0/3670491689 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.517+0000 7f692aa00640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c1503a0 0x7f692c0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.517+0000 7f692b400640 1 --2- 172.21.15.67:0/3670491689 >> v2:172.21.15.82:3300/0 conn(0x7f692c06e2d0 0x7f692c0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.517+0000 7f692a000640 1 -- 172.21.15.67:0/3670491689 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140004639 0 0) 0x7f692c150780 con 0x7f692c0a9430 2024-02-26T10:40:14.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.517+0000 7f692a000640 1 -- 172.21.15.67:0/3670491689 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6910003520 con 0x7f692c0a9430 2024-02-26T10:40:14.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.518+0000 7f692b400640 1 -- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c1503a0 msgr2=0x7f692c0ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.518+0000 7f692b400640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c1503a0 0x7f692c0ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.518+0000 7f692b400640 1 -- 172.21.15.67:0/3670491689 >> v1:172.21.15.67:6789/0 conn(0x7f692c0a9430 legacy=0x7f692c1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.518+0000 7f692b400640 1 -- 172.21.15.67:0/3670491689 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c0a9e80 con 0x7f692c06e2d0 2024-02-26T10:40:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.518+0000 7f692aa00640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c1503a0 0x7f692c0ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:40:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.519+0000 7f692b400640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c06e2d0 0x7f692c0c2e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6924009d90 tx=0x7f692402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01ad605a3496feb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.519+0000 7f692a000640 1 -- 172.21.15.67:0/3670491689 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924039070 con 0x7f692c06e2d0 2024-02-26T10:40:14.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.519+0000 7f692a000640 1 -- 172.21.15.67:0/3670491689 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692402cc50 con 0x7f692c06e2d0 2024-02-26T10:40:14.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.519+0000 7f692a000640 1 -- 172.21.15.67:0/3670491689 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924034440 con 0x7f692c06e2d0 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 -- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c06e2d0 msgr2=0x7f692c0c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c06e2d0 0x7f692c0c2e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6924009d90 tx=0x7f692402bea0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 -- 172.21.15.67:0/3670491689 shutdown_connections 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c06e2d0 0x7f692c0c2e10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 --2- 172.21.15.67:0/3670491689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c1503a0 0x7f692c0ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 -- 172.21.15.67:0/3670491689 >> 172.21.15.67:0/3670491689 conn(0x7f692c0c6380 msgr2=0x7f692c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 -- 172.21.15.67:0/3670491689 shutdown_connections 2024-02-26T10:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.520+0000 7f692be00640 1 -- 172.21.15.67:0/3670491689 wait complete. 2024-02-26T10:40:14.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.523+0000 7f692be00640 1 Processor -- start 2024-02-26T10:40:14.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.523+0000 7f692be00640 1 -- start start 2024-02-26T10:40:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c06e2d0 0x7f692c0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c1503a0 0x7f692c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 0x7f692c149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c0f80 con 0x7f692c14a5f0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c06ddb0 con 0x7f692c1503a0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c06e760 con 0x7f692c06e2d0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 0x7f692c149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 0x7f692c149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36350/0 (socket says 172.21.15.67:36350) 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.524+0000 7f692b400640 1 -- 172.21.15.67:0/1502487246 learned_addr learned my addr 172.21.15.67:0/1502487246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692aa00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c06e2d0 0x7f692c0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f6923e00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c1503a0 0x7f692c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692b400640 1 -- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c06e2d0 msgr2=0x7f692c0c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692b400640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c06e2d0 0x7f692c0c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692b400640 1 -- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c1503a0 msgr2=0x7f692c1499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692b400640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c1503a0 0x7f692c1499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692b400640 1 -- 172.21.15.67:0/1502487246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c1eb430 con 0x7f692c14a5f0 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692aa00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c06e2d0 0x7f692c0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.525+0000 7f692b400640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 0x7f692c149f10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6924009eb0 tx=0x7f6924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.526+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924039050 con 0x7f692c14a5f0 2024-02-26T10:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.526+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924045070 con 0x7f692c14a5f0 2024-02-26T10:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.526+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924009450 con 0x7f692c14a5f0 2024-02-26T10:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.526+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c1eb710 con 0x7f692c14a5f0 2024-02-26T10:40:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.526+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c154140 con 0x7f692c14a5f0 2024-02-26T10:40:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.527+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c0678e0 con 0x7f692c14a5f0 2024-02-26T10:40:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.528+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f692403e610 con 0x7f692c14a5f0 2024-02-26T10:40:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.529+0000 7f6922a00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f690806b4d0 0x7f690806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.529+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f69240ad100 con 0x7f692c14a5f0 2024-02-26T10:40:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.530+0000 7f692aa00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f690806b4d0 0x7f690806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.531+0000 7f692aa00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f690806b4d0 0x7f690806d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f692c1eefe0 tx=0x7f691c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.534+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f692407bd60 con 0x7f692c14a5f0 2024-02-26T10:40:14.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.733+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f692c074a30 con 0x7f692c14a5f0 2024-02-26T10:40:14.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.734+0000 7f6922a00640 1 -- 172.21.15.67:0/1502487246 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f692407b660 con 0x7f692c14a5f0 2024-02-26T10:40:14.734 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f690806b4d0 msgr2=0x7f690806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f690806b4d0 0x7f690806d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f692c1eefe0 tx=0x7f691c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 msgr2=0x7f692c149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 0x7f692c149f10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6924009eb0 tx=0x7f6924004a40 comp rx=0 tx=0).stop 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 shutdown_connections 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f692c14a5f0 0x7f692c149f10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f690806b4d0 0x7f690806d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692c1503a0 0x7f692c1499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 --2- 172.21.15.67:0/1502487246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f692c06e2d0 0x7f692c0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 >> 172.21.15.67:0/1502487246 conn(0x7f692c0c6380 msgr2=0x7f692c153390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 shutdown_connections 2024-02-26T10:40:14.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.737+0000 7f692be00640 1 -- 172.21.15.67:0/1502487246 wait complete. 2024-02-26T10:40:14.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:40:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.985+0000 7f4aef200640 1 Processor -- start 2024-02-26T10:40:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.986+0000 7f4aef200640 1 -- start start 2024-02-26T10:40:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.987+0000 7f4aef200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4af01562f0 0x7f4af006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.987+0000 7f4aef200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af006c110 con 0x7f4af00c5290 2024-02-26T10:40:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.987+0000 7f4aef200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af00c72c0 con 0x7f4af006ce30 2024-02-26T10:40:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.987+0000 7f4aef200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00c75a0 con 0x7f4af01562f0 2024-02-26T10:40:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aed400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4af01562f0 0x7f4af006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aed400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4af01562f0 0x7f4af006c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36362/0 (socket says 172.21.15.67:36362) 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aed400640 1 -- 172.21.15.67:0/82916423 learned_addr learned my addr 172.21.15.67:0/82916423 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aede00640 1 --1- 172.21.15.67:0/82916423 >> v1:172.21.15.67:6789/0 conn(0x7f4af00c5290 0x7f4af0155880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34308/0 (socket says 172.21.15.67:34308) 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aed400640 1 -- 172.21.15.67:0/82916423 >> v1:172.21.15.82:6789/0 conn(0x7f4af006ce30 legacy=0x7f4af00af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aed400640 1 -- 172.21.15.67:0/82916423 >> v1:172.21.15.67:6789/0 conn(0x7f4af00c5290 legacy=0x7f4af0155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.988+0000 7f4aed400640 1 -- 172.21.15.67:0/82916423 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00c7880 con 0x7f4af01562f0 2024-02-26T10:40:14.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.989+0000 7f4aed400640 1 --2- 172.21.15.67:0/82916423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af01562f0 0x7f4af006c8f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009870 tx=0x7f4adc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db1866ced1b029e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.990+0000 7f4aeca00640 1 -- 172.21.15.67:0/82916423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc035020 con 0x7f4af01562f0 2024-02-26T10:40:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.990+0000 7f4aeca00640 1 -- 172.21.15.67:0/82916423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc02d040 con 0x7f4af01562f0 2024-02-26T10:40:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.990+0000 7f4aeca00640 1 -- 172.21.15.67:0/82916423 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc031a80 con 0x7f4af01562f0 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.990+0000 7f4aef200640 1 -- 172.21.15.67:0/82916423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af01562f0 msgr2=0x7f4af006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.990+0000 7f4aef200640 1 --2- 172.21.15.67:0/82916423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af01562f0 0x7f4af006c8f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009870 tx=0x7f4adc02b900 comp rx=0 tx=0).stop 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.991+0000 7f4aef200640 1 -- 172.21.15.67:0/82916423 shutdown_connections 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.991+0000 7f4aef200640 1 --2- 172.21.15.67:0/82916423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af01562f0 0x7f4af006c8f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.991+0000 7f4aef200640 1 -- 172.21.15.67:0/82916423 >> 172.21.15.67:0/82916423 conn(0x7f4af00ac230 msgr2=0x7f4af00ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.991+0000 7f4aef200640 1 -- 172.21.15.67:0/82916423 shutdown_connections 2024-02-26T10:40:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.991+0000 7f4aef200640 1 -- 172.21.15.67:0/82916423 wait complete. 2024-02-26T10:40:14.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.992+0000 7f4aef200640 1 Processor -- start 2024-02-26T10:40:14.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.992+0000 7f4aef200640 1 -- start start 2024-02-26T10:40:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.993+0000 7f4aef200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4af00c5290 0x7f4af00c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.993+0000 7f4aef200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4af01562f0 0x7f4af0150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.993+0000 7f4aef200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 0x7f4af00c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.993+0000 7f4aef200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b0130 con 0x7f4af00c8270 2024-02-26T10:40:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.993+0000 7f4aef200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00affb0 con 0x7f4af00c5290 2024-02-26T10:40:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.993+0000 7f4aef200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00c5670 con 0x7f4af01562f0 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 0x7f4af00c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 0x7f4af00c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36376/0 (socket says 172.21.15.67:36376) 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 -- 172.21.15.67:0/4209328753 learned_addr learned my addr 172.21.15.67:0/4209328753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aede00640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4af00c5290 0x7f4af00c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aed400640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4af01562f0 0x7f4af0150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 -- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4af01562f0 msgr2=0x7f4af0150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4af01562f0 0x7f4af0150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 -- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4af00c5290 msgr2=0x7f4af00c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4af00c5290 0x7f4af00c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 -- 172.21.15.67:0/4209328753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af01edb90 con 0x7f4af00c8270 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aede00640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4af00c5290 0x7f4af00c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.994+0000 7f4aee800640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 0x7f4af00c99a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c3e0 tx=0x7f4ae000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.995+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011790 con 0x7f4af00c8270 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.995+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae0017070 con 0x7f4af00c8270 2024-02-26T10:40:14.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.995+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011bb0 con 0x7f4af00c8270 2024-02-26T10:40:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.995+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af00c4620 con 0x7f4af00c8270 2024-02-26T10:40:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.995+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af00c9240 con 0x7f4af00c8270 2024-02-26T10:40:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.996+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af00c9440 con 0x7f4af00c8270 2024-02-26T10:40:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.997+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ae001c460 con 0x7f4af00c8270 2024-02-26T10:40:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.998+0000 7f4ae6000640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ad006b4d0 0x7f4ad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.998+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4ae008a340 con 0x7f4af00c8270 2024-02-26T10:40:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:14.999+0000 7f4aede00640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ad006b4d0 0x7f4ad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.000+0000 7f4aede00640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ad006b4d0 0x7f4ad006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4af0150ae0 tx=0x7f4ae8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.002+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ae0058e50 con 0x7f4af00c8270 2024-02-26T10:40:15.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.203+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4af0074a30 con 0x7f4af00c8270 2024-02-26T10:40:15.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.203+0000 7f4ae6000640 1 -- 172.21.15.67:0/4209328753 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ae0058750 con 0x7f4af00c8270 2024-02-26T10:40:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ad006b4d0 msgr2=0x7f4ad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ad006b4d0 0x7f4ad006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4af0150ae0 tx=0x7f4ae8009040 comp rx=0 tx=0).stop 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 msgr2=0x7f4af00c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 0x7f4af00c99a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000c3e0 tx=0x7f4ae000c8a0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 shutdown_connections 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4af00c8270 0x7f4af00c99a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4af01562f0 0x7f4af0150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ad006b4d0 0x7f4ad006d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 --2- 172.21.15.67:0/4209328753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4af00c5290 0x7f4af00c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 >> 172.21.15.67:0/4209328753 conn(0x7f4af00ac230 msgr2=0x7f4af0154630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 shutdown_connections 2024-02-26T10:40:15.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.206+0000 7f4aef200640 1 -- 172.21.15.67:0/4209328753 wait complete. 2024-02-26T10:40:15.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:40:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.445+0000 7fb64a800640 1 Processor -- start 2024-02-26T10:40:15.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.445+0000 7fb64a800640 1 -- start start 2024-02-26T10:40:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb64a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb6440ab480 0x7fb6440c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb64a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64406e9e0 con 0x7fb6440a9430 2024-02-26T10:40:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb64a800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb644150780 con 0x7fb6441503a0 2024-02-26T10:40:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb64a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6440ab8d0 con 0x7fb6440ab480 2024-02-26T10:40:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb648a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb6440a9430 0x7fb64406e2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34318/0 (socket says 172.21.15.67:34318) 2024-02-26T10:40:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb648a00640 1 -- 172.21.15.67:0/186236040 learned_addr learned my addr 172.21.15.67:0/186236040 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.447+0000 7fb649e00640 1 --2- 172.21.15.67:0/186236040 >> v2:172.21.15.67:3300/0 conn(0x7fb6440ab480 0x7fb6440c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.448+0000 7fb643e00640 1 -- 172.21.15.67:0/186236040 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754881835 0 0) 0x7fb64406e9e0 con 0x7fb6440a9430 2024-02-26T10:40:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.448+0000 7fb643e00640 1 -- 172.21.15.67:0/186236040 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb62c003520 con 0x7fb6440a9430 2024-02-26T10:40:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.448+0000 7fb649e00640 1 -- 172.21.15.67:0/186236040 >> v1:172.21.15.82:6789/0 conn(0x7fb6441503a0 legacy=0x7fb6441535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.448+0000 7fb649e00640 1 -- 172.21.15.67:0/186236040 >> v1:172.21.15.67:6789/0 conn(0x7fb6440a9430 legacy=0x7fb64406e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.448+0000 7fb649e00640 1 -- 172.21.15.67:0/186236040 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6440a98c0 con 0x7fb6440ab480 2024-02-26T10:40:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.449+0000 7fb649e00640 1 --2- 172.21.15.67:0/186236040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ab480 0x7fb6440c2e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009d90 tx=0x7fb63c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fe31ffa510898a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.449+0000 7fb643e00640 1 -- 172.21.15.67:0/186236040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c039070 con 0x7fb6440ab480 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.449+0000 7fb643e00640 1 -- 172.21.15.67:0/186236040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c02cd00 con 0x7fb6440ab480 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.449+0000 7fb643e00640 1 -- 172.21.15.67:0/186236040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c034640 con 0x7fb6440ab480 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 -- 172.21.15.67:0/186236040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ab480 msgr2=0x7fb6440c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 --2- 172.21.15.67:0/186236040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ab480 0x7fb6440c2e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009d90 tx=0x7fb63c02bf50 comp rx=0 tx=0).stop 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 -- 172.21.15.67:0/186236040 shutdown_connections 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 --2- 172.21.15.67:0/186236040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440ab480 0x7fb6440c2e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 -- 172.21.15.67:0/186236040 >> 172.21.15.67:0/186236040 conn(0x7fb6440c6380 msgr2=0x7fb6440c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 -- 172.21.15.67:0/186236040 shutdown_connections 2024-02-26T10:40:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.450+0000 7fb64a800640 1 -- 172.21.15.67:0/186236040 wait complete. 2024-02-26T10:40:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.452+0000 7fb64a800640 1 Processor -- start 2024-02-26T10:40:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.452+0000 7fb64a800640 1 -- start start 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb64a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6440ab480 0x7fb6441df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb64a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 0x7fb644151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb64a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6441524f0 0x7fb6441e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb64a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64406dad0 con 0x7fb6441503a0 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb64a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440a9e80 con 0x7fb6441524f0 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb64a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64406ddb0 con 0x7fb6440ab480 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb648a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 0x7fb644151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb648a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 0x7fb644151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36390/0 (socket says 172.21.15.67:36390) 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb648a00640 1 -- 172.21.15.67:0/2690629026 learned_addr learned my addr 172.21.15.67:0/2690629026 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb649400640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6440ab480 0x7fb6441df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb649e00640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6441524f0 0x7fb6441e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb648a00640 1 -- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6440ab480 msgr2=0x7fb6441df9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.453+0000 7fb648a00640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6440ab480 0x7fb6441df9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb648a00640 1 -- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6441524f0 msgr2=0x7fb6441e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb648a00640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6441524f0 0x7fb6441e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb648a00640 1 -- 172.21.15.67:0/2690629026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6441ec3d0 con 0x7fb6441503a0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb649400640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6440ab480 0x7fb6441df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb648a00640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 0x7fb644151fb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c500 tx=0x7fb63000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630017070 con 0x7fb6441503a0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630005ce0 con 0x7fb6441503a0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644153b30 con 0x7fb6441503a0 2024-02-26T10:40:15.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630007330 con 0x7fb6441503a0 2024-02-26T10:40:15.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.454+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6440c5fb0 con 0x7fb6441503a0 2024-02-26T10:40:15.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.455+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6440678e0 con 0x7fb6441503a0 2024-02-26T10:40:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.456+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb63001a070 con 0x7fb6441503a0 2024-02-26T10:40:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.457+0000 7fb641600640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb62406b4d0 0x7fb62406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.457+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb630089990 con 0x7fb6441503a0 2024-02-26T10:40:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.458+0000 7fb649400640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb62406b4d0 0x7fb62406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.459+0000 7fb649400640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb62406b4d0 0x7fb62406d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6380015e0 tx=0x7fb638009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.462+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb630058560 con 0x7fb6441503a0 2024-02-26T10:40:15.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.658+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6441ec8b0 con 0x7fb6441503a0 2024-02-26T10:40:15.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.660+0000 7fb641600640 1 -- 172.21.15.67:0/2690629026 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v38) v1 ==== 105+0+11 (secure 0 0 0) 0x7fb630057e60 con 0x7fb6441503a0 2024-02-26T10:40:15.660 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb62406b4d0 msgr2=0x7fb62406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb62406b4d0 0x7fb62406d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6380015e0 tx=0x7fb638009040 comp rx=0 tx=0).stop 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 msgr2=0x7fb644151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 0x7fb644151fb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c500 tx=0x7fb63000c9c0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 shutdown_connections 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb62406b4d0 0x7fb62406d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6441524f0 0x7fb6441e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6441503a0 0x7fb644151fb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 --2- 172.21.15.67:0/2690629026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6440ab480 0x7fb6441df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 >> 172.21.15.67:0/2690629026 conn(0x7fb6440c6380 msgr2=0x7fb644152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:15.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 shutdown_connections 2024-02-26T10:40:15.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.662+0000 7fb64a800640 1 -- 172.21.15.67:0/2690629026 wait complete. 2024-02-26T10:40:15.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:40:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.890+0000 7f85d3000640 1 Processor -- start 2024-02-26T10:40:15.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.891+0000 7f85d3000640 1 -- start start 2024-02-26T10:40:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.892+0000 7f85d3000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85cc1523b0 0x7f85cc0c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.892+0000 7f85d3000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc0c3840 con 0x7f85cc071fd0 2024-02-26T10:40:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.892+0000 7f85d3000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc06dc80 con 0x7f85cc154cf0 2024-02-26T10:40:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.892+0000 7f85d3000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85cc152790 con 0x7f85cc1523b0 2024-02-26T10:40:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85d2600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f85cc071fd0 0x7f85cc0723e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34324/0 (socket says 172.21.15.67:34324) 2024-02-26T10:40:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85d2600640 1 -- 172.21.15.67:0/3224627926 learned_addr learned my addr 172.21.15.67:0/3224627926 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85d1c00640 1 --2- 172.21.15.67:0/3224627926 >> v2:172.21.15.82:3300/0 conn(0x7f85cc1523b0 0x7f85cc0c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030334307 0 0) 0x7f85cc0c3840 con 0x7f85cc071fd0 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b0003520 con 0x7f85cc071fd0 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043334753 0 0) 0x7f85b0003520 con 0x7f85cc071fd0 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 >> v2:172.21.15.82:3300/0 conn(0x7f85cc1523b0 msgr2=0x7f85cc0c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.893+0000 7f85cbe00640 1 --2- 172.21.15.67:0/3224627926 >> v2:172.21.15.82:3300/0 conn(0x7f85cc1523b0 0x7f85cc0c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 >> v1:172.21.15.82:6789/0 conn(0x7f85cc154cf0 legacy=0x7f85cc0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc06df60 con 0x7f85cc071fd0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f85c4001670 con 0x7f85cc071fd0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 >> v1:172.21.15.67:6789/0 conn(0x7f85cc071fd0 legacy=0x7f85cc0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b00039b0 0x7f85b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b0004800 0x7f85b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85b00052e0 0x7f85b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc152790 con 0x7f85b0004800 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b000c1b0 con 0x7f85b00039b0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b000c3b0 con 0x7f85b00052e0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85d1200640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b0004800 0x7f85b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85d1c00640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b00039b0 0x7f85b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.894+0000 7f85d2600640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85b00052e0 0x7f85b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.895+0000 7f85d1200640 1 -- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85b00052e0 msgr2=0x7f85b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.895+0000 7f85d1200640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85b00052e0 0x7f85b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.895+0000 7f85d1200640 1 -- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b00039b0 msgr2=0x7f85b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.895+0000 7f85d1200640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b00039b0 0x7f85b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.895+0000 7f85d1200640 1 -- 172.21.15.67:0/3224627926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0003520 con 0x7f85b0004800 2024-02-26T10:40:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.895+0000 7f85d2600640 1 -- 172.21.15.67:0/3224627926 reap_dead start 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.896+0000 7f85d1200640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b0004800 0x7f85b0004c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001720 tx=0x7f85b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fc0afe167bce9b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.896+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8001370 con 0x7f85b0004800 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.896+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b80052d0 con 0x7f85b0004800 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.896+0000 7f85cbe00640 1 -- 172.21.15.67:0/3224627926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8003830 con 0x7f85b0004800 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 -- 172.21.15.67:0/3224627926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b0004800 msgr2=0x7f85b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b0004800 0x7f85b0004c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001720 tx=0x7f85b0005830 comp rx=0 tx=0).stop 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 -- 172.21.15.67:0/3224627926 shutdown_connections 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 --2- 172.21.15.67:0/3224627926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b0004800 0x7f85b0004c80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 -- 172.21.15.67:0/3224627926 >> 172.21.15.67:0/3224627926 conn(0x7f85cc0c6380 msgr2=0x7f85cc0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 -- 172.21.15.67:0/3224627926 shutdown_connections 2024-02-26T10:40:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.897+0000 7f85d3000640 1 -- 172.21.15.67:0/3224627926 wait complete. 2024-02-26T10:40:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.899+0000 7f85d3000640 1 Processor -- start 2024-02-26T10:40:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.899+0000 7f85d3000640 1 -- start start 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.899+0000 7f85d3000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b0004800 0x7f85cc1461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 0x7f85cc1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d3000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc148d00 0x7f85cc1edde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc06e520 con 0x7f85cc1470a0 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d3000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc06da30 con 0x7f85b0004800 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d3000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc06e800 con 0x7f85cc148d00 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 0x7f85cc1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 0x7f85cc1487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36398/0 (socket says 172.21.15.67:36398) 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 -- 172.21.15.67:0/2917166526 learned_addr learned my addr 172.21.15.67:0/2917166526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1c00640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b0004800 0x7f85cc1461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d2600640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc148d00 0x7f85cc1edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 -- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc148d00 msgr2=0x7f85cc1edde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc148d00 0x7f85cc1edde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 -- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b0004800 msgr2=0x7f85cc1461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b0004800 0x7f85cc1461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1200640 1 -- 172.21.15.67:0/2917166526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc158b80 con 0x7f85cc1470a0 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d2600640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc148d00 0x7f85cc1edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.900+0000 7f85d1c00640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b0004800 0x7f85cc1461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.901+0000 7f85d1200640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 0x7f85cc1487c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005040 tx=0x7f85b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.901+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8041070 con 0x7f85cc1470a0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.901+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8005480 con 0x7f85cc1470a0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.901+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8007a30 con 0x7f85cc1470a0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.901+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc147ce0 con 0x7f85cc1470a0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.901+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc158810 con 0x7f85cc1470a0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.902+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85cc0678e0 con 0x7f85cc1470a0 2024-02-26T10:40:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.903+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85b8044070 con 0x7f85cc1470a0 2024-02-26T10:40:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.904+0000 7f85c9600640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85ac06b4d0 0x7f85ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.904+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f85b80ac1d0 con 0x7f85cc1470a0 2024-02-26T10:40:15.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.905+0000 7f85d1c00640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85ac06b4d0 0x7f85ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.906+0000 7f85d1c00640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85ac06b4d0 0x7f85ac06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f85cc068340 tx=0x7f85bc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:15.909+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85b807abc0 con 0x7f85cc1470a0 2024-02-26T10:40:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.108+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f85cc0ac4c0 con 0x7f85cc1470a0 2024-02-26T10:40:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.109+0000 7f85c9600640 1 -- 172.21.15.67:0/2917166526 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v38) v1 ==== 101+0+11 (secure 0 0 0) 0x7f85b807a4c0 con 0x7f85cc1470a0 2024-02-26T10:40:16.109 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.111+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85ac06b4d0 msgr2=0x7f85ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.111+0000 7f85d3000640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85ac06b4d0 0x7f85ac06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f85cc068340 tx=0x7f85bc00b040 comp rx=0 tx=0).stop 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.111+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 msgr2=0x7f85cc1487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.111+0000 7f85d3000640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 0x7f85cc1487c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005040 tx=0x7f85b8001120 comp rx=0 tx=0).stop 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 shutdown_connections 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc148d00 0x7f85cc1edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc1470a0 0x7f85cc1487c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85ac06b4d0 0x7f85ac06d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 --2- 172.21.15.67:0/2917166526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b0004800 0x7f85cc1461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 >> 172.21.15.67:0/2917166526 conn(0x7f85cc0c6380 msgr2=0x7f85cc152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:16.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 shutdown_connections 2024-02-26T10:40:16.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.112+0000 7f85d3000640 1 -- 172.21.15.67:0/2917166526 wait complete. 2024-02-26T10:40:16.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:40:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.339+0000 7fef25c00640 1 Processor -- start 2024-02-26T10:40:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.340+0000 7fef25c00640 1 -- start start 2024-02-26T10:40:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.341+0000 7fef25c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fef20152380 0x7fef200c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef25c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fef2006d290 0x7fef200c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef25c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef20152760 con 0x7fef200c7020 2024-02-26T10:40:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef25c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200c7550 con 0x7fef2006d290 2024-02-26T10:40:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef25c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200c7830 con 0x7fef20152380 2024-02-26T10:40:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef1f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fef200c7020 0x7fef20155790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34332/0 (socket says 172.21.15.67:34332) 2024-02-26T10:40:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef1f400640 1 -- 172.21.15.67:0/3868558988 learned_addr learned my addr 172.21.15.67:0/3868558988 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef25200640 1 --2- 172.21.15.67:0/3868558988 >> v2:172.21.15.67:3300/0 conn(0x7fef2006d290 0x7fef200c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef1fe00640 1 --2- 172.21.15.67:0/3868558988 >> v2:172.21.15.82:3300/0 conn(0x7fef20152380 0x7fef200c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef1ea00640 1 -- 172.21.15.67:0/3868558988 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140821144 0 0) 0x7fef20152760 con 0x7fef200c7020 2024-02-26T10:40:16.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.342+0000 7fef1ea00640 1 -- 172.21.15.67:0/3868558988 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef04003520 con 0x7fef200c7020 2024-02-26T10:40:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.343+0000 7fef25200640 1 -- 172.21.15.67:0/3868558988 >> v2:172.21.15.82:3300/0 conn(0x7fef20152380 msgr2=0x7fef200c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.343+0000 7fef25200640 1 --2- 172.21.15.67:0/3868558988 >> v2:172.21.15.82:3300/0 conn(0x7fef20152380 0x7fef200c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.343+0000 7fef25200640 1 -- 172.21.15.67:0/3868558988 >> v1:172.21.15.67:6789/0 conn(0x7fef200c7020 legacy=0x7fef20155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.343+0000 7fef25200640 1 -- 172.21.15.67:0/3868558988 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2006ca90 con 0x7fef2006d290 2024-02-26T10:40:16.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.344+0000 7fef25200640 1 --2- 172.21.15.67:0/3868558988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef2006d290 0x7fef200c4f90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fef2006ddd0 tx=0x7fef1802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29749b10a70ef217 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.344+0000 7fef1ea00640 1 -- 172.21.15.67:0/3868558988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180315a0 con 0x7fef2006d290 2024-02-26T10:40:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.344+0000 7fef1ea00640 1 -- 172.21.15.67:0/3868558988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef18037070 con 0x7fef2006d290 2024-02-26T10:40:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.344+0000 7fef1ea00640 1 -- 172.21.15.67:0/3868558988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18031940 con 0x7fef2006d290 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 -- 172.21.15.67:0/3868558988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef2006d290 msgr2=0x7fef200c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 --2- 172.21.15.67:0/3868558988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef2006d290 0x7fef200c4f90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fef2006ddd0 tx=0x7fef1802b840 comp rx=0 tx=0).stop 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 -- 172.21.15.67:0/3868558988 shutdown_connections 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 --2- 172.21.15.67:0/3868558988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef2006d290 0x7fef200c4f90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 --2- 172.21.15.67:0/3868558988 >> v2:172.21.15.82:3300/0 conn(0x7fef20152380 0x7fef200c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 -- 172.21.15.67:0/3868558988 >> 172.21.15.67:0/3868558988 conn(0x7fef200ac1a0 msgr2=0x7fef200ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 -- 172.21.15.67:0/3868558988 shutdown_connections 2024-02-26T10:40:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.345+0000 7fef25c00640 1 -- 172.21.15.67:0/3868558988 wait complete. 2024-02-26T10:40:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.347+0000 7fef25c00640 1 Processor -- start 2024-02-26T10:40:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.348+0000 7fef25c00640 1 -- start start 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef2006d290 0x7fef201ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef200c7020 0x7fef200c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 0x7fef201ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2006d880 con 0x7fef20152380 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2006d700 con 0x7fef200c7020 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2006da00 con 0x7fef2006d290 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 0x7fef201ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 0x7fef201ee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36416/0 (socket says 172.21.15.67:36416) 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25200640 1 -- 172.21.15.67:0/2550977782 learned_addr learned my addr 172.21.15.67:0/2550977782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:16.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef1fe00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef2006d290 0x7fef201ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef1f400640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef200c7020 0x7fef200c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.349+0000 7fef25200640 1 -- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef2006d290 msgr2=0x7fef201ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25200640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef2006d290 0x7fef201ea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25200640 1 -- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef200c7020 msgr2=0x7fef200c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25200640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef200c7020 0x7fef200c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25200640 1 -- 172.21.15.67:0/2550977782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef201561a0 con 0x7fef20152380 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef1fe00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef2006d290 0x7fef201ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25200640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 0x7fef201ee3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fef18034040 tx=0x7fef180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18032630 con 0x7fef20152380 2024-02-26T10:40:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef180327f0 con 0x7fef20152380 2024-02-26T10:40:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18032be0 con 0x7fef20152380 2024-02-26T10:40:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef20156440 con 0x7fef20152380 2024-02-26T10:40:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.350+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef200c8cc0 con 0x7fef20152380 2024-02-26T10:40:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.351+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef20074a30 con 0x7fef20152380 2024-02-26T10:40:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.352+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef1804c020 con 0x7fef20152380 2024-02-26T10:40:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.354+0000 7fef17e00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feefc06b4d0 0x7feefc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.354+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fef1802f020 con 0x7fef20152380 2024-02-26T10:40:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.354+0000 7fef1fe00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feefc06b4d0 0x7feefc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.355+0000 7fef1fe00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feefc06b4d0 0x7feefc06d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef10007440 tx=0x7fef1000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.359+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef18040070 con 0x7fef20152380 2024-02-26T10:40:16.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.572+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef200c7c50 con 0x7fef20152380 2024-02-26T10:40:16.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.573+0000 7fef17e00640 1 -- 172.21.15.67:0/2550977782 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7fef1807ab90 con 0x7fef20152380 2024-02-26T10:40:16.574 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:40:16.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feefc06b4d0 msgr2=0x7feefc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feefc06b4d0 0x7feefc06d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef10007440 tx=0x7fef1000b040 comp rx=0 tx=0).stop 2024-02-26T10:40:16.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 msgr2=0x7fef201ee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 0x7fef201ee3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fef18034040 tx=0x7fef180046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 shutdown_connections 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef20152380 0x7fef201ee3b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feefc06b4d0 0x7feefc06d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef200c7020 0x7fef200c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 --2- 172.21.15.67:0/2550977782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef2006d290 0x7fef201ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 >> 172.21.15.67:0/2550977782 conn(0x7fef200ac1a0 msgr2=0x7fef20155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.576+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 shutdown_connections 2024-02-26T10:40:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.577+0000 7fef25c00640 1 -- 172.21.15.67:0/2550977782 wait complete. 2024-02-26T10:40:16.596 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:40:16.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:16.597 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:16.711 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:40:16.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:16.711 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:40:16.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.991+0000 7fd543200640 1 Processor -- start 2024-02-26T10:40:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.991+0000 7fd543200640 1 -- start start 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd543200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd5440c1c10 0x7fd5441557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd543200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd54406e180 0x7fd54406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd543200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd544155cf0 con 0x7fd5440c0850 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd543200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54406dca0 con 0x7fd54406e180 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd543200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5440c2a70 con 0x7fd5440c1c10 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd542800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd54406e180 0x7fd54406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd542800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd54406e180 0x7fd54406e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36430/0 (socket says 172.21.15.67:36430) 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.993+0000 7fd542800640 1 -- 172.21.15.67:0/2321256188 learned_addr learned my addr 172.21.15.67:0/2321256188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.994+0000 7fd541e00640 1 --2- 172.21.15.67:0/2321256188 >> v2:172.21.15.82:3300/0 conn(0x7fd5440c1c10 0x7fd5441557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:16.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.994+0000 7fd542800640 1 -- 172.21.15.67:0/2321256188 >> v2:172.21.15.82:3300/0 conn(0x7fd5440c1c10 msgr2=0x7fd5441557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:16.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.994+0000 7fd542800640 1 --2- 172.21.15.67:0/2321256188 >> v2:172.21.15.82:3300/0 conn(0x7fd5440c1c10 0x7fd5441557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.994+0000 7fd542800640 1 -- 172.21.15.67:0/2321256188 >> v1:172.21.15.67:6789/0 conn(0x7fd5440c0850 legacy=0x7fd54406ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.994+0000 7fd542800640 1 -- 172.21.15.67:0/2321256188 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440c21d0 con 0x7fd54406e180 2024-02-26T10:40:16.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.995+0000 7fd542800640 1 --2- 172.21.15.67:0/2321256188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd54406e630 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd5440bfbd0 tx=0x7fd53c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2e79a2c3b113d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:16.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.995+0000 7fd540a00640 1 -- 172.21.15.67:0/2321256188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c02c9f0 con 0x7fd54406e180 2024-02-26T10:40:16.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.995+0000 7fd540a00640 1 -- 172.21.15.67:0/2321256188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c02cb90 con 0x7fd54406e180 2024-02-26T10:40:16.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.995+0000 7fd540a00640 1 -- 172.21.15.67:0/2321256188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0343f0 con 0x7fd54406e180 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 -- 172.21.15.67:0/2321256188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 msgr2=0x7fd54406e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 --2- 172.21.15.67:0/2321256188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd54406e630 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd5440bfbd0 tx=0x7fd53c02be50 comp rx=0 tx=0).stop 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 -- 172.21.15.67:0/2321256188 shutdown_connections 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 --2- 172.21.15.67:0/2321256188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd54406e630 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 --2- 172.21.15.67:0/2321256188 >> v2:172.21.15.82:3300/0 conn(0x7fd5440c1c10 0x7fd5441557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 -- 172.21.15.67:0/2321256188 >> 172.21.15.67:0/2321256188 conn(0x7fd5440c63f0 msgr2=0x7fd5440c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 -- 172.21.15.67:0/2321256188 shutdown_connections 2024-02-26T10:40:16.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.996+0000 7fd543200640 1 -- 172.21.15.67:0/2321256188 wait complete. 2024-02-26T10:40:16.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.999+0000 7fd543200640 1 Processor -- start 2024-02-26T10:40:16.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:16.999+0000 7fd543200640 1 -- start start 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd543200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd544156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd543200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5440c0850 0x7fd544150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd543200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5440c1c10 0x7fd544158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd543200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54406b330 con 0x7fd54406e180 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd543200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440c2790 con 0x7fd5440c1c10 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd543200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54406b4b0 con 0x7fd5440c0850 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd541e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd544156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd541e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd544156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36432/0 (socket says 172.21.15.67:36432) 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.000+0000 7fd541e00640 1 -- 172.21.15.67:0/3310812839 learned_addr learned my addr 172.21.15.67:0/3310812839 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd542800640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5440c1c10 0x7fd544158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541400640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5440c0850 0x7fd544150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541e00640 1 -- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5440c0850 msgr2=0x7fd544150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541e00640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5440c0850 0x7fd544150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541e00640 1 -- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5440c1c10 msgr2=0x7fd544158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541e00640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5440c1c10 0x7fd544158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541e00640 1 -- 172.21.15.67:0/3310812839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd544159090 con 0x7fd54406e180 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541400640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5440c0850 0x7fd544150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd541e00640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd544156a40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd53400d5f0 tx=0x7fd53400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340125f0 con 0x7fd54406e180 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534019070 con 0x7fd54406e180 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340129d0 con 0x7fd54406e180 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.001+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5441eb380 con 0x7fd54406e180 2024-02-26T10:40:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.002+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54414bbf0 con 0x7fd54406e180 2024-02-26T10:40:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.003+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd544074a30 con 0x7fd54406e180 2024-02-26T10:40:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.004+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd534012b70 con 0x7fd54406e180 2024-02-26T10:40:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.004+0000 7fd53a000640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd52406b4d0 0x7fd52406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.005+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd53408a960 con 0x7fd54406e180 2024-02-26T10:40:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.005+0000 7fd541400640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd52406b4d0 0x7fd52406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.006+0000 7fd541400640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd52406b4d0 0x7fd52406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd54414eb40 tx=0x7fd530009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.009+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd534059470 con 0x7fd54406e180 2024-02-26T10:40:17.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.246+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd544159260 con 0x7fd54406e180 2024-02-26T10:40:17.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.251+0000 7fd53a000640 1 -- 172.21.15.67:0/3310812839 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+6171 (secure 0 0 0) 0x7fd534058d70 con 0x7fd54406e180 2024-02-26T10:40:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:17.252 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:17.252 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 22 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd52406b4d0 msgr2=0x7fd52406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd52406b4d0 0x7fd52406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd54414eb40 tx=0x7fd530009040 comp rx=0 tx=0).stop 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 msgr2=0x7fd544156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd544156a40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd53400d5f0 tx=0x7fd53400dab0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 shutdown_connections 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd52406b4d0 0x7fd52406d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5440c1c10 0x7fd544158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5440c0850 0x7fd544150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 --2- 172.21.15.67:0/3310812839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd54406e180 0x7fd544156a40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.254+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 >> 172.21.15.67:0/3310812839 conn(0x7fd5440c63f0 msgr2=0x7fd544152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.255+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 shutdown_connections 2024-02-26T10:40:17.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.255+0000 7fd543200640 1 -- 172.21.15.67:0/3310812839 wait complete. 2024-02-26T10:40:17.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:17.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.498+0000 7fdca5000640 1 Processor -- start 2024-02-26T10:40:17.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.498+0000 7fdca5000640 1 -- start start 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.499+0000 7fdca5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdca006ac20 0x7fdca00a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.499+0000 7fdca5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca006b460 con 0x7fdca006db00 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.499+0000 7fdca5000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca006df90 con 0x7fdca006b000 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.499+0000 7fdca5000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca00a9a80 con 0x7fdca006ac20 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdca006ac20 0x7fdca00a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdca006db00 0x7fdca00abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34352/0 (socket says 172.21.15.67:34352) 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdca006ac20 0x7fdca00a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36448/0 (socket says 172.21.15.67:36448) 2024-02-26T10:40:17.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9fe00640 1 -- 172.21.15.67:0/3990517951 learned_addr learned my addr 172.21.15.67:0/3990517951 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:17.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9f400640 1 -- 172.21.15.67:0/3990517951 >> v1:172.21.15.82:6789/0 conn(0x7fdca006b000 legacy=0x7fdca0153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:17.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9f400640 1 -- 172.21.15.67:0/3990517951 >> v1:172.21.15.67:6789/0 conn(0x7fdca006db00 legacy=0x7fdca00abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.500+0000 7fdc9f400640 1 -- 172.21.15.67:0/3990517951 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00a9d60 con 0x7fdca006ac20 2024-02-26T10:40:17.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.501+0000 7fdc9f400640 1 --2- 172.21.15.67:0/3990517951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006ac20 0x7fdca00a94a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc900097a0 tx=0x7fdc9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423a392a70164eab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.502+0000 7fdc9ea00640 1 -- 172.21.15.67:0/3990517951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90035020 con 0x7fdca006ac20 2024-02-26T10:40:17.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.502+0000 7fdc9ea00640 1 -- 172.21.15.67:0/3990517951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90031830 con 0x7fdca006ac20 2024-02-26T10:40:17.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.502+0000 7fdc9ea00640 1 -- 172.21.15.67:0/3990517951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90031c00 con 0x7fdca006ac20 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.502+0000 7fdca5000640 1 -- 172.21.15.67:0/3990517951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006ac20 msgr2=0x7fdca00a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.502+0000 7fdca5000640 1 --2- 172.21.15.67:0/3990517951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006ac20 0x7fdca00a94a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc900097a0 tx=0x7fdc9002b8d0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.503+0000 7fdca5000640 1 -- 172.21.15.67:0/3990517951 shutdown_connections 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.503+0000 7fdca5000640 1 --2- 172.21.15.67:0/3990517951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006ac20 0x7fdca00a94a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.503+0000 7fdca5000640 1 -- 172.21.15.67:0/3990517951 >> 172.21.15.67:0/3990517951 conn(0x7fdca00c63f0 msgr2=0x7fdca00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.503+0000 7fdca5000640 1 -- 172.21.15.67:0/3990517951 shutdown_connections 2024-02-26T10:40:17.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.503+0000 7fdca5000640 1 -- 172.21.15.67:0/3990517951 wait complete. 2024-02-26T10:40:17.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.504+0000 7fdca5000640 1 Processor -- start 2024-02-26T10:40:17.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.505+0000 7fdca5000640 1 -- start start 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.505+0000 7fdca5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca006ac20 0x7fdca014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.505+0000 7fdca5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 0x7fdca01dd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.505+0000 7fdca5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca006db00 0x7fdca01ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdca5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca0150410 con 0x7fdca006b000 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdca5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00aba30 con 0x7fdca006db00 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdca5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca0150590 con 0x7fdca006ac20 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 0x7fdca01dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 0x7fdca01dd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36454/0 (socket says 172.21.15.67:36454) 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 -- 172.21.15.67:0/2293507235 learned_addr learned my addr 172.21.15.67:0/2293507235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc9f400640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca006ac20 0x7fdca014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 -- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca006ac20 msgr2=0x7fdca014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc9fe00640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca006db00 0x7fdca01ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca006ac20 0x7fdca014ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 -- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca006db00 msgr2=0x7fdca01ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca006db00 0x7fdca01ddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.506+0000 7fdc97e00640 1 -- 172.21.15.67:0/2293507235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca014a930 con 0x7fdca006b000 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.507+0000 7fdc97e00640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 0x7fdca01dd750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c400 tx=0x7fdc8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.507+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88011790 con 0x7fdca006b000 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.507+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88017070 con 0x7fdca006b000 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.507+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca014a090 con 0x7fdca006b000 2024-02-26T10:40:17.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.507+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88011bb0 con 0x7fdca006b000 2024-02-26T10:40:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.507+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca01eb560 con 0x7fdca006b000 2024-02-26T10:40:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.508+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca006bea0 con 0x7fdca006b000 2024-02-26T10:40:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.509+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc8801c460 con 0x7fdca006b000 2024-02-26T10:40:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.510+0000 7fdc97400640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.510+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fdc8808a3c0 con 0x7fdca006b000 2024-02-26T10:40:17.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.511+0000 7fdc9f400640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.512+0000 7fdc9f400640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc90036000 tx=0x7fdc90008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.514+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdca006bea0 con 0x7fdca006b000 2024-02-26T10:40:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.715+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdca0074a30 con 0x7fdca006b000 2024-02-26T10:40:17.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.719+0000 7fdc97400640 1 -- 172.21.15.67:0/2293507235 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+20323 (secure 0 0 0) 0x7fdc88058ed0 con 0x7fdca006b000 2024-02-26T10:40:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:17.720 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:57.124601+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:17.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc7c06b4d0 msgr2=0x7fdc7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc90036000 tx=0x7fdc90008030 comp rx=0 tx=0).stop 2024-02-26T10:40:17.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 msgr2=0x7fdca01dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 0x7fdca01dd750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c400 tx=0x7fdc8800c8c0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 shutdown_connections 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca006db00 0x7fdca01ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdca006b000 0x7fdca01dd750 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 --2- 172.21.15.67:0/2293507235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca006ac20 0x7fdca014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 >> 172.21.15.67:0/2293507235 conn(0x7fdca00c63f0 msgr2=0x7fdca014f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 shutdown_connections 2024-02-26T10:40:17.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.722+0000 7fdca5000640 1 -- 172.21.15.67:0/2293507235 wait complete. 2024-02-26T10:40:17.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:39:37.371486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:39:38.475801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:39:53.146836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:39:54.623593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:17.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.968+0000 7f1b06400640 1 Processor -- start 2024-02-26T10:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.969+0000 7f1b06400640 1 -- start start 2024-02-26T10:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.970+0000 7f1b06400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b000c5290 0x7f1b001562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.970+0000 7f1b06400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 0x7f1b0006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1b06400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0006c110 con 0x7f1b0006ce30 2024-02-26T10:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1b06400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b000c72c0 con 0x7f1b000c5290 2024-02-26T10:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1b06400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000c75a0 con 0x7f1b00155880 2024-02-26T10:40:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1affe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 0x7f1b0006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1affe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 0x7f1b0006c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:42016/0 (socket says 172.21.15.67:42016) 2024-02-26T10:40:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1affe00640 1 -- 172.21.15.67:0/618015489 learned_addr learned my addr 172.21.15.67:0/618015489 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1b05000640 1 --2- 172.21.15.67:0/618015489 >> v2:172.21.15.82:3300/0 conn(0x7f1b000c5290 0x7f1b001562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.971+0000 7f1b05a00640 1 --1- 172.21.15.67:0/618015489 >> v1:172.21.15.82:6789/0 conn(0x7f1b0006ce30 0x7f1b000af8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:55134/0 (socket says 172.21.15.67:55134) 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.972+0000 7f1affe00640 1 -- 172.21.15.67:0/618015489 >> v2:172.21.15.82:3300/0 conn(0x7f1b000c5290 msgr2=0x7f1b001562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.972+0000 7f1affe00640 1 --2- 172.21.15.67:0/618015489 >> v2:172.21.15.82:3300/0 conn(0x7f1b000c5290 0x7f1b001562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.972+0000 7f1aff400640 1 -- 172.21.15.67:0/618015489 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504137971 0 0) 0x7f1b0006c110 con 0x7f1b0006ce30 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.972+0000 7f1affe00640 1 -- 172.21.15.67:0/618015489 >> v1:172.21.15.82:6789/0 conn(0x7f1b0006ce30 legacy=0x7f1b000af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.972+0000 7f1affe00640 1 -- 172.21.15.67:0/618015489 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000c7880 con 0x7f1b00155880 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.972+0000 7f1b05000640 1 --2- 172.21.15.67:0/618015489 >> v2:172.21.15.82:3300/0 conn(0x7f1b000c5290 0x7f1b001562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:40:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.973+0000 7f1affe00640 1 --2- 172.21.15.67:0/618015489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 0x7f1b0006c8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009e50 tx=0x7f1ae802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4303fd206b35b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.974+0000 7f1aff400640 1 -- 172.21.15.67:0/618015489 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae802ca80 con 0x7f1b00155880 2024-02-26T10:40:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.974+0000 7f1aff400640 1 -- 172.21.15.67:0/618015489 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae802cc20 con 0x7f1b00155880 2024-02-26T10:40:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.974+0000 7f1aff400640 1 -- 172.21.15.67:0/618015489 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80343f0 con 0x7f1b00155880 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.974+0000 7f1b06400640 1 -- 172.21.15.67:0/618015489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 msgr2=0x7f1b0006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.974+0000 7f1b06400640 1 --2- 172.21.15.67:0/618015489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 0x7f1b0006c8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009e50 tx=0x7f1ae802bee0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.975+0000 7f1b06400640 1 -- 172.21.15.67:0/618015489 shutdown_connections 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.975+0000 7f1b06400640 1 --2- 172.21.15.67:0/618015489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b00155880 0x7f1b0006c8f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.975+0000 7f1b06400640 1 --2- 172.21.15.67:0/618015489 >> v2:172.21.15.82:3300/0 conn(0x7f1b000c5290 0x7f1b001562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.975+0000 7f1b06400640 1 -- 172.21.15.67:0/618015489 >> 172.21.15.67:0/618015489 conn(0x7f1b000ac230 msgr2=0x7f1b000ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.975+0000 7f1b06400640 1 -- 172.21.15.67:0/618015489 shutdown_connections 2024-02-26T10:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.975+0000 7f1b06400640 1 -- 172.21.15.67:0/618015489 wait complete. 2024-02-26T10:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.976+0000 7f1b06400640 1 Processor -- start 2024-02-26T10:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.977+0000 7f1b06400640 1 -- start start 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b06400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b000c5290 0x7f1b001ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b06400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 0x7f1b000c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b06400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b000c2c90 0x7f1b000c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b06400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000affb0 con 0x7f1b00155880 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b06400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00155cc0 con 0x7f1b000c2c90 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b06400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000b0130 con 0x7f1b000c5290 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 0x7f1b000c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 0x7f1b000c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36462/0 (socket says 172.21.15.67:36462) 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 -- 172.21.15.67:0/4245837587 learned_addr learned my addr 172.21.15.67:0/4245837587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b05a00640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b000c2c90 0x7f1b000c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1b05000640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b000c5290 0x7f1b001ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 -- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b000c5290 msgr2=0x7f1b001ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b000c5290 0x7f1b001ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 -- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b000c2c90 msgr2=0x7f1b000c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.978+0000 7f1affe00640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b000c2c90 0x7f1b000c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1affe00640 1 -- 172.21.15.67:0/4245837587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000c3200 con 0x7f1b00155880 2024-02-26T10:40:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1affe00640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 0x7f1b000c9530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009e50 tx=0x7f1ae8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8043020 con 0x7f1b00155880 2024-02-26T10:40:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae803e450 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8009510 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b000c34e0 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.979+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b001edcd0 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.981+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b00074a30 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.981+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ae8005d10 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.982+0000 7f1afcc00640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1adc06b4d0 0x7f1adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.982+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1ae80ad010 con 0x7f1b00155880 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.983+0000 7f1b05000640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1adc06b4d0 0x7f1adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.984+0000 7f1b05000640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1adc06b4d0 0x7f1adc06d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009a30 tx=0x7f1af0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:17.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:17.987+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ae807bb40 con 0x7f1b00155880 2024-02-26T10:40:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.185+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b0006d950 con 0x7f1b00155880 2024-02-26T10:40:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.186+0000 7f1afcc00640 1 -- 172.21.15.67:0/4245837587 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ae807b440 con 0x7f1b00155880 2024-02-26T10:40:18.187 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1adc06b4d0 msgr2=0x7f1adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1adc06b4d0 0x7f1adc06d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009a30 tx=0x7f1af0008040 comp rx=0 tx=0).stop 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 msgr2=0x7f1b000c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 0x7f1b000c9530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009e50 tx=0x7f1ae8004a40 comp rx=0 tx=0).stop 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 shutdown_connections 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1adc06b4d0 0x7f1adc06d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b000c2c90 0x7f1b000c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b00155880 0x7f1b000c9530 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 --2- 172.21.15.67:0/4245837587 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b000c5290 0x7f1b001ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 >> 172.21.15.67:0/4245837587 conn(0x7f1b000ac230 msgr2=0x7f1b00158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 shutdown_connections 2024-02-26T10:40:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.189+0000 7f1b06400640 1 -- 172.21.15.67:0/4245837587 wait complete. 2024-02-26T10:40:18.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:40:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.430+0000 7fa89f200640 1 Processor -- start 2024-02-26T10:40:18.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.430+0000 7fa89f200640 1 -- start start 2024-02-26T10:40:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa89f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa8a00c25a0 0x7fa8a00c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa89f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a0151980 0x7fa8a0151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa89f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a00c1be0 con 0x7fa8a00c1eb0 2024-02-26T10:40:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa89f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00c1750 con 0x7fa8a00c25a0 2024-02-26T10:40:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa89f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c3660 con 0x7fa8a0151980 2024-02-26T10:40:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa89de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa8a00c25a0 0x7fa8a00c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa895400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa8a00c1eb0 0x7fa8a014ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34358/0 (socket says 172.21.15.67:34358) 2024-02-26T10:40:18.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.432+0000 7fa895400640 1 -- 172.21.15.67:0/2916606254 learned_addr learned my addr 172.21.15.67:0/2916606254 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.433+0000 7fa89e800640 1 --2- 172.21.15.67:0/2916606254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a0151980 0x7fa8a0151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.433+0000 7fa89de00640 1 -- 172.21.15.67:0/2916606254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a0151980 msgr2=0x7fa8a0151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.433+0000 7fa89de00640 1 --2- 172.21.15.67:0/2916606254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a0151980 0x7fa8a0151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.433+0000 7fa89de00640 1 -- 172.21.15.67:0/2916606254 >> v1:172.21.15.67:6789/0 conn(0x7fa8a00c1eb0 legacy=0x7fa8a014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.433+0000 7fa89de00640 1 -- 172.21.15.67:0/2916606254 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00c0e20 con 0x7fa8a00c25a0 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.433+0000 7fa89d400640 1 -- 172.21.15.67:0/2916606254 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419187455 0 0) 0x7fa8a00c1be0 con 0x7fa8a00c1eb0 2024-02-26T10:40:18.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.434+0000 7fa89de00640 1 --2- 172.21.15.67:0/2916606254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c2e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa890009d90 tx=0x7fa89002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb35fc9e5de38240 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.434+0000 7fa89d400640 1 -- 172.21.15.67:0/2916606254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890039070 con 0x7fa8a00c25a0 2024-02-26T10:40:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.434+0000 7fa89d400640 1 -- 172.21.15.67:0/2916606254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89002cbc0 con 0x7fa8a00c25a0 2024-02-26T10:40:18.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.434+0000 7fa89d400640 1 -- 172.21.15.67:0/2916606254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890034440 con 0x7fa8a00c25a0 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 -- 172.21.15.67:0/2916606254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 msgr2=0x7fa8a00c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 --2- 172.21.15.67:0/2916606254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c2e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa890009d90 tx=0x7fa89002be10 comp rx=0 tx=0).stop 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 -- 172.21.15.67:0/2916606254 shutdown_connections 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 --2- 172.21.15.67:0/2916606254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a0151980 0x7fa8a0151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 --2- 172.21.15.67:0/2916606254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c2e10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 -- 172.21.15.67:0/2916606254 >> 172.21.15.67:0/2916606254 conn(0x7fa8a00c6380 msgr2=0x7fa8a00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 -- 172.21.15.67:0/2916606254 shutdown_connections 2024-02-26T10:40:18.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.435+0000 7fa89f200640 1 -- 172.21.15.67:0/2916606254 wait complete. 2024-02-26T10:40:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.437+0000 7fa89f200640 1 Processor -- start 2024-02-26T10:40:18.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.437+0000 7fa89f200640 1 -- start start 2024-02-26T10:40:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa89f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a00c1eb0 0x7fa8a0144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa89f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa89f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a0151980 0x7fa8a0152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa89f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c3350 con 0x7fa8a00c25a0 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa89f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c13e0 con 0x7fa8a0151980 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa89f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01522f0 con 0x7fa8a00c1eb0 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa895400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa895400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36482/0 (socket says 172.21.15.67:36482) 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.438+0000 7fa895400640 1 -- 172.21.15.67:0/1628884124 learned_addr learned my addr 172.21.15.67:0/1628884124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa89de00640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a00c1eb0 0x7fa8a0144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa89e800640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a0151980 0x7fa8a0152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa895400640 1 -- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a00c1eb0 msgr2=0x7fa8a0144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa895400640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a00c1eb0 0x7fa8a0144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa895400640 1 -- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a0151980 msgr2=0x7fa8a0152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa895400640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a0151980 0x7fa8a0152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa895400640 1 -- 172.21.15.67:0/1628884124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0156340 con 0x7fa8a00c25a0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa89de00640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a00c1eb0 0x7fa8a0144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa89e800640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a0151980 0x7fa8a0152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa895400640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c4fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c470 tx=0x7fa88800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888017070 con 0x7fa8a00c25a0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888005ce0 con 0x7fa8a00c25a0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880072b0 con 0x7fa8a00c25a0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.439+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a0144670 con 0x7fa8a00c25a0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.440+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a01eaf60 con 0x7fa8a00c25a0 2024-02-26T10:40:18.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.441+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a0074a30 con 0x7fa8a00c25a0 2024-02-26T10:40:18.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.442+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa88801a070 con 0x7fa8a00c25a0 2024-02-26T10:40:18.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.443+0000 7fa896a00640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa87c06b4d0 0x7fa87c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.443+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa8880898d0 con 0x7fa8a00c25a0 2024-02-26T10:40:18.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.443+0000 7fa89de00640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa87c06b4d0 0x7fa87c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.445+0000 7fa89de00640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa87c06b4d0 0x7fa87c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa890036000 tx=0x7fa890030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:18.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.447+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8880583e0 con 0x7fa8a00c25a0 2024-02-26T10:40:18.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.645+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8a006e590 con 0x7fa8a00c25a0 2024-02-26T10:40:18.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.646+0000 7fa896a00640 1 -- 172.21.15.67:0/1628884124 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa888087e20 con 0x7fa8a00c25a0 2024-02-26T10:40:18.646 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:18.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.648+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa87c06b4d0 msgr2=0x7fa87c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.648+0000 7fa89f200640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa87c06b4d0 0x7fa87c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa890036000 tx=0x7fa890030000 comp rx=0 tx=0).stop 2024-02-26T10:40:18.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 msgr2=0x7fa8a00c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c4fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c470 tx=0x7fa88800c930 comp rx=0 tx=0).stop 2024-02-26T10:40:18.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 shutdown_connections 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa87c06b4d0 0x7fa87c06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a0151980 0x7fa8a0152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8a00c25a0 0x7fa8a00c4fc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 --2- 172.21.15.67:0/1628884124 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8a00c1eb0 0x7fa8a0144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 >> 172.21.15.67:0/1628884124 conn(0x7fa8a00c6380 msgr2=0x7fa8a0070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 shutdown_connections 2024-02-26T10:40:18.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.649+0000 7fa89f200640 1 -- 172.21.15.67:0/1628884124 wait complete. 2024-02-26T10:40:18.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:40:18.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.893+0000 7ff599200640 1 Processor -- start 2024-02-26T10:40:18.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.893+0000 7ff599200640 1 -- start start 2024-02-26T10:40:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff599200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5940a9430 0x7ff5940ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff599200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff59406e2d0 0x7ff5940c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff599200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff594150780 con 0x7ff5941503a0 2024-02-26T10:40:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff599200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940a98c0 con 0x7ff59406e2d0 2024-02-26T10:40:18.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff599200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940a9ba0 con 0x7ff5940a9430 2024-02-26T10:40:18.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff593e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff59406e2d0 0x7ff5940c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff593e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff59406e2d0 0x7ff5940c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36486/0 (socket says 172.21.15.67:36486) 2024-02-26T10:40:18.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.895+0000 7ff593e00640 1 -- 172.21.15.67:0/869333418 learned_addr learned my addr 172.21.15.67:0/869333418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:18.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.896+0000 7ff592a00640 1 --2- 172.21.15.67:0/869333418 >> v2:172.21.15.82:3300/0 conn(0x7ff5940a9430 0x7ff5940ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.896+0000 7ff593e00640 1 -- 172.21.15.67:0/869333418 >> v2:172.21.15.82:3300/0 conn(0x7ff5940a9430 msgr2=0x7ff5940ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:18.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.896+0000 7ff593e00640 1 --2- 172.21.15.67:0/869333418 >> v2:172.21.15.82:3300/0 conn(0x7ff5940a9430 0x7ff5940ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.896+0000 7ff593e00640 1 -- 172.21.15.67:0/869333418 >> v1:172.21.15.82:6789/0 conn(0x7ff5941503a0 legacy=0x7ff5941535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.896+0000 7ff593e00640 1 -- 172.21.15.67:0/869333418 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940a9e80 con 0x7ff59406e2d0 2024-02-26T10:40:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.897+0000 7ff593e00640 1 --2- 172.21.15.67:0/869333418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff59406e2d0 0x7ff5940c2e10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff59406f1c0 tx=0x7ff58c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc5137f93f2939a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.897+0000 7ff592000640 1 -- 172.21.15.67:0/869333418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c02cb10 con 0x7ff59406e2d0 2024-02-26T10:40:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.897+0000 7ff592000640 1 -- 172.21.15.67:0/869333418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c02ccb0 con 0x7ff59406e2d0 2024-02-26T10:40:18.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.897+0000 7ff592000640 1 -- 172.21.15.67:0/869333418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c02ce60 con 0x7ff59406e2d0 2024-02-26T10:40:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 -- 172.21.15.67:0/869333418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff59406e2d0 msgr2=0x7ff5940c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 --2- 172.21.15.67:0/869333418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff59406e2d0 0x7ff5940c2e10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff59406f1c0 tx=0x7ff58c02bf70 comp rx=0 tx=0).stop 2024-02-26T10:40:18.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 -- 172.21.15.67:0/869333418 shutdown_connections 2024-02-26T10:40:18.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 --2- 172.21.15.67:0/869333418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff59406e2d0 0x7ff5940c2e10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 --2- 172.21.15.67:0/869333418 >> v2:172.21.15.82:3300/0 conn(0x7ff5940a9430 0x7ff5940ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 -- 172.21.15.67:0/869333418 >> 172.21.15.67:0/869333418 conn(0x7ff5940c6380 msgr2=0x7ff5940c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:18.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 -- 172.21.15.67:0/869333418 shutdown_connections 2024-02-26T10:40:18.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.898+0000 7ff599200640 1 -- 172.21.15.67:0/869333418 wait complete. 2024-02-26T10:40:18.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.900+0000 7ff599200640 1 Processor -- start 2024-02-26T10:40:18.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.900+0000 7ff599200640 1 -- start start 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff599200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff59406e2d0 0x7ff5941edd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff599200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 0x7ff59414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff599200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5940c4f90 0x7ff59414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff599200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940c0f80 con 0x7ff5941503a0 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff599200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59406ddb0 con 0x7ff5940c4f90 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff599200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59406e720 con 0x7ff59406e2d0 2024-02-26T10:40:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.901+0000 7ff592a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 0x7ff59414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 0x7ff59414dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36502/0 (socket says 172.21.15.67:36502) 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 -- 172.21.15.67:0/1245390862 learned_addr learned my addr 172.21.15.67:0/1245390862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff593400640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff59406e2d0 0x7ff5941edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 -- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff59406e2d0 msgr2=0x7ff5941edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff59406e2d0 0x7ff5941edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff593e00640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5940c4f90 0x7ff59414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 -- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5940c4f90 msgr2=0x7ff59414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5940c4f90 0x7ff59414e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 -- 172.21.15.67:0/1245390862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5941ee9e0 con 0x7ff5941503a0 2024-02-26T10:40:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.902+0000 7ff592a00640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 0x7ff59414dbc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff58000d800 tx=0x7ff58000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.903+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000b6c0 con 0x7ff5941503a0 2024-02-26T10:40:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.903+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580019070 con 0x7ff5941503a0 2024-02-26T10:40:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.903+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000bac0 con 0x7ff5941503a0 2024-02-26T10:40:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.903+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5940c4ab0 con 0x7ff5941503a0 2024-02-26T10:40:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.903+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940729e0 con 0x7ff5941503a0 2024-02-26T10:40:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.904+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5940678e0 con 0x7ff5941503a0 2024-02-26T10:40:18.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.905+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff58000bc60 con 0x7ff5941503a0 2024-02-26T10:40:18.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.906+0000 7ff58b400640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff57406b4d0 0x7ff57406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:18.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.906+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff58008aa50 con 0x7ff5941503a0 2024-02-26T10:40:18.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.907+0000 7ff593400640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff57406b4d0 0x7ff57406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:18.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.908+0000 7ff593400640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff57406b4d0 0x7ff57406d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5941dee90 tx=0x7ff584009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:18.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:18.911+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff580059560 con 0x7ff5941503a0 2024-02-26T10:40:19.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.115+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff594074a30 con 0x7ff5941503a0 2024-02-26T10:40:19.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.116+0000 7ff58b400640 1 -- 172.21.15.67:0/1245390862 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff580058e60 con 0x7ff5941503a0 2024-02-26T10:40:19.116 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.118+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff57406b4d0 msgr2=0x7ff57406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.118+0000 7ff599200640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff57406b4d0 0x7ff57406d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5941dee90 tx=0x7ff584009040 comp rx=0 tx=0).stop 2024-02-26T10:40:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 msgr2=0x7ff59414dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 0x7ff59414dbc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff58000d800 tx=0x7ff58000dcc0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 shutdown_connections 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff57406b4d0 0x7ff57406d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5940c4f90 0x7ff59414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5941503a0 0x7ff59414dbc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 --2- 172.21.15.67:0/1245390862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff59406e2d0 0x7ff5941edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 >> 172.21.15.67:0/1245390862 conn(0x7ff5940c6380 msgr2=0x7ff5940c24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 shutdown_connections 2024-02-26T10:40:19.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.119+0000 7ff599200640 1 -- 172.21.15.67:0/1245390862 wait complete. 2024-02-26T10:40:19.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:40:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.365+0000 7fa715800640 1 Processor -- start 2024-02-26T10:40:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.365+0000 7fa715800640 1 -- start start 2024-02-26T10:40:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.366+0000 7fa715800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa7101591b0 0x7fa710158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.366+0000 7fa715800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7100c0ef0 0x7fa710070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.366+0000 7fa715800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa710072ce0 con 0x7fa710071410 2024-02-26T10:40:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.366+0000 7fa715800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa710073010 con 0x7fa7100c0ef0 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.366+0000 7fa715800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71014f490 con 0x7fa7101591b0 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7100c0ef0 0x7fa710070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7100c0ef0 0x7fa710070ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36514/0 (socket says 172.21.15.67:36514) 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 -- 172.21.15.67:0/3622265542 learned_addr learned my addr 172.21.15.67:0/3622265542 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 -- 172.21.15.67:0/3622265542 >> v2:172.21.15.82:3300/0 conn(0x7fa7101591b0 msgr2=0x7fa710158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70ea00640 1 --2- 172.21.15.67:0/3622265542 >> v2:172.21.15.82:3300/0 conn(0x7fa7101591b0 0x7fa710158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 --2- 172.21.15.67:0/3622265542 >> v2:172.21.15.82:3300/0 conn(0x7fa7101591b0 0x7fa710158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 -- 172.21.15.67:0/3622265542 >> v1:172.21.15.82:6789/0 conn(0x7fa710071410 legacy=0x7fa71006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70e000640 1 -- 172.21.15.67:0/3622265542 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa710155bf0 con 0x7fa7100c0ef0 2024-02-26T10:40:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.367+0000 7fa70ea00640 1 --2- 172.21.15.67:0/3622265542 >> v2:172.21.15.82:3300/0 conn(0x7fa7101591b0 0x7fa710158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.369+0000 7fa70e000640 1 --2- 172.21.15.67:0/3622265542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa710070ed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa7100c3bf0 tx=0x7fa6f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c14a64a0f124a9f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.369+0000 7fa70d600640 1 -- 172.21.15.67:0/3622265542 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f802e070 con 0x7fa7100c0ef0 2024-02-26T10:40:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.369+0000 7fa70d600640 1 -- 172.21.15.67:0/3622265542 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f8037070 con 0x7fa7100c0ef0 2024-02-26T10:40:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.369+0000 7fa70d600640 1 -- 172.21.15.67:0/3622265542 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8031a00 con 0x7fa7100c0ef0 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 -- 172.21.15.67:0/3622265542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 msgr2=0x7fa710070ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 --2- 172.21.15.67:0/3622265542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa710070ed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa7100c3bf0 tx=0x7fa6f802b880 comp rx=0 tx=0).stop 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 -- 172.21.15.67:0/3622265542 shutdown_connections 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 --2- 172.21.15.67:0/3622265542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa710070ed0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 --2- 172.21.15.67:0/3622265542 >> v2:172.21.15.82:3300/0 conn(0x7fa7101591b0 0x7fa710158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 -- 172.21.15.67:0/3622265542 >> 172.21.15.67:0/3622265542 conn(0x7fa7100c6da0 msgr2=0x7fa7100c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 -- 172.21.15.67:0/3622265542 shutdown_connections 2024-02-26T10:40:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.370+0000 7fa715800640 1 -- 172.21.15.67:0/3622265542 wait complete. 2024-02-26T10:40:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.372+0000 7fa715800640 1 Processor -- start 2024-02-26T10:40:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.372+0000 7fa715800640 1 -- start start 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa715800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa710071410 0x7fa7100aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa715800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa7101eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa715800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7101591b0 0x7fa71014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa715800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006e6e0 con 0x7fa7100c0ef0 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa715800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006e560 con 0x7fa7101591b0 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa715800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100a8ba0 con 0x7fa710071410 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa7101eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa7101eacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51864/0 (socket says 172.21.15.67:51864) 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70e000640 1 -- 172.21.15.67:0/4067407231 learned_addr learned my addr 172.21.15.67:0/4067407231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70f400640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7101591b0 0x7fa71014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70ea00640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa710071410 0x7fa7100aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70e000640 1 -- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa710071410 msgr2=0x7fa7100aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.373+0000 7fa70e000640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa710071410 0x7fa7100aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa70e000640 1 -- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7101591b0 msgr2=0x7fa71014e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa70e000640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7101591b0 0x7fa71014e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa70e000640 1 -- 172.21.15.67:0/4067407231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7101defd0 con 0x7fa7100c0ef0 2024-02-26T10:40:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa70e000640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa7101eacb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8000c00 tx=0x7fa6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8032880 con 0x7fa7100c0ef0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f8032a20 con 0x7fa7100c0ef0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8032e10 con 0x7fa7100c0ef0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7101eef10 con 0x7fa7100c0ef0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.374+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101df390 con 0x7fa7100c0ef0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.375+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa710074a30 con 0x7fa7100c0ef0 2024-02-26T10:40:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.376+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6f804c060 con 0x7fa7100c0ef0 2024-02-26T10:40:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.377+0000 7fa706a00640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6ec06b4d0 0x7fa6ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.377+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa6f80ac120 con 0x7fa7100c0ef0 2024-02-26T10:40:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.378+0000 7fa70ea00640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6ec06b4d0 0x7fa6ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.379+0000 7fa70ea00640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6ec06b4d0 0x7fa6ec06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa700007510 tx=0x7fa7000074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:19.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.382+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6f8040070 con 0x7fa7100c0ef0 2024-02-26T10:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.577+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa71006eb10 con 0x7fa7100c0ef0 2024-02-26T10:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.578+0000 7fa706a00640 1 -- 172.21.15.67:0/4067407231 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v38) v1 ==== 105+0+11 (secure 0 0 0) 0x7fa6f807adf0 con 0x7fa7100c0ef0 2024-02-26T10:40:19.578 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6ec06b4d0 msgr2=0x7fa6ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6ec06b4d0 0x7fa6ec06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa700007510 tx=0x7fa7000074a0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 msgr2=0x7fa7101eacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa7101eacb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8000c00 tx=0x7fa6f80046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 shutdown_connections 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6ec06b4d0 0x7fa6ec06d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7101591b0 0x7fa71014e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7100c0ef0 0x7fa7101eacb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 --2- 172.21.15.67:0/4067407231 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa710071410 0x7fa7100aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 >> 172.21.15.67:0/4067407231 conn(0x7fa7100c6da0 msgr2=0x7fa7100c01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 shutdown_connections 2024-02-26T10:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.580+0000 7fa715800640 1 -- 172.21.15.67:0/4067407231 wait complete. 2024-02-26T10:40:19.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:40:19.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.800+0000 7f4305800640 1 Processor -- start 2024-02-26T10:40:19.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.800+0000 7f4305800640 1 -- start start 2024-02-26T10:40:19.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.801+0000 7f4305800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 0x7f43001562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.801+0000 7f4305800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4300155880 0x7f430006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.801+0000 7f4305800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430006c110 con 0x7f430006ce30 2024-02-26T10:40:19.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.801+0000 7f4305800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43000c72c0 con 0x7f4300155880 2024-02-26T10:40:19.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.801+0000 7f4305800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000c75a0 con 0x7f43000c5290 2024-02-26T10:40:19.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.802+0000 7f42ffe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 0x7f43001562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.802+0000 7f42f7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4300155880 0x7f430006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.802+0000 7f42ffe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 0x7f43001562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:34674/0 (socket says 172.21.15.67:34674) 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.802+0000 7f42ffe00640 1 -- 172.21.15.67:0/2235496906 learned_addr learned my addr 172.21.15.67:0/2235496906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.802+0000 7f4304e00640 1 --1- 172.21.15.67:0/2235496906 >> v1:172.21.15.82:6789/0 conn(0x7f430006ce30 0x7f43000af8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:54060/0 (socket says 172.21.15.67:54060) 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.803+0000 7f42f7e00640 1 -- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 msgr2=0x7f43001562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.803+0000 7f42f7e00640 1 --2- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 0x7f43001562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.803+0000 7f42f7e00640 1 -- 172.21.15.67:0/2235496906 >> v1:172.21.15.82:6789/0 conn(0x7f430006ce30 legacy=0x7f43000af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.803+0000 7f42f7e00640 1 -- 172.21.15.67:0/2235496906 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000c7880 con 0x7f4300155880 2024-02-26T10:40:19.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.803+0000 7f42ffe00640 1 --2- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 0x7f43001562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:40:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.805+0000 7f42f7e00640 1 --2- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4300155880 0x7f430006c8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f42e80097a0 tx=0x7f42e802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d9be3bd549b33e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.805+0000 7f42ff400640 1 -- 172.21.15.67:0/2235496906 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8031780 con 0x7f4300155880 2024-02-26T10:40:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.805+0000 7f42ff400640 1 -- 172.21.15.67:0/2235496906 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8037070 con 0x7f4300155880 2024-02-26T10:40:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.805+0000 7f42ff400640 1 -- 172.21.15.67:0/2235496906 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8031b40 con 0x7f4300155880 2024-02-26T10:40:19.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 -- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4300155880 msgr2=0x7f430006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 --2- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4300155880 0x7f430006c8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f42e80097a0 tx=0x7f42e802b820 comp rx=0 tx=0).stop 2024-02-26T10:40:19.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 -- 172.21.15.67:0/2235496906 shutdown_connections 2024-02-26T10:40:19.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 --2- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4300155880 0x7f430006c8f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 --2- 172.21.15.67:0/2235496906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43000c5290 0x7f43001562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 -- 172.21.15.67:0/2235496906 >> 172.21.15.67:0/2235496906 conn(0x7f43000ac230 msgr2=0x7f43000ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:19.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 -- 172.21.15.67:0/2235496906 shutdown_connections 2024-02-26T10:40:19.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.806+0000 7f4305800640 1 -- 172.21.15.67:0/2235496906 wait complete. 2024-02-26T10:40:19.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.808+0000 7f4305800640 1 Processor -- start 2024-02-26T10:40:19.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.808+0000 7f4305800640 1 -- start start 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4305800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43000c5290 0x7f43000ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4305800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4300155880 0x7f430014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4305800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 0x7f43000c1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4305800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43000affb0 con 0x7f4300154ff0 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4305800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300155cc0 con 0x7f43000c5290 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4305800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000b0130 con 0x7f4300155880 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4304e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 0x7f43000c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4304e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 0x7f43000c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51880/0 (socket says 172.21.15.67:51880) 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f4304e00640 1 -- 172.21.15.67:0/2937922318 learned_addr learned my addr 172.21.15.67:0/2937922318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f42f7e00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4300155880 0x7f430014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.809+0000 7f42ffe00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43000c5290 0x7f43000ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4304e00640 1 -- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4300155880 msgr2=0x7f430014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4304e00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4300155880 0x7f430014e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4304e00640 1 -- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43000c5290 msgr2=0x7f43000ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4304e00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43000c5290 0x7f43000ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4304e00640 1 -- 172.21.15.67:0/2937922318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43001edc60 con 0x7f4300154ff0 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f42ffe00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43000c5290 0x7f43000ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4304e00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 0x7f43000c1770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42f800c6c0 tx=0x7f42f800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f8018070 con 0x7f4300154ff0 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f8004d60 con 0x7f4300154ff0 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43001edf40 con 0x7f4300154ff0 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.810+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800a840 con 0x7f4300154ff0 2024-02-26T10:40:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.811+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43001ee500 con 0x7f4300154ff0 2024-02-26T10:40:19.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.812+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4300074a30 con 0x7f4300154ff0 2024-02-26T10:40:19.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.813+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42f801b070 con 0x7f4300154ff0 2024-02-26T10:40:19.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.813+0000 7f42fcc00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42dc06b4d0 0x7f42dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:19.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.813+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f42f8089ab0 con 0x7f4300154ff0 2024-02-26T10:40:19.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.814+0000 7f42ffe00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42dc06b4d0 0x7f42dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.815+0000 7f42ffe00640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42dc06b4d0 0x7f42dc06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009b80 tx=0x7f42f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:19.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:19.818+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42f8057f70 con 0x7f4300154ff0 2024-02-26T10:40:20.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.019+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f430006d950 con 0x7f4300154ff0 2024-02-26T10:40:20.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.020+0000 7f42fcc00640 1 -- 172.21.15.67:0/2937922318 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v38) v1 ==== 101+0+11 (secure 0 0 0) 0x7f42f805c010 con 0x7f4300154ff0 2024-02-26T10:40:20.021 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42dc06b4d0 msgr2=0x7f42dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42dc06b4d0 0x7f42dc06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009b80 tx=0x7f42f0008040 comp rx=0 tx=0).stop 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 msgr2=0x7f43000c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 0x7f43000c1770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42f800c6c0 tx=0x7f42f800cb80 comp rx=0 tx=0).stop 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 shutdown_connections 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4300154ff0 0x7f43000c1770 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4300155880 0x7f430014e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f42dc06b4d0 0x7f42dc06d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 --2- 172.21.15.67:0/2937922318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43000c5290 0x7f43000ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 >> 172.21.15.67:0/2937922318 conn(0x7f43000ac230 msgr2=0x7f43000aa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:20.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 shutdown_connections 2024-02-26T10:40:20.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.023+0000 7f4305800640 1 -- 172.21.15.67:0/2937922318 wait complete. 2024-02-26T10:40:20.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.246+0000 7f98c2200640 1 Processor -- start 2024-02-26T10:40:20.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.246+0000 7f98c2200640 1 -- start start 2024-02-26T10:40:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98c2200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98bc06e410 0x7f98bc0aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98c2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f98bc156230 0x7f98bc0ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98c2200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98bc152e50 con 0x7f98bc150300 2024-02-26T10:40:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98c2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98bc1506e0 con 0x7f98bc156230 2024-02-26T10:40:20.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98c2200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98bc153a60 con 0x7f98bc06e410 2024-02-26T10:40:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98bb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f98bc150300 0x7f98bc0c2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55464/0 (socket says 172.21.15.67:55464) 2024-02-26T10:40:20.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98bb400640 1 -- 172.21.15.67:0/1955087817 learned_addr learned my addr 172.21.15.67:0/1955087817 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.247+0000 7f98bbe00640 1 --2- 172.21.15.67:0/1955087817 >> v2:172.21.15.67:3300/0 conn(0x7f98bc156230 0x7f98bc0ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98baa00640 1 --2- 172.21.15.67:0/1955087817 >> v2:172.21.15.82:3300/0 conn(0x7f98bc06e410 0x7f98bc0aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98ba000640 1 -- 172.21.15.67:0/1955087817 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466949905 0 0) 0x7f98bc152e50 con 0x7f98bc150300 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98bbe00640 1 -- 172.21.15.67:0/1955087817 >> v2:172.21.15.82:3300/0 conn(0x7f98bc06e410 msgr2=0x7f98bc0aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98bbe00640 1 --2- 172.21.15.67:0/1955087817 >> v2:172.21.15.82:3300/0 conn(0x7f98bc06e410 0x7f98bc0aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98bbe00640 1 -- 172.21.15.67:0/1955087817 >> v1:172.21.15.67:6789/0 conn(0x7f98bc150300 legacy=0x7f98bc0c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98bbe00640 1 -- 172.21.15.67:0/1955087817 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc153130 con 0x7f98bc156230 2024-02-26T10:40:20.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.248+0000 7f98baa00640 1 --2- 172.21.15.67:0/1955087817 >> v2:172.21.15.82:3300/0 conn(0x7f98bc06e410 0x7f98bc0aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:20.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.250+0000 7f98bbe00640 1 --2- 172.21.15.67:0/1955087817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc156230 0x7f98bc0ab0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0c0ba0 tx=0x7f98ac02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b455f98c213a44 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.250+0000 7f98ba000640 1 -- 172.21.15.67:0/1955087817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac038070 con 0x7f98bc156230 2024-02-26T10:40:20.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.250+0000 7f98ba000640 1 -- 172.21.15.67:0/1955087817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac031710 con 0x7f98bc156230 2024-02-26T10:40:20.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.250+0000 7f98ba000640 1 -- 172.21.15.67:0/1955087817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac031ae0 con 0x7f98bc156230 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 -- 172.21.15.67:0/1955087817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc156230 msgr2=0x7f98bc0ab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 --2- 172.21.15.67:0/1955087817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc156230 0x7f98bc0ab0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0c0ba0 tx=0x7f98ac02b7b0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 -- 172.21.15.67:0/1955087817 shutdown_connections 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 --2- 172.21.15.67:0/1955087817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc156230 0x7f98bc0ab0e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 --2- 172.21.15.67:0/1955087817 >> v2:172.21.15.82:3300/0 conn(0x7f98bc06e410 0x7f98bc0aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 -- 172.21.15.67:0/1955087817 >> 172.21.15.67:0/1955087817 conn(0x7f98bc0a84a0 msgr2=0x7f98bc0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 -- 172.21.15.67:0/1955087817 shutdown_connections 2024-02-26T10:40:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.251+0000 7f98c2200640 1 -- 172.21.15.67:0/1955087817 wait complete. 2024-02-26T10:40:20.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.252+0000 7f98c2200640 1 Processor -- start 2024-02-26T10:40:20.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 -- start start 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc06e410 0x7f98bc14e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 0x7f98bc14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98bc156230 0x7f98bc14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc06dd80 con 0x7f98bc150300 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc1536f0 con 0x7f98bc06e410 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.253+0000 7f98c2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0c3700 con 0x7f98bc156230 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 0x7f98bc14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 0x7f98bc14cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51894/0 (socket says 172.21.15.67:51894) 2024-02-26T10:40:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 -- 172.21.15.67:0/703979369 learned_addr learned my addr 172.21.15.67:0/703979369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 -- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98bc156230 msgr2=0x7f98bc14d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98bc156230 0x7f98bc14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 -- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc06e410 msgr2=0x7f98bc14e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc06e410 0x7f98bc14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 -- 172.21.15.67:0/703979369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc1ea9a0 con 0x7f98bc150300 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.254+0000 7f98baa00640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 0x7f98bc14cea0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c7f0 tx=0x7f98a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.255+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4015070 con 0x7f98bc150300 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.255+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4007a30 con 0x7f98bc150300 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.255+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4007e00 con 0x7f98bc150300 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.255+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc1eaf00 con 0x7f98bc150300 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.255+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc1eac20 con 0x7f98bc150300 2024-02-26T10:40:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.256+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc074a30 con 0x7f98bc150300 2024-02-26T10:40:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.257+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98a4018070 con 0x7f98bc150300 2024-02-26T10:40:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.258+0000 7f98b3400640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f989806b4d0 0x7f989806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.258+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f98a408a480 con 0x7f98bc150300 2024-02-26T10:40:20.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.259+0000 7f98bb400640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f989806b4d0 0x7f989806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.260+0000 7f98bb400640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f989806b4d0 0x7f989806d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f98b4001560 tx=0x7f98b4007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.263+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98bc1ea9a0 con 0x7f98bc150300 2024-02-26T10:40:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.526+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98bc06e0d0 con 0x7f98bc150300 2024-02-26T10:40:20.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.527+0000 7f98b3400640 1 -- 172.21.15.67:0/703979369 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+6171 (secure 0 0 0) 0x7f98a4058f90 con 0x7f98bc150300 2024-02-26T10:40:20.528 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:20.528 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:20.528 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 22 2024-02-26T10:40:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f989806b4d0 msgr2=0x7f989806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f989806b4d0 0x7f989806d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f98b4001560 tx=0x7f98b4007ca0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 msgr2=0x7f98bc14cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 0x7f98bc14cea0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c7f0 tx=0x7f98a400ccb0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 shutdown_connections 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f98bc156230 0x7f98bc14d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98bc150300 0x7f98bc14cea0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f989806b4d0 0x7f989806d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 --2- 172.21.15.67:0/703979369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc06e410 0x7f98bc14e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 >> 172.21.15.67:0/703979369 conn(0x7f98bc0a84a0 msgr2=0x7f98bc0a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 shutdown_connections 2024-02-26T10:40:20.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.530+0000 7f98c2200640 1 -- 172.21.15.67:0/703979369 wait complete. 2024-02-26T10:40:20.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:20.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.750+0000 7fdb3a400640 1 Processor -- start 2024-02-26T10:40:20.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.750+0000 7fdb3a400640 1 -- start start 2024-02-26T10:40:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.751+0000 7fdb3a400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdb34158d80 0x7fdb340c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.751+0000 7fdb3a400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdb340c54e0 0x7fdb34154d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.751+0000 7fdb3a400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb34155840 con 0x7fdb34152460 2024-02-26T10:40:20.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb3a400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb34158330 con 0x7fdb34158d80 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb3a400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb341588f0 con 0x7fdb340c54e0 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdb34158d80 0x7fdb340c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdb34158d80 0x7fdb340c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51896/0 (socket says 172.21.15.67:51896) 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 -- 172.21.15.67:0/2461262660 learned_addr learned my addr 172.21.15.67:0/2461262660 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 -- 172.21.15.67:0/2461262660 >> v2:172.21.15.82:3300/0 conn(0x7fdb340c54e0 msgr2=0x7fdb34154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 --2- 172.21.15.67:0/2461262660 >> v2:172.21.15.82:3300/0 conn(0x7fdb340c54e0 0x7fdb34154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 -- 172.21.15.67:0/2461262660 >> v1:172.21.15.82:6789/0 conn(0x7fdb34152460 legacy=0x7fdb34156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:20.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.752+0000 7fdb33e00640 1 -- 172.21.15.67:0/2461262660 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34155b20 con 0x7fdb34158d80 2024-02-26T10:40:20.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.753+0000 7fdb33e00640 1 --2- 172.21.15.67:0/2461262660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb34158d80 0x7fdb340c4fa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb3406b530 tx=0x7fdb2002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380c7718148c71ba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb33400640 1 -- 172.21.15.67:0/2461262660 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20038070 con 0x7fdb34158d80 2024-02-26T10:40:20.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb33400640 1 -- 172.21.15.67:0/2461262660 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2002f050 con 0x7fdb34158d80 2024-02-26T10:40:20.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb33400640 1 -- 172.21.15.67:0/2461262660 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20031970 con 0x7fdb34158d80 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 -- 172.21.15.67:0/2461262660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb34158d80 msgr2=0x7fdb340c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 --2- 172.21.15.67:0/2461262660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb34158d80 0x7fdb340c4fa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb3406b530 tx=0x7fdb2002b7f0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 -- 172.21.15.67:0/2461262660 shutdown_connections 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 --2- 172.21.15.67:0/2461262660 >> v2:172.21.15.82:3300/0 conn(0x7fdb340c54e0 0x7fdb34154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 --2- 172.21.15.67:0/2461262660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb34158d80 0x7fdb340c4fa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 -- 172.21.15.67:0/2461262660 >> 172.21.15.67:0/2461262660 conn(0x7fdb340ac1b0 msgr2=0x7fdb340ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.754+0000 7fdb3a400640 1 -- 172.21.15.67:0/2461262660 shutdown_connections 2024-02-26T10:40:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.755+0000 7fdb3a400640 1 -- 172.21.15.67:0/2461262660 wait complete. 2024-02-26T10:40:20.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.756+0000 7fdb3a400640 1 Processor -- start 2024-02-26T10:40:20.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.756+0000 7fdb3a400640 1 -- start start 2024-02-26T10:40:20.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb3a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 0x7fdb34153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb3a400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb34152460 0x7fdb341dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb3a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb34158d80 0x7fdb341e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb3a400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3406d9e0 con 0x7fdb340c54e0 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb3a400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34159200 con 0x7fdb34158d80 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb3a400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3406db60 con 0x7fdb34152460 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb39000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 0x7fdb34153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb39000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 0x7fdb34153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51908/0 (socket says 172.21.15.67:51908) 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.757+0000 7fdb39000640 1 -- 172.21.15.67:0/4128362447 learned_addr learned my addr 172.21.15.67:0/4128362447 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39000640 1 -- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb34152460 msgr2=0x7fdb341dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb33e00640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb34152460 0x7fdb341dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39000640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb34152460 0x7fdb341dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39000640 1 -- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb34158d80 msgr2=0x7fdb341e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39a00640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb34158d80 0x7fdb341e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39000640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb34158d80 0x7fdb341e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39000640 1 -- 172.21.15.67:0/4128362447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb341de520 con 0x7fdb340c54e0 2024-02-26T10:40:20.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39a00640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb34158d80 0x7fdb341e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb39000640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 0x7fdb34153eb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b420 tx=0x7fdb2800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28011830 con 0x7fdb340c54e0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28016070 con 0x7fdb340c54e0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28011c10 con 0x7fdb340c54e0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.758+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb341de800 con 0x7fdb340c54e0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.759+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb341dedc0 con 0x7fdb340c54e0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.760+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3406bfd0 con 0x7fdb340c54e0 2024-02-26T10:40:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.760+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb2801b460 con 0x7fdb340c54e0 2024-02-26T10:40:20.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.761+0000 7fdb30c00640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdb1406b4d0 0x7fdb1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:20.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.761+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fdb28089fe0 con 0x7fdb340c54e0 2024-02-26T10:40:20.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.762+0000 7fdb33e00640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdb1406b4d0 0x7fdb1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:20.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.763+0000 7fdb33e00640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdb1406b4d0 0x7fdb1406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb20035000 tx=0x7fdb20008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:20.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.766+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb3406bfd0 con 0x7fdb340c54e0 2024-02-26T10:40:20.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.962+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb3406acd0 con 0x7fdb340c54e0 2024-02-26T10:40:20.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.963+0000 7fdb30c00640 1 -- 172.21.15.67:0/4128362447 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+20323 (secure 0 0 0) 0x7fdb3406acd0 con 0x7fdb340c54e0 2024-02-26T10:40:20.964 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:20.964 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":38,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:39:57.124601+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:20.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdb1406b4d0 msgr2=0x7fdb1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdb1406b4d0 0x7fdb1406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb20035000 tx=0x7fdb20008030 comp rx=0 tx=0).stop 2024-02-26T10:40:20.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 msgr2=0x7fdb34153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 0x7fdb34153eb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b420 tx=0x7fdb2800b8e0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 shutdown_connections 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdb1406b4d0 0x7fdb1406d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb34158d80 0x7fdb341e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb34152460 0x7fdb341dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 --2- 172.21.15.67:0/4128362447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb340c54e0 0x7fdb34153eb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 >> 172.21.15.67:0/4128362447 conn(0x7fdb340ac1b0 msgr2=0x7fdb340aa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 shutdown_connections 2024-02-26T10:40:20.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:20.966+0000 7fdb3a400640 1 -- 172.21.15.67:0/4128362447 wait complete. 2024-02-26T10:40:20.985 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T10:40:20.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.189+0000 7f21b1800640 1 Processor -- start 2024-02-26T10:40:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.189+0000 7f21b1800640 1 -- start start 2024-02-26T10:40:21.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.190+0000 7f21b1800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 0x7f21ac06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21b1800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21ac0c2310 0x7f21ac1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21b1800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac155f10 con 0x7f21ac152f20 2024-02-26T10:40:21.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21b1800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac153aa0 con 0x7f21ac0c2310 2024-02-26T10:40:21.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21b1800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0c28d0 con 0x7f21ac06ac20 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 0x7f21ac06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 0x7f21ac06b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:34702/0 (socket says 172.21.15.67:34702) 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.191+0000 7f21abe00640 1 -- 172.21.15.67:0/3362020853 learned_addr learned my addr 172.21.15.67:0/3362020853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.192+0000 7f21a3400640 1 --2- 172.21.15.67:0/3362020853 >> v2:172.21.15.82:3300/0 conn(0x7f21ac0c2310 0x7f21ac1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.192+0000 7f21abe00640 1 -- 172.21.15.67:0/3362020853 >> v2:172.21.15.82:3300/0 conn(0x7f21ac0c2310 msgr2=0x7f21ac1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.192+0000 7f21abe00640 1 --2- 172.21.15.67:0/3362020853 >> v2:172.21.15.82:3300/0 conn(0x7f21ac0c2310 0x7f21ac1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.192+0000 7f21abe00640 1 -- 172.21.15.67:0/3362020853 >> v1:172.21.15.82:6789/0 conn(0x7f21ac152f20 legacy=0x7f21ac153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.192+0000 7f21abe00640 1 -- 172.21.15.67:0/3362020853 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac155150 con 0x7f21ac06ac20 2024-02-26T10:40:21.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.194+0000 7f21abe00640 1 --2- 172.21.15.67:0/3362020853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 0x7f21ac06b000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44562045d3a0d07 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:21.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.194+0000 7f21ab400640 1 -- 172.21.15.67:0/3362020853 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940317c0 con 0x7f21ac06ac20 2024-02-26T10:40:21.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.194+0000 7f21ab400640 1 -- 172.21.15.67:0/3362020853 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194037070 con 0x7f21ac06ac20 2024-02-26T10:40:21.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.194+0000 7f21ab400640 1 -- 172.21.15.67:0/3362020853 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031b80 con 0x7f21ac06ac20 2024-02-26T10:40:21.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 -- 172.21.15.67:0/3362020853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 msgr2=0x7f21ac06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 --2- 172.21.15.67:0/3362020853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 0x7f21ac06b000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219402b860 comp rx=0 tx=0).stop 2024-02-26T10:40:21.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 -- 172.21.15.67:0/3362020853 shutdown_connections 2024-02-26T10:40:21.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 --2- 172.21.15.67:0/3362020853 >> v2:172.21.15.82:3300/0 conn(0x7f21ac0c2310 0x7f21ac1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 --2- 172.21.15.67:0/3362020853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac06ac20 0x7f21ac06b000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 -- 172.21.15.67:0/3362020853 >> 172.21.15.67:0/3362020853 conn(0x7f21ac0c63f0 msgr2=0x7f21ac0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 -- 172.21.15.67:0/3362020853 shutdown_connections 2024-02-26T10:40:21.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.195+0000 7f21b1800640 1 -- 172.21.15.67:0/3362020853 wait complete. 2024-02-26T10:40:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.197+0000 7f21b1800640 1 Processor -- start 2024-02-26T10:40:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.197+0000 7f21b1800640 1 -- start start 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21b1800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac06ac20 0x7f21ac142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21b1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 0x7f21ac143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21b1800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac152f20 0x7f21ac0aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21b1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1559f0 con 0x7f21ac0c2310 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21b1800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac155710 con 0x7f21ac06ac20 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21b1800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac148850 con 0x7f21ac152f20 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 0x7f21ac143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 0x7f21ac143210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51918/0 (socket says 172.21.15.67:51918) 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21a3400640 1 -- 172.21.15.67:0/647320771 learned_addr learned my addr 172.21.15.67:0/647320771 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:21.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.198+0000 7f21abe00640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac06ac20 0x7f21ac142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21b0e00640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac152f20 0x7f21ac0aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a3400640 1 -- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac152f20 msgr2=0x7f21ac0aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a3400640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac152f20 0x7f21ac0aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a3400640 1 -- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac06ac20 msgr2=0x7f21ac142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a3400640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac06ac20 0x7f21ac142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a3400640 1 -- 172.21.15.67:0/647320771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0c5f60 con 0x7f21ac0c2310 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21b0e00640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac152f20 0x7f21ac0aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:21.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a3400640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 0x7f21ac143210 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f219c00e870 tx=0x7f219c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c017070 con 0x7f21ac0c2310 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac1e9390 con 0x7f21ac0c2310 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c00c940 con 0x7f21ac0c2310 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c00cd30 con 0x7f21ac0c2310 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.199+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac1de770 con 0x7f21ac0c2310 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.200+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac074a30 con 0x7f21ac0c2310 2024-02-26T10:40:21.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.201+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f219c01a070 con 0x7f21ac0c2310 2024-02-26T10:40:21.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.202+0000 7f21a8c00640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f218806b4d0 0x7f218806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.202+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f219c08a870 con 0x7f21ac0c2310 2024-02-26T10:40:21.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.203+0000 7f21abe00640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f218806b4d0 0x7f218806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.204+0000 7f21abe00640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f218806b4d0 0x7f218806d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:21.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.207+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f219c059300 con 0x7f21ac0c2310 2024-02-26T10:40:21.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.454+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21ac0c20f0 con 0x7f21ac0c2310 2024-02-26T10:40:21.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.455+0000 7f21a8c00640 1 -- 172.21.15.67:0/647320771 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+6171 (secure 0 0 0) 0x7f219c058c00 con 0x7f21ac0c2310 2024-02-26T10:40:21.455 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:21.456 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:21.456 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 22 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f218806b4d0 msgr2=0x7f218806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f218806b4d0 0x7f218806d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403d040 comp rx=0 tx=0).stop 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 msgr2=0x7f21ac143210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 0x7f21ac143210 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f219c00e870 tx=0x7f219c00ed30 comp rx=0 tx=0).stop 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 shutdown_connections 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21ac152f20 0x7f21ac0aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21ac0c2310 0x7f21ac143210 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f218806b4d0 0x7f218806d990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 --2- 172.21.15.67:0/647320771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21ac06ac20 0x7f21ac142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 >> 172.21.15.67:0/647320771 conn(0x7f21ac0c63f0 msgr2=0x7f21ac0c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:21.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 shutdown_connections 2024-02-26T10:40:21.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.459+0000 7f21b1800640 1 -- 172.21.15.67:0/647320771 wait complete. 2024-02-26T10:40:21.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:21.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.699+0000 7f5012e00640 1 Processor -- start 2024-02-26T10:40:21.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.699+0000 7f5012e00640 1 -- start start 2024-02-26T10:40:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5012e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f500c150300 0x7f500c0c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5012e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f500c06e410 0x7f500c0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5012e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500c152e50 con 0x7f500c156230 2024-02-26T10:40:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5012e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c1506e0 con 0x7f500c150300 2024-02-26T10:40:21.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5012e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c153a60 con 0x7f500c06e410 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f500be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f500c150300 0x7f500c0c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5010e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f500c156230 0x7f500c0aaba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55474/0 (socket says 172.21.15.67:55474) 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f5010e00640 1 -- 172.21.15.67:0/2334425490 learned_addr learned my addr 172.21.15.67:0/2334425490 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.701+0000 7f500b400640 1 --2- 172.21.15.67:0/2334425490 >> v2:172.21.15.82:3300/0 conn(0x7f500c06e410 0x7f500c0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500aa00640 1 -- 172.21.15.67:0/2334425490 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388710979 0 0) 0x7f500c152e50 con 0x7f500c156230 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500aa00640 1 -- 172.21.15.67:0/2334425490 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ff0003520 con 0x7f500c156230 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500be00640 1 -- 172.21.15.67:0/2334425490 >> v2:172.21.15.82:3300/0 conn(0x7f500c06e410 msgr2=0x7f500c0c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500be00640 1 --2- 172.21.15.67:0/2334425490 >> v2:172.21.15.82:3300/0 conn(0x7f500c06e410 0x7f500c0c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500be00640 1 -- 172.21.15.67:0/2334425490 >> v1:172.21.15.67:6789/0 conn(0x7f500c156230 legacy=0x7f500c0aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500be00640 1 -- 172.21.15.67:0/2334425490 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c153130 con 0x7f500c150300 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.702+0000 7f500b400640 1 --2- 172.21.15.67:0/2334425490 >> v2:172.21.15.82:3300/0 conn(0x7f500c06e410 0x7f500c0c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.703+0000 7f500be00640 1 --2- 172.21.15.67:0/2334425490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c150300 0x7f500c0c2ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f500c0c0ba0 tx=0x7f4ffc02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf93036e3b57add server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:21.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.703+0000 7f500aa00640 1 -- 172.21.15.67:0/2334425490 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc038070 con 0x7f500c150300 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.703+0000 7f500aa00640 1 -- 172.21.15.67:0/2334425490 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc031520 con 0x7f500c150300 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.703+0000 7f500aa00640 1 -- 172.21.15.67:0/2334425490 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0318d0 con 0x7f500c150300 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 -- 172.21.15.67:0/2334425490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c150300 msgr2=0x7f500c0c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 --2- 172.21.15.67:0/2334425490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c150300 0x7f500c0c2ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f500c0c0ba0 tx=0x7f4ffc02b7c0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 -- 172.21.15.67:0/2334425490 shutdown_connections 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 --2- 172.21.15.67:0/2334425490 >> v2:172.21.15.82:3300/0 conn(0x7f500c06e410 0x7f500c0c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 --2- 172.21.15.67:0/2334425490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c150300 0x7f500c0c2ff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 -- 172.21.15.67:0/2334425490 >> 172.21.15.67:0/2334425490 conn(0x7f500c0a84a0 msgr2=0x7f500c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 -- 172.21.15.67:0/2334425490 shutdown_connections 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.704+0000 7f5012e00640 1 -- 172.21.15.67:0/2334425490 wait complete. 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.705+0000 7f5012e00640 1 Processor -- start 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.706+0000 7f5012e00640 1 -- start start 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.706+0000 7f5012e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500c06e410 0x7f500c14d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.706+0000 7f5012e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f500c150300 0x7f500c1e8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.706+0000 7f5012e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 0x7f500c1eb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5012e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c06dd80 con 0x7f500c156230 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5012e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c1536f0 con 0x7f500c06e410 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5012e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0ab2b0 con 0x7f500c150300 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 0x7f500c1eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 0x7f500c1eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51946/0 (socket says 172.21.15.67:51946) 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 -- 172.21.15.67:0/477838879 learned_addr learned my addr 172.21.15.67:0/477838879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f500b400640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f500c150300 0x7f500c1e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 -- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f500c150300 msgr2=0x7f500c1e8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f500be00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500c06e410 0x7f500c14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f500c150300 0x7f500c1e8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 -- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500c06e410 msgr2=0x7f500c14d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500c06e410 0x7f500c14d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.707+0000 7f5010e00640 1 -- 172.21.15.67:0/477838879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c1e8190 con 0x7f500c156230 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.708+0000 7f5010e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 0x7f500c1eb780 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f500400caa0 tx=0x7f500400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.708+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004017070 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.708+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500400ac00 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.708+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004012400 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.708+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c152490 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.708+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c152a50 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.709+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c074a30 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.710+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f500401a070 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.711+0000 7f5003e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fec06b4d0 0x7f4fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.711+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f500408a3a0 con 0x7f500c156230 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.712+0000 7f500be00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fec06b4d0 0x7f4fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.713+0000 7f500be00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fec06b4d0 0x7f4fec06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc035000 tx=0x7f4ffc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.716+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5004058e80 con 0x7f500c156230 2024-02-26T10:40:21.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.953+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f500c06e0d0 con 0x7f500c156230 2024-02-26T10:40:21.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.955+0000 7f5003e00640 1 -- 172.21.15.67:0/477838879 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+6171 (secure 0 0 0) 0x7f5004058780 con 0x7f500c156230 2024-02-26T10:40:21.955 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:21.955 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:39.463009+0000","modified":"2024-02-26T10:39:42.519117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4753":{"gid":4753,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/155874470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":155874470},{"type":"v1","addr":"172.21.15.82:6837","nonce":155874470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:21.955 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 22 2024-02-26T10:40:21.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.957+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fec06b4d0 msgr2=0x7f4fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.957+0000 7f5012e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fec06b4d0 0x7f4fec06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc035000 tx=0x7f4ffc030000 comp rx=0 tx=0).stop 2024-02-26T10:40:21.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.957+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 msgr2=0x7f500c1eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.957+0000 7f5012e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 0x7f500c1eb780 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f500400caa0 tx=0x7f500400cf60 comp rx=0 tx=0).stop 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 shutdown_connections 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f500c156230 0x7f500c1eb780 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f500c150300 0x7f500c1e8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4fec06b4d0 0x7f4fec06d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 --2- 172.21.15.67:0/477838879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500c06e410 0x7f500c14d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 >> 172.21.15.67:0/477838879 conn(0x7f500c0a84a0 msgr2=0x7f500c0c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 shutdown_connections 2024-02-26T10:40:21.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:21.958+0000 7f5012e00640 1 -- 172.21.15.67:0/477838879 wait complete. 2024-02-26T10:40:21.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:22.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.196+0000 7f507f200640 1 Processor -- start 2024-02-26T10:40:22.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.196+0000 7f507f200640 1 -- start start 2024-02-26T10:40:22.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f50800a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f508006b000 0x7f508006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508006b460 con 0x7f5080153610 2024-02-26T10:40:22.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50800a9a80 con 0x7f508006b000 2024-02-26T10:40:22.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50800a9d60 con 0x7f508006ac20 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5080153610 0x7f50800abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55486/0 (socket says 172.21.15.67:55486) 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507e800640 1 -- 172.21.15.67:0/96650542 learned_addr learned my addr 172.21.15.67:0/96650542 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f5077e00640 1 --2- 172.21.15.67:0/96650542 >> v2:172.21.15.67:3300/0 conn(0x7f508006b000 0x7f508006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.198+0000 7f507de00640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f50800a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.199+0000 7f5077e00640 1 -- 172.21.15.67:0/96650542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 msgr2=0x7f50800a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.199+0000 7f5077e00640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f50800a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.199+0000 7f507d400640 1 -- 172.21.15.67:0/96650542 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889326204 0 0) 0x7f508006b460 con 0x7f5080153610 2024-02-26T10:40:22.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.199+0000 7f5077e00640 1 -- 172.21.15.67:0/96650542 >> v1:172.21.15.67:6789/0 conn(0x7f5080153610 legacy=0x7f50800abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.199+0000 7f5077e00640 1 -- 172.21.15.67:0/96650542 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800ab750 con 0x7f508006b000 2024-02-26T10:40:22.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.199+0000 7f507de00640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f50800a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.200+0000 7f5077e00640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508006b000 0x7f508006db00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f50680097a0 tx=0x7f506802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326d0d94e179e318 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.200+0000 7f507d400640 1 -- 172.21.15.67:0/96650542 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068035020 con 0x7f508006b000 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.200+0000 7f507d400640 1 -- 172.21.15.67:0/96650542 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5068031810 con 0x7f508006b000 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.200+0000 7f507d400640 1 -- 172.21.15.67:0/96650542 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068031be0 con 0x7f508006b000 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 -- 172.21.15.67:0/96650542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508006b000 msgr2=0x7f508006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508006b000 0x7f508006db00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f50680097a0 tx=0x7f506802b8b0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 -- 172.21.15.67:0/96650542 shutdown_connections 2024-02-26T10:40:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f508006b000 0x7f508006db00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 --2- 172.21.15.67:0/96650542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f50800a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 -- 172.21.15.67:0/96650542 >> 172.21.15.67:0/96650542 conn(0x7f50800c63f0 msgr2=0x7f50800c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 -- 172.21.15.67:0/96650542 shutdown_connections 2024-02-26T10:40:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.201+0000 7f507f200640 1 -- 172.21.15.67:0/96650542 wait complete. 2024-02-26T10:40:22.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.203+0000 7f507f200640 1 Processor -- start 2024-02-26T10:40:22.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.203+0000 7f507f200640 1 -- start start 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f5080155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508006b000 0x7f50801ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 0x7f50801513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080150410 con 0x7f5080153610 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080153aa0 con 0x7f508006b000 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5080150590 con 0x7f508006ac20 2024-02-26T10:40:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 0x7f50801513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 0x7f50801513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51962/0 (socket says 172.21.15.67:51962) 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 -- 172.21.15.67:0/226270489 learned_addr learned my addr 172.21.15.67:0/226270489 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507de00640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f5080155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f5077e00640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508006b000 0x7f50801ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 -- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 msgr2=0x7f5080155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f5080155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 -- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508006b000 msgr2=0x7f50801ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508006b000 0x7f50801ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.204+0000 7f507e800640 1 -- 172.21.15.67:0/226270489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50801eedb0 con 0x7f5080153610 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f5077e00640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508006b000 0x7f50801ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f507e800640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 0x7f50801513e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f507800c700 tx=0x7f507800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078018070 con 0x7f5080153610 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507800ce10 con 0x7f5080153610 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507800a800 con 0x7f5080153610 2024-02-26T10:40:22.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50801dc290 con 0x7f5080153610 2024-02-26T10:40:22.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.205+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50801eaae0 con 0x7f5080153610 2024-02-26T10:40:22.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.206+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50801dc7b0 con 0x7f5080153610 2024-02-26T10:40:22.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.207+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f507801b070 con 0x7f5080153610 2024-02-26T10:40:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.209+0000 7f5076000640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505c06b500 0x7f505c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.209+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f50780899a0 con 0x7f5080153610 2024-02-26T10:40:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.209+0000 7f507de00640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505c06b500 0x7f505c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.210+0000 7f507de00640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505c06b500 0x7f505c06d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f507000d8b0 tx=0x7f5070008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:22.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.214+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50801dc7b0 con 0x7f5080153610 2024-02-26T10:40:22.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.420+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5080074a30 con 0x7f5080153610 2024-02-26T10:40:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.427+0000 7f5076000640 1 -- 172.21.15.67:0/226270489 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20419 (secure 0 0 0) 0x7f5078058520 con 0x7f5080153610 2024-02-26T10:40:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:22.429 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":40,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:22.282998+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:39:37.371486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-02-26T10:39:38.475801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505c06b500 msgr2=0x7f505c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505c06b500 0x7f505c06d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f507000d8b0 tx=0x7f5070008040 comp rx=0 tx=0).stop 2024-02-26T10:40:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 msgr2=0x7f50801513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 0x7f50801513e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f507800c700 tx=0x7f507800cbc0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 shutdown_connections 2024-02-26T10:40:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5080153610 0x7f50801513e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f505c06b500 0x7f505c06d9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508006b000 0x7f50801ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 --2- 172.21.15.67:0/226270489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f508006ac20 0x7f5080155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 >> 172.21.15.67:0/226270489 conn(0x7f50800c63f0 msgr2=0x7f5080150a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 shutdown_connections 2024-02-26T10:40:22.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.432+0000 7f507f200640 1 -- 172.21.15.67:0/226270489 wait complete. 2024-02-26T10:40:22.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T10:40:22.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.665+0000 7fc573200640 1 Processor -- start 2024-02-26T10:40:22.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.665+0000 7fc573200640 1 -- start start 2024-02-26T10:40:22.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.666+0000 7fc573200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc56c14f3d0 0x7fc56c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.666+0000 7fc573200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc56c0c2870 0x7fc56c155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.666+0000 7fc573200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c06ed10 con 0x7fc56c1556a0 2024-02-26T10:40:22.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.666+0000 7fc573200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c06dd80 con 0x7fc56c14f3d0 2024-02-26T10:40:22.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.666+0000 7fc573200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c155c00 con 0x7fc56c0c2870 2024-02-26T10:40:22.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc56c14f3d0 0x7fc56c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc56c14f3d0 0x7fc56c06e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51968/0 (socket says 172.21.15.67:51968) 2024-02-26T10:40:22.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 -- 172.21.15.67:0/3930420583 learned_addr learned my addr 172.21.15.67:0/3930420583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:22.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56b400640 1 --2- 172.21.15.67:0/3930420583 >> v2:172.21.15.82:3300/0 conn(0x7fc56c0c2870 0x7fc56c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 -- 172.21.15.67:0/3930420583 >> v2:172.21.15.82:3300/0 conn(0x7fc56c0c2870 msgr2=0x7fc56c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:22.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 --2- 172.21.15.67:0/3930420583 >> v2:172.21.15.82:3300/0 conn(0x7fc56c0c2870 0x7fc56c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 -- 172.21.15.67:0/3930420583 >> v1:172.21.15.82:6789/0 conn(0x7fc56c1556a0 legacy=0x7fc56c071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:22.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.667+0000 7fc56be00640 1 -- 172.21.15.67:0/3930420583 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c155ee0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.669+0000 7fc56be00640 1 --2- 172.21.15.67:0/3930420583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c06e7d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0c1120 tx=0x7fc55402dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81f3bc5b4a0634a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:22.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.669+0000 7fc56aa00640 1 -- 172.21.15.67:0/3930420583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55403b070 con 0x7fc56c14f3d0 2024-02-26T10:40:22.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.669+0000 7fc56aa00640 1 -- 172.21.15.67:0/3930420583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55402ed70 con 0x7fc56c14f3d0 2024-02-26T10:40:22.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.669+0000 7fc56aa00640 1 -- 172.21.15.67:0/3930420583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554036660 con 0x7fc56c14f3d0 2024-02-26T10:40:22.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 -- 172.21.15.67:0/3930420583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 msgr2=0x7fc56c06e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 --2- 172.21.15.67:0/3930420583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c06e7d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0c1120 tx=0x7fc55402dfc0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 -- 172.21.15.67:0/3930420583 shutdown_connections 2024-02-26T10:40:22.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 --2- 172.21.15.67:0/3930420583 >> v2:172.21.15.82:3300/0 conn(0x7fc56c0c2870 0x7fc56c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 --2- 172.21.15.67:0/3930420583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c06e7d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 -- 172.21.15.67:0/3930420583 >> 172.21.15.67:0/3930420583 conn(0x7fc56c0a84a0 msgr2=0x7fc56c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:22.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 -- 172.21.15.67:0/3930420583 shutdown_connections 2024-02-26T10:40:22.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.670+0000 7fc573200640 1 -- 172.21.15.67:0/3930420583 wait complete. 2024-02-26T10:40:22.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.672+0000 7fc573200640 1 Processor -- start 2024-02-26T10:40:22.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.672+0000 7fc573200640 1 -- start start 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc573200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc56c0c2870 0x7fc56c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc573200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc573200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc56c1556a0 0x7fc56c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc573200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0c1810 con 0x7fc56c14f3d0 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc573200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c06e2b0 con 0x7fc56c1556a0 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc573200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c071e10 con 0x7fc56c0c2870 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc56b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc56b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51982/0 (socket says 172.21.15.67:51982) 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc56b400640 1 -- 172.21.15.67:0/4155086149 learned_addr learned my addr 172.21.15.67:0/4155086149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:22.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.673+0000 7fc571200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc56c1556a0 0x7fc56c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56be00640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc56c0c2870 0x7fc56c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56b400640 1 -- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc56c0c2870 msgr2=0x7fc56c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56b400640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc56c0c2870 0x7fc56c152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56b400640 1 -- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc56c1556a0 msgr2=0x7fc56c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56b400640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc56c1556a0 0x7fc56c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56b400640 1 -- 172.21.15.67:0/4155086149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0a9ce0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56be00640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc56c0c2870 0x7fc56c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc56b400640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c148a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc55c007e40 tx=0x7fc55c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c014620 con 0x7fc56c14f3d0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c019070 con 0x7fc56c14f3d0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c152f90 con 0x7fc56c14f3d0 2024-02-26T10:40:22.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c014a00 con 0x7fc56c14f3d0 2024-02-26T10:40:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.674+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c0a9fc0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.676+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c074a30 con 0x7fc56c14f3d0 2024-02-26T10:40:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.677+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc55c014ba0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.678+0000 7fc563e00640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc54806b4d0 0x7fc54806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.678+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6623+0+0 (secure 0 0 0) 0x7fc55c08adf0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.678+0000 7fc56be00640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc54806b4d0 0x7fc54806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:22.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.679+0000 7fc56be00640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc54806b4d0 0x7fc54806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc554038000 tx=0x7fc554008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:22.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.682+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc55c0598d0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.918+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc56c0c13f0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.919 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:40:22.919+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:40:22.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.923+0000 7fc563e00640 1 -- 172.21.15.67:0/4155086149 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc55c0591d0 con 0x7fc56c14f3d0 2024-02-26T10:40:22.923 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:40:22.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc54806b4d0 msgr2=0x7fc54806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc54806b4d0 0x7fc54806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc554038000 tx=0x7fc554008040 comp rx=0 tx=0).stop 2024-02-26T10:40:22.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 msgr2=0x7fc56c148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:22.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c148a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc55c007e40 tx=0x7fc55c00e4b0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 shutdown_connections 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc54806b4d0 0x7fc54806d990 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc554038000 tx=0x7fc554008040 comp rx=0 tx=0).stop 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc56c1556a0 0x7fc56c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc56c14f3d0 0x7fc56c148a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 --2- 172.21.15.67:0/4155086149 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc56c0c2870 0x7fc56c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 >> 172.21.15.67:0/4155086149 conn(0x7fc56c0a84a0 msgr2=0x7fc56c0c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.926+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 shutdown_connections 2024-02-26T10:40:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:22.927+0000 7fc573200640 1 -- 172.21.15.67:0/4155086149 wait complete. 2024-02-26T10:40:22.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T10:40:22.951 INFO:tasks.ceph.mds.d.smithi082.stderr: -3271> 2024-02-26T10:39:42.823+0000 7f19f2e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:40:23.045 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-02-26T10:40:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.163+0000 7fd16f200640 1 Processor -- start 2024-02-26T10:40:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.163+0000 7fd16f200640 1 -- start start 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1680aca30 0x7fd16806f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1680a84a0 0x7fd16813b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd168152e00 con 0x7fd16813bea0 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1681530e0 con 0x7fd1680aca30 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1681533c0 con 0x7fd1680a84a0 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1680aca30 0x7fd16806f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd16813bea0 0x7fd168156330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55498/0 (socket says 172.21.15.67:55498) 2024-02-26T10:40:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16e800640 1 -- 172.21.15.67:0/2227631010 learned_addr learned my addr 172.21.15.67:0/2227631010 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.165+0000 7fd16d400640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1680a84a0 0x7fd16813b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.166+0000 7fd16de00640 1 -- 172.21.15.67:0/2227631010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1680a84a0 msgr2=0x7fd16813b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.166+0000 7fd16de00640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1680a84a0 0x7fd16813b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.166+0000 7fd16de00640 1 -- 172.21.15.67:0/2227631010 >> v1:172.21.15.67:6789/0 conn(0x7fd16813bea0 legacy=0x7fd168156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.166+0000 7fd16de00640 1 -- 172.21.15.67:0/2227631010 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1681536a0 con 0x7fd1680aca30 2024-02-26T10:40:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.166+0000 7fd16d400640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1680a84a0 0x7fd16813b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.166+0000 7fd16ca00640 1 -- 172.21.15.67:0/2227631010 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142434756 0 0) 0x7fd168152e00 con 0x7fd16813bea0 2024-02-26T10:40:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.167+0000 7fd16de00640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1680aca30 0x7fd16806f820 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0097a0 tx=0x7fd15c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd80927612396e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.167+0000 7fd16ca00640 1 -- 172.21.15.67:0/2227631010 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c035020 con 0x7fd1680aca30 2024-02-26T10:40:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.167+0000 7fd16ca00640 1 -- 172.21.15.67:0/2227631010 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c031810 con 0x7fd1680aca30 2024-02-26T10:40:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.167+0000 7fd16ca00640 1 -- 172.21.15.67:0/2227631010 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c031be0 con 0x7fd1680aca30 2024-02-26T10:40:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 -- 172.21.15.67:0/2227631010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1680aca30 msgr2=0x7fd16806f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1680aca30 0x7fd16806f820 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0097a0 tx=0x7fd15c02b8b0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 -- 172.21.15.67:0/2227631010 shutdown_connections 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1680a84a0 0x7fd16813b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 --2- 172.21.15.67:0/2227631010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1680aca30 0x7fd16806f820 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 -- 172.21.15.67:0/2227631010 >> 172.21.15.67:0/2227631010 conn(0x7fd1680c7400 msgr2=0x7fd1680c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 -- 172.21.15.67:0/2227631010 shutdown_connections 2024-02-26T10:40:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.168+0000 7fd16f200640 1 -- 172.21.15.67:0/2227631010 wait complete. 2024-02-26T10:40:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.170+0000 7fd16f200640 1 Processor -- start 2024-02-26T10:40:23.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.170+0000 7fd16f200640 1 -- start start 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1680a84a0 0x7fd1680c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 0x7fd168156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd168156ef0 0x7fd168136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16813c2f0 con 0x7fd16813bea0 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680a8c90 con 0x7fd1680a84a0 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16813c470 con 0x7fd168156ef0 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 0x7fd168156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 0x7fd168156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52000/0 (socket says 172.21.15.67:52000) 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.171+0000 7fd16d400640 1 -- 172.21.15.67:0/1666192454 learned_addr learned my addr 172.21.15.67:0/1666192454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16e800640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd168156ef0 0x7fd168136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16d400640 1 -- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd168156ef0 msgr2=0x7fd168136cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16d400640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd168156ef0 0x7fd168136cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16d400640 1 -- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1680a84a0 msgr2=0x7fd1680c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16d400640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1680a84a0 0x7fd1680c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16d400640 1 -- 172.21.15.67:0/1666192454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1681eb5e0 con 0x7fd16813bea0 2024-02-26T10:40:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16e800640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd168156ef0 0x7fd168136cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16d400640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 0x7fd168156020 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c7f0 tx=0x7fd15800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158015070 con 0x7fd16813bea0 2024-02-26T10:40:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd158007820 con 0x7fd16813bea0 2024-02-26T10:40:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1681eb880 con 0x7fd16813bea0 2024-02-26T10:40:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.172+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158007bd0 con 0x7fd16813bea0 2024-02-26T10:40:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.173+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1681ebe40 con 0x7fd16813bea0 2024-02-26T10:40:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.174+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd168074a30 con 0x7fd16813bea0 2024-02-26T10:40:23.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.175+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd158018070 con 0x7fd16813bea0 2024-02-26T10:40:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.176+0000 7fd162000640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd14c06b500 0x7fd14c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.176+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd15808a670 con 0x7fd16813bea0 2024-02-26T10:40:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.177+0000 7fd16de00640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd14c06b500 0x7fd14c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:23.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.178+0000 7fd16de00640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd14c06b500 0x7fd14c06d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd15c036000 tx=0x7fd15c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:23.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.180+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1580590f0 con 0x7fd16813bea0 2024-02-26T10:40:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.425+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd1680c2e30 con 0x7fd16813bea0 2024-02-26T10:40:23.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.926+0000 7fd162000640 1 -- 172.21.15.67:0/1666192454 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd1580589f0 con 0x7fd16813bea0 2024-02-26T10:40:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd14c06b500 msgr2=0x7fd14c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd14c06b500 0x7fd14c06d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd15c036000 tx=0x7fd15c030000 comp rx=0 tx=0).stop 2024-02-26T10:40:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 msgr2=0x7fd168156020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 0x7fd168156020 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c7f0 tx=0x7fd15800ccb0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 shutdown_connections 2024-02-26T10:40:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd168156ef0 0x7fd168136cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd16813bea0 0x7fd168156020 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd14c06b500 0x7fd14c06d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 --2- 172.21.15.67:0/1666192454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1680a84a0 0x7fd1680c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 >> 172.21.15.67:0/1666192454 conn(0x7fd1680c7400 msgr2=0x7fd168071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 shutdown_connections 2024-02-26T10:40:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:23.931+0000 7fd16f200640 1 -- 172.21.15.67:0/1666192454 wait complete. 2024-02-26T10:40:23.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T10:40:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.167+0000 7f03f5000640 1 Processor -- start 2024-02-26T10:40:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.167+0000 7f03f5000640 1 -- start start 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03f5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03f006ac20 0x7f03f006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03f5000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f03f00a94a0 0x7f03f006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03f5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f00a99b0 con 0x7f03f0153610 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03f5000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f00aba30 con 0x7f03f006ac20 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03f5000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f00ac5d0 con 0x7f03f00a94a0 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03efe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f03f0153610 0x7f03f00abec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55500/0 (socket says 172.21.15.67:55500) 2024-02-26T10:40:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03efe00640 1 -- 172.21.15.67:0/514553769 learned_addr learned my addr 172.21.15.67:0/514553769 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03ef400640 1 --2- 172.21.15.67:0/514553769 >> v2:172.21.15.67:3300/0 conn(0x7f03f006ac20 0x7f03f006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.169+0000 7f03eea00640 1 --2- 172.21.15.67:0/514553769 >> v2:172.21.15.82:3300/0 conn(0x7f03f00a94a0 0x7f03f006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.170+0000 7f03ef400640 1 -- 172.21.15.67:0/514553769 >> v2:172.21.15.82:3300/0 conn(0x7f03f00a94a0 msgr2=0x7f03f006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.170+0000 7f03ef400640 1 --2- 172.21.15.67:0/514553769 >> v2:172.21.15.82:3300/0 conn(0x7f03f00a94a0 0x7f03f006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.170+0000 7f03ef400640 1 -- 172.21.15.67:0/514553769 >> v1:172.21.15.67:6789/0 conn(0x7f03f0153610 legacy=0x7f03f00abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.170+0000 7f03ef400640 1 -- 172.21.15.67:0/514553769 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00a9c90 con 0x7f03f006ac20 2024-02-26T10:40:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.171+0000 7f03ef400640 1 --2- 172.21.15.67:0/514553769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f006db00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f03f006ed00 tx=0x7f03e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85937c2f155b1a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.171+0000 7f03ee000640 1 -- 172.21.15.67:0/514553769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00315a0 con 0x7f03f006ac20 2024-02-26T10:40:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.171+0000 7f03ee000640 1 -- 172.21.15.67:0/514553769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0037070 con 0x7f03f006ac20 2024-02-26T10:40:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.171+0000 7f03ee000640 1 -- 172.21.15.67:0/514553769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0031960 con 0x7f03f006ac20 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 -- 172.21.15.67:0/514553769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 msgr2=0x7f03f006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 --2- 172.21.15.67:0/514553769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f006db00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f03f006ed00 tx=0x7f03e002b840 comp rx=0 tx=0).stop 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 -- 172.21.15.67:0/514553769 shutdown_connections 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 --2- 172.21.15.67:0/514553769 >> v2:172.21.15.82:3300/0 conn(0x7f03f00a94a0 0x7f03f006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 --2- 172.21.15.67:0/514553769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f006db00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 -- 172.21.15.67:0/514553769 >> 172.21.15.67:0/514553769 conn(0x7f03f00c63f0 msgr2=0x7f03f00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 -- 172.21.15.67:0/514553769 shutdown_connections 2024-02-26T10:40:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.172+0000 7f03f5000640 1 -- 172.21.15.67:0/514553769 wait complete. 2024-02-26T10:40:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.174+0000 7f03f5000640 1 Processor -- start 2024-02-26T10:40:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.174+0000 7f03f5000640 1 -- start start 2024-02-26T10:40:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03f5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f0072820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03f5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03f00a94a0 0x7f03f0152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03f5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03f0153610 0x7f03f0149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03f5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0153a40 con 0x7f03f006ac20 2024-02-26T10:40:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03f5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f01504f0 con 0x7f03f00a94a0 2024-02-26T10:40:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03f5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f01eea20 con 0x7f03f0153610 2024-02-26T10:40:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03ef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f0072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03ef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f0072820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52008/0 (socket says 172.21.15.67:52008) 2024-02-26T10:40:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.175+0000 7f03ef400640 1 -- 172.21.15.67:0/1886095743 learned_addr learned my addr 172.21.15.67:0/1886095743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03ef400640 1 -- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03f0153610 msgr2=0x7f03f0149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03ef400640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03f0153610 0x7f03f0149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03ef400640 1 -- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03f00a94a0 msgr2=0x7f03f0152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03ef400640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03f00a94a0 0x7f03f0152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03ef400640 1 -- 172.21.15.67:0/1886095743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f0072d60 con 0x7f03f006ac20 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03ef400640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f0072820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f03e0000c00 tx=0x7f03e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032840 con 0x7f03f006ac20 2024-02-26T10:40:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0032a00 con 0x7f03f006ac20 2024-02-26T10:40:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032df0 con 0x7f03f006ac20 2024-02-26T10:40:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.176+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f01ede10 con 0x7f03f006ac20 2024-02-26T10:40:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.177+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f01ee3d0 con 0x7f03f006ac20 2024-02-26T10:40:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.178+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0074a30 con 0x7f03f006ac20 2024-02-26T10:40:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.179+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03e004c020 con 0x7f03f006ac20 2024-02-26T10:40:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.179+0000 7f03e7400640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f03d006b4d0 0x7f03d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.180+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f03e002f020 con 0x7f03f006ac20 2024-02-26T10:40:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.180+0000 7f03eea00640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f03d006b4d0 0x7f03d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.181+0000 7f03eea00640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f03d006b4d0 0x7f03d006d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f03f01535e0 tx=0x7f03dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.184+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03e0040070 con 0x7f03f006ac20 2024-02-26T10:40:24.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:24.386+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03f006b9b0 con 0x7f03f006ac20 2024-02-26T10:40:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.323+0000 7f03e7400640 1 -- 172.21.15.67:0/1886095743 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03e007b070 con 0x7f03f006ac20 2024-02-26T10:40:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-02-26T10:40:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f03d006b4d0 msgr2=0x7f03d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f03d006b4d0 0x7f03d006d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f03f01535e0 tx=0x7f03dc008040 comp rx=0 tx=0).stop 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 msgr2=0x7f03f0072820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f0072820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f03e0000c00 tx=0x7f03e00046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 shutdown_connections 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03f0153610 0x7f03f0149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f03d006b4d0 0x7f03d006d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03f00a94a0 0x7f03f0152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 --2- 172.21.15.67:0/1886095743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03f006ac20 0x7f03f0072820 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 >> 172.21.15.67:0/1886095743 conn(0x7f03f00c63f0 msgr2=0x7f03f014ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 shutdown_connections 2024-02-26T10:40:25.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.327+0000 7f03f5000640 1 -- 172.21.15.67:0/1886095743 wait complete. 2024-02-26T10:40:25.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T10:40:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.553+0000 7f4ce9a00640 1 Processor -- start 2024-02-26T10:40:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.553+0000 7f4ce9a00640 1 -- start start 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce41523b0 0x7f4ce40c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ce4154cf0 0x7f4ce40c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce9a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce406dc80 con 0x7f4ce4071fd0 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce4152790 con 0x7f4ce4154cf0 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce406da30 con 0x7f4ce41523b0 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ce4154cf0 0x7f4ce40c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ce4154cf0 0x7f4ce40c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52016/0 (socket says 172.21.15.67:52016) 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.555+0000 7f4ce3400640 1 -- 172.21.15.67:0/873468192 learned_addr learned my addr 172.21.15.67:0/873468192 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.556+0000 7f4ce3e00640 1 --2- 172.21.15.67:0/873468192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce41523b0 0x7f4ce40c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.556+0000 7f4ce3400640 1 -- 172.21.15.67:0/873468192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce41523b0 msgr2=0x7f4ce40c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.556+0000 7f4ce3400640 1 --2- 172.21.15.67:0/873468192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce41523b0 0x7f4ce40c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.556+0000 7f4ce3400640 1 -- 172.21.15.67:0/873468192 >> v1:172.21.15.82:6789/0 conn(0x7f4ce4071fd0 legacy=0x7f4ce4072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.556+0000 7f4ce3400640 1 -- 172.21.15.67:0/873468192 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce406df60 con 0x7f4ce4154cf0 2024-02-26T10:40:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.557+0000 7f4ce3400640 1 --2- 172.21.15.67:0/873468192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce4154cf0 0x7f4ce40c3130 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009870 tx=0x7f4cd002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fb49f4297e93ff server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.557+0000 7f4ce2a00640 1 -- 172.21.15.67:0/873468192 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0035020 con 0x7f4ce4154cf0 2024-02-26T10:40:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce2a00640 1 -- 172.21.15.67:0/873468192 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd0031640 con 0x7f4ce4154cf0 2024-02-26T10:40:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce2a00640 1 -- 172.21.15.67:0/873468192 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd002d040 con 0x7f4ce4154cf0 2024-02-26T10:40:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 -- 172.21.15.67:0/873468192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce4154cf0 msgr2=0x7f4ce40c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/873468192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce4154cf0 0x7f4ce40c3130 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009870 tx=0x7f4cd002b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 -- 172.21.15.67:0/873468192 shutdown_connections 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/873468192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce4154cf0 0x7f4ce40c3130 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/873468192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce41523b0 0x7f4ce40c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 -- 172.21.15.67:0/873468192 >> 172.21.15.67:0/873468192 conn(0x7f4ce40c6380 msgr2=0x7f4ce40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 -- 172.21.15.67:0/873468192 shutdown_connections 2024-02-26T10:40:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.558+0000 7f4ce9a00640 1 -- 172.21.15.67:0/873468192 wait complete. 2024-02-26T10:40:25.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.560+0000 7f4ce9a00640 1 Processor -- start 2024-02-26T10:40:25.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.560+0000 7f4ce9a00640 1 -- start start 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce4071fd0 0x7f4ce40c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 0x7f4ce4143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce4154cf0 0x7f4ce41eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce406e800 con 0x7f4ce41523b0 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce406e520 con 0x7f4ce4071fd0 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40c08e0 con 0x7f4ce4154cf0 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 0x7f4ce4143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 0x7f4ce4143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52030/0 (socket says 172.21.15.67:52030) 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.561+0000 7f4ce3400640 1 -- 172.21.15.67:0/2333187239 learned_addr learned my addr 172.21.15.67:0/2333187239 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3e00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce4071fd0 0x7f4ce40c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce9000640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce4154cf0 0x7f4ce41eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:25.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3400640 1 -- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce4154cf0 msgr2=0x7f4ce41eacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3400640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce4154cf0 0x7f4ce41eacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3400640 1 -- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce4071fd0 msgr2=0x7f4ce40c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3400640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce4071fd0 0x7f4ce40c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3400640 1 -- 172.21.15.67:0/2333187239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce4144290 con 0x7f4ce41523b0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4ce3400640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 0x7f4ce4143740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c00 tx=0x7f4cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.562+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0040070 con 0x7f4ce41523b0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.563+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd0039bf0 con 0x7f4ce41523b0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.563+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0035040 con 0x7f4ce41523b0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.563+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce4143fb0 con 0x7f4ce41523b0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.563+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce41de6d0 con 0x7f4ce41523b0 2024-02-26T10:40:25.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.564+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce40678e0 con 0x7f4ce41523b0 2024-02-26T10:40:25.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.565+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cd004b060 con 0x7f4ce41523b0 2024-02-26T10:40:25.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.566+0000 7f4cdbe00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4cc406b4d0 0x7f4cc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:25.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.566+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6271+0+0 (secure 0 0 0) 0x7f4cd00abd70 con 0x7f4ce41523b0 2024-02-26T10:40:25.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.566+0000 7f4ce3e00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4cc406b4d0 0x7f4cc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.568+0000 7f4ce3e00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4cc406b4d0 0x7f4cc406d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006fd0 tx=0x7f4cd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:25.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.571+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cd002f300 con 0x7f4ce41523b0 2024-02-26T10:40:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:25.767+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4ce4074a30 con 0x7f4ce41523b0 2024-02-26T10:40:26.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.337+0000 7f4cdbe00640 1 -- 172.21.15.67:0/2333187239 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4cd007a9b0 con 0x7f4ce41523b0 2024-02-26T10:40:26.338 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.342+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4cc406b4d0 msgr2=0x7f4cc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.342+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4cc406b4d0 0x7f4cc406d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006fd0 tx=0x7f4cd4008040 comp rx=0 tx=0).stop 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.342+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 msgr2=0x7f4ce4143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.342+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 0x7f4ce4143740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c00 tx=0x7f4cd00046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.343+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 shutdown_connections 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.343+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce4154cf0 0x7f4ce41eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.343+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ce41523b0 0x7f4ce4143740 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.343+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4cc406b4d0 0x7f4cc406d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.343+0000 7f4ce9a00640 1 --2- 172.21.15.67:0/2333187239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce4071fd0 0x7f4ce40c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.343+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 >> 172.21.15.67:0/2333187239 conn(0x7f4ce40c6380 msgr2=0x7f4ce406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:26.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.344+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 shutdown_connections 2024-02-26T10:40:26.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.345+0000 7f4ce9a00640 1 -- 172.21.15.67:0/2333187239 wait complete. 2024-02-26T10:40:26.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:26.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.568+0000 7fbc33c00640 1 Processor -- start 2024-02-26T10:40:26.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.569+0000 7fbc33c00640 1 -- start start 2024-02-26T10:40:26.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.570+0000 7fbc33c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc2c150300 0x7fbc2c0c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.570+0000 7fbc33c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc2c06e410 0x7fbc2c0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.570+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c156230 0x7fbc2c0aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.570+0000 7fbc33c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c1506e0 con 0x7fbc2c06e410 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.570+0000 7fbc33c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c153a60 con 0x7fbc2c150300 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.570+0000 7fbc33c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c152e50 con 0x7fbc2c156230 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc2c06e410 0x7fbc2c0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbc2c06e410 0x7fbc2c0c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52032/0 (socket says 172.21.15.67:52032) 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 -- 172.21.15.67:0/903130310 learned_addr learned my addr 172.21.15.67:0/903130310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc31200640 1 --2- 172.21.15.67:0/903130310 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c150300 0x7fbc2c0c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc31c00640 1 --2- 172.21.15.67:0/903130310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c156230 0x7fbc2c0aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 -- 172.21.15.67:0/903130310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c156230 msgr2=0x7fbc2c0aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 --2- 172.21.15.67:0/903130310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c156230 0x7fbc2c0aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 -- 172.21.15.67:0/903130310 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c150300 msgr2=0x7fbc2c0c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 --2- 172.21.15.67:0/903130310 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c150300 0x7fbc2c0c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc2be00640 1 -- 172.21.15.67:0/903130310 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c153130 con 0x7fbc2c06e410 2024-02-26T10:40:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.571+0000 7fbc31200640 1 --2- 172.21.15.67:0/903130310 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c150300 0x7fbc2c0c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.572+0000 7fbc2be00640 1 --2- 172.21.15.67:0/903130310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c06e410 0x7fbc2c0c3530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c06bcc0 tx=0x7fbc1802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8c4183dc143950 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.573+0000 7fbc2b400640 1 -- 172.21.15.67:0/903130310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18038070 con 0x7fbc2c06e410 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.573+0000 7fbc2b400640 1 -- 172.21.15.67:0/903130310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc18031740 con 0x7fbc2c06e410 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.573+0000 7fbc2b400640 1 -- 172.21.15.67:0/903130310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1802f050 con 0x7fbc2c06e410 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.573+0000 7fbc33c00640 1 -- 172.21.15.67:0/903130310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c06e410 msgr2=0x7fbc2c0c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.573+0000 7fbc33c00640 1 --2- 172.21.15.67:0/903130310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c06e410 0x7fbc2c0c3530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c06bcc0 tx=0x7fbc1802b7e0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.573+0000 7fbc33c00640 1 -- 172.21.15.67:0/903130310 shutdown_connections 2024-02-26T10:40:26.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.574+0000 7fbc33c00640 1 --2- 172.21.15.67:0/903130310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c156230 0x7fbc2c0aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.574+0000 7fbc33c00640 1 --2- 172.21.15.67:0/903130310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c06e410 0x7fbc2c0c3530 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.574+0000 7fbc33c00640 1 --2- 172.21.15.67:0/903130310 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c150300 0x7fbc2c0c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.574+0000 7fbc33c00640 1 -- 172.21.15.67:0/903130310 >> 172.21.15.67:0/903130310 conn(0x7fbc2c0a84a0 msgr2=0x7fbc2c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.574+0000 7fbc33c00640 1 -- 172.21.15.67:0/903130310 shutdown_connections 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.574+0000 7fbc33c00640 1 -- 172.21.15.67:0/903130310 wait complete. 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.575+0000 7fbc33c00640 1 Processor -- start 2024-02-26T10:40:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.575+0000 7fbc33c00640 1 -- start start 2024-02-26T10:40:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.576+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c06e410 0x7fbc2c1ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.576+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c150300 0x7fbc2c1df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.576+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 0x7fbc2c14a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.576+0000 7fbc33c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c06dd80 con 0x7fbc2c156230 2024-02-26T10:40:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.576+0000 7fbc33c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c1536f0 con 0x7fbc2c06e410 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.576+0000 7fbc33c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0ab0e0 con 0x7fbc2c150300 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 0x7fbc2c14a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 0x7fbc2c14a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52044/0 (socket says 172.21.15.67:52044) 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 -- 172.21.15.67:0/2777073735 learned_addr learned my addr 172.21.15.67:0/2777073735 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 -- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c150300 msgr2=0x7fbc2c1df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc2be00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c150300 0x7fbc2c1df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31200640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c06e410 0x7fbc2c1ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c150300 0x7fbc2c1df6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 -- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c06e410 msgr2=0x7fbc2c1ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c06e410 0x7fbc2c1ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31c00640 1 -- 172.21.15.67:0/2777073735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c1e00b0 con 0x7fbc2c156230 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.577+0000 7fbc31200640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c06e410 0x7fbc2c1ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.578+0000 7fbc31c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 0x7fbc2c14a3e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc24007e90 tx=0x7fbc2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.578+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24014420 con 0x7fbc2c156230 2024-02-26T10:40:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.578+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24019070 con 0x7fbc2c156230 2024-02-26T10:40:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.578+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24014800 con 0x7fbc2c156230 2024-02-26T10:40:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.578+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c1dfdd0 con 0x7fbc2c156230 2024-02-26T10:40:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.578+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1ec2c0 con 0x7fbc2c156230 2024-02-26T10:40:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.579+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c074a30 con 0x7fbc2c156230 2024-02-26T10:40:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.580+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc240149a0 con 0x7fbc2c156230 2024-02-26T10:40:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.581+0000 7fbc28c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.581+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7fbc2408aae0 con 0x7fbc2c156230 2024-02-26T10:40:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.582+0000 7fbc31200640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:26.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.583+0000 7fbc31200640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.586+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc240598b0 con 0x7fbc2c156230 2024-02-26T10:40:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.784+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc2c06e0d0 con 0x7fbc2c156230 2024-02-26T10:40:26.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.788+0000 7fbc28c00640 1 -- 172.21.15.67:0/2777073735 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16768 (secure 0 0 0) 0x7fbc240591b0 con 0x7fbc2c156230 2024-02-26T10:40:26.788 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:26.788 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":43,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:26.324828+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/155874470":"2024-02-27T10:40:22.920450+0000","172.21.15.82:6835/155874470":"2024-02-27T10:40:22.920450+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc0c06b4d0 msgr2=0x7fbc0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc20006fd0 tx=0x7fbc20008040 comp rx=0 tx=0).stop 2024-02-26T10:40:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 msgr2=0x7fbc2c14a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 0x7fbc2c14a3e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbc24007e90 tx=0x7fbc2400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 shutdown_connections 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc2c156230 0x7fbc2c14a3e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c150300 0x7fbc2c1df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbc0c06b4d0 0x7fbc0c06d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 --2- 172.21.15.67:0/2777073735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c06e410 0x7fbc2c1ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 >> 172.21.15.67:0/2777073735 conn(0x7fbc2c0a84a0 msgr2=0x7fbc2c0a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 shutdown_connections 2024-02-26T10:40:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:26.791+0000 7fbc33c00640 1 -- 172.21.15.67:0/2777073735 wait complete. 2024-02-26T10:40:26.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:39:53.146836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:39:54.623593+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:26.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:27.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.034+0000 7f2553200640 1 Processor -- start 2024-02-26T10:40:27.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.035+0000 7f2553200640 1 -- start start 2024-02-26T10:40:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.036+0000 7f2553200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f255406efe0 0x7f255406f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.036+0000 7f2553200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f255406f900 0x7f255414eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.036+0000 7f2553200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255414f400 0x7f25540a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.036+0000 7f2553200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2554072cd0 con 0x7f255406efe0 2024-02-26T10:40:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.036+0000 7f2553200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25540a84a0 con 0x7f255406f900 2024-02-26T10:40:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.036+0000 7f2553200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2554071ed0 con 0x7f255414f400 2024-02-26T10:40:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f255406efe0 0x7f255406f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2552800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255414f400 0x7f25540a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f255406efe0 0x7f255406f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52052/0 (socket says 172.21.15.67:52052) 2024-02-26T10:40:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 -- 172.21.15.67:0/1479435475 learned_addr learned my addr 172.21.15.67:0/1479435475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551400640 1 --2- 172.21.15.67:0/1479435475 >> v2:172.21.15.82:3300/0 conn(0x7f255406f900 0x7f255414eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 -- 172.21.15.67:0/1479435475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255414f400 msgr2=0x7f25540a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:27.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 --2- 172.21.15.67:0/1479435475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255414f400 0x7f25540a8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 -- 172.21.15.67:0/1479435475 >> v2:172.21.15.82:3300/0 conn(0x7f255406f900 msgr2=0x7f255414eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:27.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 --2- 172.21.15.67:0/1479435475 >> v2:172.21.15.82:3300/0 conn(0x7f255406f900 0x7f255414eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.037+0000 7f2551e00640 1 -- 172.21.15.67:0/1479435475 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540721b0 con 0x7f255406efe0 2024-02-26T10:40:27.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.039+0000 7f2551e00640 1 --2- 172.21.15.67:0/1479435475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406efe0 0x7f255406f3c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f25540c8530 tx=0x7f254402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86d15c7c1b0c03e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.039+0000 7f2550a00640 1 -- 172.21.15.67:0/1479435475 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254402c9f0 con 0x7f255406efe0 2024-02-26T10:40:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.039+0000 7f2550a00640 1 -- 172.21.15.67:0/1479435475 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254402cb90 con 0x7f255406efe0 2024-02-26T10:40:27.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.039+0000 7f2550a00640 1 -- 172.21.15.67:0/1479435475 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440343f0 con 0x7f255406efe0 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 -- 172.21.15.67:0/1479435475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406efe0 msgr2=0x7f255406f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 --2- 172.21.15.67:0/1479435475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406efe0 0x7f255406f3c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f25540c8530 tx=0x7f254402be50 comp rx=0 tx=0).stop 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 -- 172.21.15.67:0/1479435475 shutdown_connections 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 --2- 172.21.15.67:0/1479435475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255414f400 0x7f25540a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 --2- 172.21.15.67:0/1479435475 >> v2:172.21.15.82:3300/0 conn(0x7f255406f900 0x7f255414eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 --2- 172.21.15.67:0/1479435475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406efe0 0x7f255406f3c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 -- 172.21.15.67:0/1479435475 >> 172.21.15.67:0/1479435475 conn(0x7f25540aabe0 msgr2=0x7f25540c7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:27.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 -- 172.21.15.67:0/1479435475 shutdown_connections 2024-02-26T10:40:27.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.040+0000 7f2553200640 1 -- 172.21.15.67:0/1479435475 wait complete. 2024-02-26T10:40:27.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.042+0000 7f2553200640 1 Processor -- start 2024-02-26T10:40:27.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.042+0000 7f2553200640 1 -- start start 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2553200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255406efe0 0x7f25541ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2553200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 0x7f2554145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2553200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f255414f400 0x7f25541460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2553200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554072ea0 con 0x7f255406f900 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2553200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554155db0 con 0x7f255414f400 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2553200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2554073020 con 0x7f255406efe0 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2551400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 0x7f2554145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2551400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 0x7f2554145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52066/0 (socket says 172.21.15.67:52066) 2024-02-26T10:40:27.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2551400640 1 -- 172.21.15.67:0/3097698960 learned_addr learned my addr 172.21.15.67:0/3097698960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2551e00640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255406efe0 0x7f25541ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.043+0000 7f2552800640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f255414f400 0x7f25541460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551400640 1 -- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255406efe0 msgr2=0x7f25541ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551400640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255406efe0 0x7f25541ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551400640 1 -- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f255414f400 msgr2=0x7f25541460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551400640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f255414f400 0x7f25541460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551400640 1 -- 172.21.15.67:0/3097698960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255414fd50 con 0x7f255406f900 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551e00640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255406efe0 0x7f25541ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2552800640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f255414f400 0x7f25541460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2551400640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 0x7f2554145b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f254000e890 tx=0x7f254000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540017070 con 0x7f255406f900 2024-02-26T10:40:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254000c940 con 0x7f255406f900 2024-02-26T10:40:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000cd10 con 0x7f255406f900 2024-02-26T10:40:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25541ec4b0 con 0x7f255406f900 2024-02-26T10:40:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.044+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2554146610 con 0x7f255406f900 2024-02-26T10:40:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.046+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25540678e0 con 0x7f255406f900 2024-02-26T10:40:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.046+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f254001a070 con 0x7f255406f900 2024-02-26T10:40:27.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.047+0000 7f254a000640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f253406b4d0 0x7f253406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.047+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f254008a5d0 con 0x7f255406f900 2024-02-26T10:40:27.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.048+0000 7f2551e00640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f253406b4d0 0x7f253406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.049+0000 7f2551e00640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f253406b4d0 0x7f253406d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2544035000 tx=0x7f2544030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.052+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25400593a0 con 0x7f255406f900 2024-02-26T10:40:27.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.255+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2554074a30 con 0x7f255406f900 2024-02-26T10:40:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.256+0000 7f254a000640 1 -- 172.21.15.67:0/3097698960 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2540058ca0 con 0x7f255406f900 2024-02-26T10:40:27.256 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f253406b4d0 msgr2=0x7f253406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f253406b4d0 0x7f253406d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2544035000 tx=0x7f2544030000 comp rx=0 tx=0).stop 2024-02-26T10:40:27.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 msgr2=0x7f2554145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 0x7f2554145b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f254000e890 tx=0x7f254000ed50 comp rx=0 tx=0).stop 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 shutdown_connections 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f253406b4d0 0x7f253406d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f255414f400 0x7f25541460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f255406f900 0x7f2554145b90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 --2- 172.21.15.67:0/3097698960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f255406efe0 0x7f25541ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 >> 172.21.15.67:0/3097698960 conn(0x7f25540aabe0 msgr2=0x7f255406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 shutdown_connections 2024-02-26T10:40:27.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.259+0000 7f2553200640 1 -- 172.21.15.67:0/3097698960 wait complete. 2024-02-26T10:40:27.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:40:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.500+0000 7f99b3200640 1 Processor -- start 2024-02-26T10:40:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.500+0000 7f99b3200640 1 -- start start 2024-02-26T10:40:27.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.501+0000 7f99b3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99ac1503d0 0x7f99ac06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.501+0000 7f99b3200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99ac06e390 0x7f99ac1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.501+0000 7f99b3200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0a6eb0 con 0x7f99ac0c2950 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.501+0000 7f99b3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0a7190 con 0x7f99ac1503d0 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.501+0000 7f99b3200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0a7470 con 0x7f99ac06e390 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99ac1503d0 0x7f99ac06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99ac1503d0 0x7f99ac06de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52082/0 (socket says 172.21.15.67:52082) 2024-02-26T10:40:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 -- 172.21.15.67:0/4210841919 learned_addr learned my addr 172.21.15.67:0/4210841919 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 -- 172.21.15.67:0/4210841919 >> v2:172.21.15.82:3300/0 conn(0x7f99ac06e390 msgr2=0x7f99ac1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 --2- 172.21.15.67:0/4210841919 >> v2:172.21.15.82:3300/0 conn(0x7f99ac06e390 0x7f99ac1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 -- 172.21.15.67:0/4210841919 >> v1:172.21.15.82:6789/0 conn(0x7f99ac0c2950 legacy=0x7f99ac156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.502+0000 7f99abe00640 1 -- 172.21.15.67:0/4210841919 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0a7750 con 0x7f99ac1503d0 2024-02-26T10:40:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.503+0000 7f99abe00640 1 --2- 172.21.15.67:0/4210841919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac06de50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0734b0 tx=0x7f999c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22986c897f9a6308 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99aaa00640 1 -- 172.21.15.67:0/4210841919 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c038070 con 0x7f99ac1503d0 2024-02-26T10:40:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99aaa00640 1 -- 172.21.15.67:0/4210841919 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c02f050 con 0x7f99ac1503d0 2024-02-26T10:40:27.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99aaa00640 1 -- 172.21.15.67:0/4210841919 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c031970 con 0x7f99ac1503d0 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 -- 172.21.15.67:0/4210841919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 msgr2=0x7f99ac06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 --2- 172.21.15.67:0/4210841919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac06de50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0734b0 tx=0x7f999c02b7f0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 -- 172.21.15.67:0/4210841919 shutdown_connections 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 --2- 172.21.15.67:0/4210841919 >> v2:172.21.15.82:3300/0 conn(0x7f99ac06e390 0x7f99ac1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 --2- 172.21.15.67:0/4210841919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac06de50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 -- 172.21.15.67:0/4210841919 >> 172.21.15.67:0/4210841919 conn(0x7f99ac0c6da0 msgr2=0x7f99ac0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.504+0000 7f99b3200640 1 -- 172.21.15.67:0/4210841919 shutdown_connections 2024-02-26T10:40:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.505+0000 7f99b3200640 1 -- 172.21.15.67:0/4210841919 wait complete. 2024-02-26T10:40:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.506+0000 7f99b3200640 1 Processor -- start 2024-02-26T10:40:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.506+0000 7f99b3200640 1 -- start start 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99ac06e390 0x7f99ac0723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ac0c2950 0x7f99ac141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac1507b0 con 0x7f99ac1503d0 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac06ef00 con 0x7f99ac0c2950 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0c0ef0 con 0x7f99ac06e390 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.507+0000 7f99b1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52088/0 (socket says 172.21.15.67:52088) 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 -- 172.21.15.67:0/3498486566 learned_addr learned my addr 172.21.15.67:0/3498486566 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99abe00640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99ac06e390 0x7f99ac0723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99ab400640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ac0c2950 0x7f99ac141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 -- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99ac06e390 msgr2=0x7f99ac0723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99ac06e390 0x7f99ac0723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 -- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ac0c2950 msgr2=0x7f99ac141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ac0c2950 0x7f99ac141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 -- 172.21.15.67:0/3498486566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac1eae40 con 0x7f99ac1503d0 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.508+0000 7f99b1200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac151fa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f99a400c390 tx=0x7f99a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.509+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4017070 con 0x7f99ac1503d0 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.509+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a40035f0 con 0x7f99ac1503d0 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.509+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40039e0 con 0x7f99ac1503d0 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.509+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac1eb120 con 0x7f99ac1503d0 2024-02-26T10:40:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.509+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac1eb6e0 con 0x7f99ac1503d0 2024-02-26T10:40:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.510+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac074a30 con 0x7f99ac1503d0 2024-02-26T10:40:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.511+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99a401a070 con 0x7f99ac1503d0 2024-02-26T10:40:27.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.512+0000 7f99a3e00640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f998806b4d0 0x7f998806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.512+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f99a4089650 con 0x7f99ac1503d0 2024-02-26T10:40:27.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.512+0000 7f99abe00640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f998806b4d0 0x7f998806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.513+0000 7f99abe00640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f998806b4d0 0x7f998806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f999c035000 tx=0x7f999c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.516+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99a40584a0 con 0x7f99ac1503d0 2024-02-26T10:40:27.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.717+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99ac06ec50 con 0x7f99ac1503d0 2024-02-26T10:40:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.718+0000 7f99a3e00640 1 -- 172.21.15.67:0/3498486566 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v43) v1 ==== 105+0+11 (secure 0 0 0) 0x7f99a4057da0 con 0x7f99ac1503d0 2024-02-26T10:40:27.718 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:27.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f998806b4d0 msgr2=0x7f998806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f998806b4d0 0x7f998806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f999c035000 tx=0x7f999c008030 comp rx=0 tx=0).stop 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 msgr2=0x7f99ac151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac151fa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f99a400c390 tx=0x7f99a400c850 comp rx=0 tx=0).stop 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 shutdown_connections 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ac1503d0 0x7f99ac151fa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f998806b4d0 0x7f998806d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99ac0c2950 0x7f99ac141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 --2- 172.21.15.67:0/3498486566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99ac06e390 0x7f99ac0723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.720+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 >> 172.21.15.67:0/3498486566 conn(0x7f99ac0c6da0 msgr2=0x7f99ac0c2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.721+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 shutdown_connections 2024-02-26T10:40:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.721+0000 7f99b3200640 1 -- 172.21.15.67:0/3498486566 wait complete. 2024-02-26T10:40:27.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:40:27.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.965+0000 7fc097200640 1 Processor -- start 2024-02-26T10:40:27.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.965+0000 7fc097200640 1 -- start start 2024-02-26T10:40:27.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.967+0000 7fc097200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0900a9430 0x7fc0900ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.967+0000 7fc097200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc09006e2d0 0x7fc0900c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.967+0000 7fc097200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc090150780 con 0x7fc0901503a0 2024-02-26T10:40:27.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.967+0000 7fc097200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0900a98c0 con 0x7fc09006e2d0 2024-02-26T10:40:27.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.967+0000 7fc097200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900a9ba0 con 0x7fc0900a9430 2024-02-26T10:40:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.968+0000 7fc095400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0900a9430 0x7fc0900ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.968+0000 7fc095e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc0901503a0 0x7fc0901535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:54106/0 (socket says 172.21.15.67:54106) 2024-02-26T10:40:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.968+0000 7fc095e00640 1 -- 172.21.15.67:0/2651215652 learned_addr learned my addr 172.21.15.67:0/2651215652 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.968+0000 7fc096800640 1 --2- 172.21.15.67:0/2651215652 >> v2:172.21.15.82:3300/0 conn(0x7fc09006e2d0 0x7fc0900c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc094a00640 1 -- 172.21.15.67:0/2651215652 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830827738 0 0) 0x7fc090150780 con 0x7fc0901503a0 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc094a00640 1 -- 172.21.15.67:0/2651215652 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003520 con 0x7fc0901503a0 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc096800640 1 -- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0900a9430 msgr2=0x7fc0900ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc096800640 1 --2- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0900a9430 0x7fc0900ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc096800640 1 -- 172.21.15.67:0/2651215652 >> v1:172.21.15.82:6789/0 conn(0x7fc0901503a0 legacy=0x7fc0901535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc096800640 1 -- 172.21.15.67:0/2651215652 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0900a9e80 con 0x7fc09006e2d0 2024-02-26T10:40:27.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.969+0000 7fc095400640 1 --2- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0900a9430 0x7fc0900ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:40:27.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.970+0000 7fc096800640 1 --2- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc09006e2d0 0x7fc0900c2e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009d90 tx=0x7fc08c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea829d3ea070829 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.971+0000 7fc094a00640 1 -- 172.21.15.67:0/2651215652 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c039070 con 0x7fc09006e2d0 2024-02-26T10:40:27.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.971+0000 7fc094a00640 1 -- 172.21.15.67:0/2651215652 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c02cc50 con 0x7fc09006e2d0 2024-02-26T10:40:27.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.971+0000 7fc094a00640 1 -- 172.21.15.67:0/2651215652 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c034440 con 0x7fc09006e2d0 2024-02-26T10:40:27.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 -- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc09006e2d0 msgr2=0x7fc0900c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 --2- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc09006e2d0 0x7fc0900c2e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009d90 tx=0x7fc08c02bea0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 -- 172.21.15.67:0/2651215652 shutdown_connections 2024-02-26T10:40:27.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 --2- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc09006e2d0 0x7fc0900c2e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 --2- 172.21.15.67:0/2651215652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0900a9430 0x7fc0900ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 -- 172.21.15.67:0/2651215652 >> 172.21.15.67:0/2651215652 conn(0x7fc0900c6380 msgr2=0x7fc0900c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:27.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 -- 172.21.15.67:0/2651215652 shutdown_connections 2024-02-26T10:40:27.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.972+0000 7fc097200640 1 -- 172.21.15.67:0/2651215652 wait complete. 2024-02-26T10:40:27.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.974+0000 7fc097200640 1 Processor -- start 2024-02-26T10:40:27.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.974+0000 7fc097200640 1 -- start start 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.975+0000 7fc097200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc09006e2d0 0x7fc09014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.975+0000 7fc097200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0901503a0 0x7fc09013dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc097200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 0x7fc09013e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc097200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900c0f80 con 0x7fc090148180 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc097200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09006ddb0 con 0x7fc0901503a0 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc097200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09006e740 con 0x7fc09006e2d0 2024-02-26T10:40:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc096800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 0x7fc09013e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc096800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 0x7fc09013e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52104/0 (socket says 172.21.15.67:52104) 2024-02-26T10:40:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc096800640 1 -- 172.21.15.67:0/3492488195 learned_addr learned my addr 172.21.15.67:0/3492488195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc095e00640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc09006e2d0 0x7fc09014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.976+0000 7fc095400640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0901503a0 0x7fc09013dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.977+0000 7fc096800640 1 -- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc09006e2d0 msgr2=0x7fc09014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.977+0000 7fc096800640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc09006e2d0 0x7fc09014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.977+0000 7fc096800640 1 -- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0901503a0 msgr2=0x7fc09013dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.977+0000 7fc096800640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0901503a0 0x7fc09013dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.977+0000 7fc096800640 1 -- 172.21.15.67:0/3492488195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0901ec830 con 0x7fc090148180 2024-02-26T10:40:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.977+0000 7fc096800640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 0x7fc09013e230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009d90 tx=0x7fc08c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.978+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c039050 con 0x7fc090148180 2024-02-26T10:40:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.978+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c045070 con 0x7fc090148180 2024-02-26T10:40:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.978+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c009450 con 0x7fc090148180 2024-02-26T10:40:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.978+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0901ecad0 con 0x7fc090148180 2024-02-26T10:40:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.978+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc090154710 con 0x7fc090148180 2024-02-26T10:40:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.979+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0900678e0 con 0x7fc090148180 2024-02-26T10:40:27.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.980+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc08c03e610 con 0x7fc090148180 2024-02-26T10:40:27.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.981+0000 7fc08a000640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc07006b500 0x7fc07006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:27.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.981+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7fc08c0ac6a0 con 0x7fc090148180 2024-02-26T10:40:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.982+0000 7fc095e00640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc07006b500 0x7fc07006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.983+0000 7fc095e00640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc07006b500 0x7fc07006d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005880 tx=0x7fc07c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:27.986+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc08c04e020 con 0x7fc090148180 2024-02-26T10:40:28.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.195+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fc0901548e0 con 0x7fc090148180 2024-02-26T10:40:28.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.196+0000 7fc08a000640 1 -- 172.21.15.67:0/3492488195 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v43) v1 ==== 101+0+11 (secure 0 0 0) 0x7fc08c07b3f0 con 0x7fc090148180 2024-02-26T10:40:28.196 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc07006b500 msgr2=0x7fc07006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc07006b500 0x7fc07006d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005880 tx=0x7fc07c009040 comp rx=0 tx=0).stop 2024-02-26T10:40:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 msgr2=0x7fc09013e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 0x7fc09013e230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009d90 tx=0x7fc08c004a40 comp rx=0 tx=0).stop 2024-02-26T10:40:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 shutdown_connections 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc090148180 0x7fc09013e230 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc07006b500 0x7fc07006d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0901503a0 0x7fc09013dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 --2- 172.21.15.67:0/3492488195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc09006e2d0 0x7fc09014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 >> 172.21.15.67:0/3492488195 conn(0x7fc0900c6380 msgr2=0x7fc09014c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 shutdown_connections 2024-02-26T10:40:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.199+0000 7fc097200640 1 -- 172.21.15.67:0/3492488195 wait complete. 2024-02-26T10:40:28.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.439+0000 7ff33c200640 1 Processor -- start 2024-02-26T10:40:28.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.439+0000 7ff33c200640 1 -- start start 2024-02-26T10:40:28.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.440+0000 7ff33c200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff33414f3d0 0x7ff33406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.440+0000 7ff33c200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff334071700 0x7ff334071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.440+0000 7ff33c200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33406ed10 con 0x7ff3340c2870 2024-02-26T10:40:28.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.440+0000 7ff33c200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33406dd80 con 0x7ff33414f3d0 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.440+0000 7ff33c200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff334155c00 con 0x7ff334071700 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff33414f3d0 0x7ff33406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff33414f3d0 0x7ff33406e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52112/0 (socket says 172.21.15.67:52112) 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff338e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff3340c2870 0x7ff334155160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55506/0 (socket says 172.21.15.67:55506) 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 -- 172.21.15.67:0/551613438 learned_addr learned my addr 172.21.15.67:0/551613438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff33a200640 1 --2- 172.21.15.67:0/551613438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff334071700 0x7ff334071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 -- 172.21.15.67:0/551613438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff334071700 msgr2=0x7ff334071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 --2- 172.21.15.67:0/551613438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff334071700 0x7ff334071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 -- 172.21.15.67:0/551613438 >> v1:172.21.15.67:6789/0 conn(0x7ff3340c2870 legacy=0x7ff334155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff339800640 1 -- 172.21.15.67:0/551613438 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334155ee0 con 0x7ff33414f3d0 2024-02-26T10:40:28.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.441+0000 7ff333e00640 1 -- 172.21.15.67:0/551613438 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67472698 0 0) 0x7ff33406ed10 con 0x7ff3340c2870 2024-02-26T10:40:28.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.442+0000 7ff339800640 1 --2- 172.21.15.67:0/551613438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff33414f3d0 0x7ff33406e7d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff328009870 tx=0x7ff32802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218bf51d50e35717 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:28.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.443+0000 7ff333e00640 1 -- 172.21.15.67:0/551613438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328035020 con 0x7ff33414f3d0 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.443+0000 7ff333e00640 1 -- 172.21.15.67:0/551613438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff328031640 con 0x7ff33414f3d0 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.443+0000 7ff333e00640 1 -- 172.21.15.67:0/551613438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32802d040 con 0x7ff33414f3d0 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.443+0000 7ff33c200640 1 -- 172.21.15.67:0/551613438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff33414f3d0 msgr2=0x7ff33406e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.443+0000 7ff33c200640 1 --2- 172.21.15.67:0/551613438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff33414f3d0 0x7ff33406e7d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff328009870 tx=0x7ff32802b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.444+0000 7ff33c200640 1 -- 172.21.15.67:0/551613438 shutdown_connections 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.444+0000 7ff33c200640 1 --2- 172.21.15.67:0/551613438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff334071700 0x7ff334071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.444+0000 7ff33c200640 1 --2- 172.21.15.67:0/551613438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff33414f3d0 0x7ff33406e7d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.444+0000 7ff33c200640 1 -- 172.21.15.67:0/551613438 >> 172.21.15.67:0/551613438 conn(0x7ff3340a84a0 msgr2=0x7ff3340a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.444+0000 7ff33c200640 1 -- 172.21.15.67:0/551613438 shutdown_connections 2024-02-26T10:40:28.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.444+0000 7ff33c200640 1 -- 172.21.15.67:0/551613438 wait complete. 2024-02-26T10:40:28.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.445+0000 7ff33c200640 1 Processor -- start 2024-02-26T10:40:28.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.445+0000 7ff33c200640 1 -- start start 2024-02-26T10:40:28.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.446+0000 7ff33c200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff334071700 0x7ff3341ea390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff33c200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 0x7ff3341e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff33c200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff33414f3d0 0x7ff334143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff33c200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340c1810 con 0x7ff3340c2870 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff33c200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33406e2b0 con 0x7ff334071700 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff33c200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff334155870 con 0x7ff33414f3d0 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 0x7ff3341e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 0x7ff3341e9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52122/0 (socket says 172.21.15.67:52122) 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 -- 172.21.15.67:0/77071773 learned_addr learned my addr 172.21.15.67:0/77071773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff33a200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff33414f3d0 0x7ff334143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 -- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff33414f3d0 msgr2=0x7ff334143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff33414f3d0 0x7ff334143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 -- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff334071700 msgr2=0x7ff3341ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff334071700 0x7ff3341ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.447+0000 7ff338e00640 1 -- 172.21.15.67:0/77071773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3341e6d90 con 0x7ff3340c2870 2024-02-26T10:40:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.448+0000 7ff338e00640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 0x7ff3341e9af0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c460 tx=0x7ff32400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:28.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.448+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324018070 con 0x7ff3340c2870 2024-02-26T10:40:28.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.448+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff324007500 con 0x7ff3340c2870 2024-02-26T10:40:28.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.448+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324004a20 con 0x7ff3340c2870 2024-02-26T10:40:28.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.448+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff334143a30 con 0x7ff3340c2870 2024-02-26T10:40:28.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.448+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff334143110 con 0x7ff3340c2870 2024-02-26T10:40:28.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.449+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff334143350 con 0x7ff3340c2870 2024-02-26T10:40:28.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.450+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff32401b070 con 0x7ff3340c2870 2024-02-26T10:40:28.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.452+0000 7ff331600640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff31806b4d0 0x7ff31806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.452+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7ff324089580 con 0x7ff3340c2870 2024-02-26T10:40:28.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.453+0000 7ff339800640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff31806b4d0 0x7ff31806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.454+0000 7ff339800640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff31806b4d0 0x7ff31806d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff328036000 tx=0x7ff328030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:28.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.457+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff324058350 con 0x7ff3340c2870 2024-02-26T10:40:28.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.687+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff334142f00 con 0x7ff3340c2870 2024-02-26T10:40:28.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.688+0000 7ff331600640 1 -- 172.21.15.67:0/77071773 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4931 (secure 0 0 0) 0x7ff324057c50 con 0x7ff3340c2870 2024-02-26T10:40:28.688 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:28.688 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/224163452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":224163452},{"type":"v1","addr":"172.21.15.82:6837","nonce":224163452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 24 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff31806b4d0 msgr2=0x7ff31806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff31806b4d0 0x7ff31806d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff328036000 tx=0x7ff328030000 comp rx=0 tx=0).stop 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 msgr2=0x7ff3341e9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 0x7ff3341e9af0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c460 tx=0x7ff32400c920 comp rx=0 tx=0).stop 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 shutdown_connections 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff33414f3d0 0x7ff334143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3340c2870 0x7ff3341e9af0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff31806b4d0 0x7ff31806d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 --2- 172.21.15.67:0/77071773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff334071700 0x7ff3341ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 >> 172.21.15.67:0/77071773 conn(0x7ff3340a84a0 msgr2=0x7ff3340741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:28.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 shutdown_connections 2024-02-26T10:40:28.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.691+0000 7ff33c200640 1 -- 172.21.15.67:0/77071773 wait complete. 2024-02-26T10:40:28.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:28.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.924+0000 7feefbe00640 1 Processor -- start 2024-02-26T10:40:28.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.924+0000 7feefbe00640 1 -- start start 2024-02-26T10:40:28.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.926+0000 7feefbe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.926+0000 7feefbe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feefc06db00 0x7feefc0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.926+0000 7feefbe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc06b460 con 0x7feefc06b000 2024-02-26T10:40:28.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.926+0000 7feefbe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc06dfb0 con 0x7feefc06db00 2024-02-26T10:40:28.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.926+0000 7feefbe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0a9a80 con 0x7feefc06ac20 2024-02-26T10:40:28.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.927+0000 7feefaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.927+0000 7feefaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:34888/0 (socket says 172.21.15.67:34888) 2024-02-26T10:40:28.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.927+0000 7feefaa00640 1 -- 172.21.15.67:0/893033362 learned_addr learned my addr 172.21.15.67:0/893033362 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:28.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.927+0000 7feefb400640 1 --2- 172.21.15.67:0/893033362 >> v2:172.21.15.82:3300/0 conn(0x7feefc06db00 0x7feefc0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.928+0000 7feefaa00640 1 -- 172.21.15.67:0/893033362 >> v2:172.21.15.82:3300/0 conn(0x7feefc06db00 msgr2=0x7feefc0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:28.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.928+0000 7feefaa00640 1 --2- 172.21.15.67:0/893033362 >> v2:172.21.15.82:3300/0 conn(0x7feefc06db00 0x7feefc0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.928+0000 7feefaa00640 1 -- 172.21.15.67:0/893033362 >> v1:172.21.15.82:6789/0 conn(0x7feefc06b000 legacy=0x7feefc153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.928+0000 7feefaa00640 1 -- 172.21.15.67:0/893033362 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0a9d60 con 0x7feefc06ac20 2024-02-26T10:40:28.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.929+0000 7feefaa00640 1 --2- 172.21.15.67:0/893033362 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc0a94a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feeec009870 tx=0x7feeec02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0423e28b904b88 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.930+0000 7feefa000640 1 -- 172.21.15.67:0/893033362 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec034070 con 0x7feefc06ac20 2024-02-26T10:40:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.930+0000 7feefa000640 1 -- 172.21.15.67:0/893033362 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec030840 con 0x7feefc06ac20 2024-02-26T10:40:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.930+0000 7feefa000640 1 -- 172.21.15.67:0/893033362 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec030c10 con 0x7feefc06ac20 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 -- 172.21.15.67:0/893033362 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 msgr2=0x7feefc0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 --2- 172.21.15.67:0/893033362 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc0a94a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feeec009870 tx=0x7feeec02b980 comp rx=0 tx=0).stop 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 -- 172.21.15.67:0/893033362 shutdown_connections 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 --2- 172.21.15.67:0/893033362 >> v2:172.21.15.82:3300/0 conn(0x7feefc06db00 0x7feefc0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 --2- 172.21.15.67:0/893033362 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc0a94a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 -- 172.21.15.67:0/893033362 >> 172.21.15.67:0/893033362 conn(0x7feefc0c63f0 msgr2=0x7feefc0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 -- 172.21.15.67:0/893033362 shutdown_connections 2024-02-26T10:40:28.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.931+0000 7feefbe00640 1 -- 172.21.15.67:0/893033362 wait complete. 2024-02-26T10:40:28.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.933+0000 7feefbe00640 1 Processor -- start 2024-02-26T10:40:28.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.934+0000 7feefbe00640 1 -- start start 2024-02-26T10:40:28.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefbe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 0x7feefc0c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feefc06db00 0x7feefc0c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0ac490 con 0x7feefc06b000 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefbe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0aba30 con 0x7feefc06db00 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefbe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc150410 con 0x7feefc06ac20 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feef3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 0x7feefc0c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feef3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 0x7feefc0c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52128/0 (socket says 172.21.15.67:52128) 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feef3e00640 1 -- 172.21.15.67:0/1477324929 learned_addr learned my addr 172.21.15.67:0/1477324929 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefaa00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.935+0000 7feefb400640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feefc06db00 0x7feefc0c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef3e00640 1 -- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 msgr2=0x7feefc071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef3e00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef3e00640 1 -- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feefc06db00 msgr2=0x7feefc0c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:28.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef3e00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feefc06db00 0x7feefc0c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:28.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef3e00640 1 -- 172.21.15.67:0/1477324929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc070f70 con 0x7feefc06b000 2024-02-26T10:40:28.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef3e00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 0x7feefc0c4bc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feee400c470 tx=0x7feee4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40052d0 con 0x7feefc06b000 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4005470 con 0x7feefc06b000 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4007260 con 0x7feefc06b000 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc149e60 con 0x7feefc06b000 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.936+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc1eb710 con 0x7feefc06b000 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.937+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc074a30 con 0x7feefc06b000 2024-02-26T10:40:28.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.939+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feee4007400 con 0x7feefc06b000 2024-02-26T10:40:28.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.939+0000 7feef2a00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feed806b500 0x7feed806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:28.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.940+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7feee4089850 con 0x7feefc06b000 2024-02-26T10:40:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.940+0000 7feefaa00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feed806b500 0x7feed806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.941+0000 7feefaa00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feed806b500 0x7feed806d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feeec036000 tx=0x7feeec03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:28.944+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feee4058590 con 0x7feefc06b000 2024-02-26T10:40:29.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.142+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feefc06b9b0 con 0x7feefc06b000 2024-02-26T10:40:29.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.145+0000 7feef2a00640 1 -- 172.21.15.67:0/1477324929 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16768 (secure 0 0 0) 0x7feee4057e90 con 0x7feefc06b000 2024-02-26T10:40:29.145 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:29.145 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":43,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:26.324828+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/155874470":"2024-02-27T10:40:22.920450+0000","172.21.15.82:6835/155874470":"2024-02-27T10:40:22.920450+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:29.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.147+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feed806b500 msgr2=0x7feed806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.147+0000 7feefbe00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feed806b500 0x7feed806d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feeec036000 tx=0x7feeec03b040 comp rx=0 tx=0).stop 2024-02-26T10:40:29.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 msgr2=0x7feefc0c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 0x7feefc0c4bc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feee400c470 tx=0x7feee4001e70 comp rx=0 tx=0).stop 2024-02-26T10:40:29.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 shutdown_connections 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feed806b500 0x7feed806d9c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feefc06db00 0x7feefc0c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feefc06b000 0x7feefc0c4bc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 --2- 172.21.15.67:0/1477324929 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feefc06ac20 0x7feefc071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 >> 172.21.15.67:0/1477324929 conn(0x7feefc0c63f0 msgr2=0x7feefc14ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 shutdown_connections 2024-02-26T10:40:29.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.148+0000 7feefbe00640 1 -- 172.21.15.67:0/1477324929 wait complete. 2024-02-26T10:40:29.168 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-02-26T10:40:29.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:29.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.375+0000 7fe99d600640 1 Processor -- start 2024-02-26T10:40:29.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.376+0000 7fe99d600640 1 -- start start 2024-02-26T10:40:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe99d600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9980bfcf0 0x7fe9980c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe99d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe998155a70 0x7fe9981583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe99d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe998154fd0 con 0x7fe9981588f0 2024-02-26T10:40:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe99d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe998073140 con 0x7fe998155a70 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe99d600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9981555e0 con 0x7fe9980bfcf0 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe996000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe998155a70 0x7fe9981583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe996000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe998155a70 0x7fe9981583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56690/0 (socket says 172.21.15.67:56690) 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe996000640 1 -- 172.21.15.67:0/526422432 learned_addr learned my addr 172.21.15.67:0/526422432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.377+0000 7fe997400640 1 --1- 172.21.15.67:0/526422432 >> v1:172.21.15.67:6789/0 conn(0x7fe9981588f0 0x7fe998072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56642/0 (socket says 172.21.15.67:56642) 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.378+0000 7fe996a00640 1 --2- 172.21.15.67:0/526422432 >> v2:172.21.15.82:3300/0 conn(0x7fe9980bfcf0 0x7fe9980c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.378+0000 7fe996000640 1 -- 172.21.15.67:0/526422432 >> v2:172.21.15.82:3300/0 conn(0x7fe9980bfcf0 msgr2=0x7fe9980c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.378+0000 7fe996000640 1 --2- 172.21.15.67:0/526422432 >> v2:172.21.15.82:3300/0 conn(0x7fe9980bfcf0 0x7fe9980c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.378+0000 7fe996000640 1 -- 172.21.15.67:0/526422432 >> v1:172.21.15.67:6789/0 conn(0x7fe9981588f0 legacy=0x7fe998072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.378+0000 7fe996000640 1 -- 172.21.15.67:0/526422432 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe998072e60 con 0x7fe998155a70 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.379+0000 7fe996000640 1 --2- 172.21.15.67:0/526422432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe9981583b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe9980c7940 tx=0x7fe98002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bcecc1c9c67fbd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.379+0000 7fe995600640 1 -- 172.21.15.67:0/526422432 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980034070 con 0x7fe998155a70 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.379+0000 7fe995600640 1 -- 172.21.15.67:0/526422432 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980030860 con 0x7fe998155a70 2024-02-26T10:40:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.379+0000 7fe995600640 1 -- 172.21.15.67:0/526422432 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980030c30 con 0x7fe998155a70 2024-02-26T10:40:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 -- 172.21.15.67:0/526422432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 msgr2=0x7fe9981583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 --2- 172.21.15.67:0/526422432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe9981583b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe9980c7940 tx=0x7fe98002b9a0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 -- 172.21.15.67:0/526422432 shutdown_connections 2024-02-26T10:40:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 --2- 172.21.15.67:0/526422432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe9981583b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 --2- 172.21.15.67:0/526422432 >> v2:172.21.15.82:3300/0 conn(0x7fe9980bfcf0 0x7fe9980c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 -- 172.21.15.67:0/526422432 >> 172.21.15.67:0/526422432 conn(0x7fe9980a8670 msgr2=0x7fe9980a8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 -- 172.21.15.67:0/526422432 shutdown_connections 2024-02-26T10:40:29.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.380+0000 7fe99d600640 1 -- 172.21.15.67:0/526422432 wait complete. 2024-02-26T10:40:29.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.382+0000 7fe99d600640 1 Processor -- start 2024-02-26T10:40:29.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.382+0000 7fe99d600640 1 -- start start 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe99d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9980bfcf0 0x7fe9980c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe99d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe99814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe99d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9981588f0 0x7fe99814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe99d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe998072a90 con 0x7fe998155a70 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe99d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c6590 con 0x7fe9981588f0 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe99d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe998072c10 con 0x7fe9980bfcf0 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe996000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe99814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe996000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe99814d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56694/0 (socket says 172.21.15.67:56694) 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.383+0000 7fe996000640 1 -- 172.21.15.67:0/3787255905 learned_addr learned my addr 172.21.15.67:0/3787255905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996a00640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9980bfcf0 0x7fe9980c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe997400640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9981588f0 0x7fe99814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996000640 1 -- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9980bfcf0 msgr2=0x7fe9980c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996000640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9980bfcf0 0x7fe9980c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996000640 1 -- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9981588f0 msgr2=0x7fe99814d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996000640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9981588f0 0x7fe99814d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996000640 1 -- 172.21.15.67:0/3787255905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99806f0d0 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe996000640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe99814d100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe98002b9e0 tx=0x7fe980004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.384+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980048070 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.385+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980030660 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.385+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980034040 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.385+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99806e000 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.385+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99806f650 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.386+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9980c57d0 con 0x7fe998155a70 2024-02-26T10:40:29.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.387+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe980037070 con 0x7fe998155a70 2024-02-26T10:40:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.388+0000 7fe98ea00640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe97406b500 0x7fe97406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.388+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7fe9800ac070 con 0x7fe998155a70 2024-02-26T10:40:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.388+0000 7fe996a00640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe97406b500 0x7fe97406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.390+0000 7fe996a00640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe97406b500 0x7fe97406d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe9880099c0 tx=0x7fe988003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:29.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.392+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe98007ae40 con 0x7fe998155a70 2024-02-26T10:40:29.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.630+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9980c0c10 con 0x7fe998155a70 2024-02-26T10:40:29.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.631+0000 7fe98ea00640 1 -- 172.21.15.67:0/3787255905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4931 (secure 0 0 0) 0x7fe98007a740 con 0x7fe998155a70 2024-02-26T10:40:29.632 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:29.632 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/224163452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":224163452},{"type":"v1","addr":"172.21.15.82:6837","nonce":224163452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:29.632 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 24 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe97406b500 msgr2=0x7fe97406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe97406b500 0x7fe97406d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe9880099c0 tx=0x7fe988003990 comp rx=0 tx=0).stop 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 msgr2=0x7fe99814d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe99814d100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe98002b9e0 tx=0x7fe980004730 comp rx=0 tx=0).stop 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 shutdown_connections 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe97406b500 0x7fe97406d9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9981588f0 0x7fe99814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe998155a70 0x7fe99814d100 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 --2- 172.21.15.67:0/3787255905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9980bfcf0 0x7fe9980c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 >> 172.21.15.67:0/3787255905 conn(0x7fe9980a8670 msgr2=0x7fe9980a6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 shutdown_connections 2024-02-26T10:40:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.634+0000 7fe99d600640 1 -- 172.21.15.67:0/3787255905 wait complete. 2024-02-26T10:40:29.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:29.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.872+0000 7f0c60a00640 1 Processor -- start 2024-02-26T10:40:29.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.872+0000 7f0c60a00640 1 -- start start 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c60a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06ac20 0x7f0c5c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c60a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c5c06db00 0x7f0c5c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c60a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c06b460 con 0x7f0c5c06b000 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c60a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c06dfb0 con 0x7f0c5c06db00 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c60a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0a9a80 con 0x7f0c5c06ac20 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c5be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c5c06db00 0x7f0c5c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c5aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0c5c06b000 0x7f0c5c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56646/0 (socket says 172.21.15.67:56646) 2024-02-26T10:40:29.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c5aa00640 1 -- 172.21.15.67:0/3624111112 learned_addr learned my addr 172.21.15.67:0/3624111112 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:29.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.874+0000 7f0c5b400640 1 --2- 172.21.15.67:0/3624111112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06ac20 0x7f0c5c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.875+0000 7f0c5be00640 1 -- 172.21.15.67:0/3624111112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06ac20 msgr2=0x7f0c5c0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:29.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.875+0000 7f0c5be00640 1 --2- 172.21.15.67:0/3624111112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06ac20 0x7f0c5c0a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.875+0000 7f0c5be00640 1 -- 172.21.15.67:0/3624111112 >> v1:172.21.15.67:6789/0 conn(0x7f0c5c06b000 legacy=0x7f0c5c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.875+0000 7f0c5a000640 1 -- 172.21.15.67:0/3624111112 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177012392 0 0) 0x7f0c5c06b460 con 0x7f0c5c06b000 2024-02-26T10:40:29.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.875+0000 7f0c5be00640 1 -- 172.21.15.67:0/3624111112 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0a9d60 con 0x7f0c5c06db00 2024-02-26T10:40:29.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.876+0000 7f0c5be00640 1 --2- 172.21.15.67:0/3624111112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06db00 0x7f0c5c0abf50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0c540097a0 tx=0x7f0c5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bfc7b294a380f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:29.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.876+0000 7f0c5a000640 1 -- 172.21.15.67:0/3624111112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54035020 con 0x7f0c5c06db00 2024-02-26T10:40:29.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.876+0000 7f0c5a000640 1 -- 172.21.15.67:0/3624111112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54031830 con 0x7f0c5c06db00 2024-02-26T10:40:29.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.876+0000 7f0c5a000640 1 -- 172.21.15.67:0/3624111112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54031c00 con 0x7f0c5c06db00 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 -- 172.21.15.67:0/3624111112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06db00 msgr2=0x7f0c5c0abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 --2- 172.21.15.67:0/3624111112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06db00 0x7f0c5c0abf50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0c540097a0 tx=0x7f0c5402b8d0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 -- 172.21.15.67:0/3624111112 shutdown_connections 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 --2- 172.21.15.67:0/3624111112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06db00 0x7f0c5c0abf50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 --2- 172.21.15.67:0/3624111112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06ac20 0x7f0c5c0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 -- 172.21.15.67:0/3624111112 >> 172.21.15.67:0/3624111112 conn(0x7f0c5c0c63f0 msgr2=0x7f0c5c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:29.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 -- 172.21.15.67:0/3624111112 shutdown_connections 2024-02-26T10:40:29.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.877+0000 7f0c60a00640 1 -- 172.21.15.67:0/3624111112 wait complete. 2024-02-26T10:40:29.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.879+0000 7f0c60a00640 1 Processor -- start 2024-02-26T10:40:29.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.879+0000 7f0c60a00640 1 -- start start 2024-02-26T10:40:29.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c60a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 0x7f0c5c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c60a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c5c06b000 0x7f0c5c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c60a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06db00 0x7f0c5c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c60a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0ac490 con 0x7f0c5c06ac20 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c60a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0aba30 con 0x7f0c5c06b000 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c60a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c150410 con 0x7f0c5c06db00 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c5b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 0x7f0c5c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c5b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 0x7f0c5c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56708/0 (socket says 172.21.15.67:56708) 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c5b400640 1 -- 172.21.15.67:0/352451238 learned_addr learned my addr 172.21.15.67:0/352451238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c5c06b000 0x7f0c5c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c5b400640 1 -- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06db00 msgr2=0x7f0c5c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.880+0000 7f0c5b400640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06db00 0x7f0c5c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c5b400640 1 -- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c5c06b000 msgr2=0x7f0c5c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c5b400640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c5c06b000 0x7f0c5c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c5b400640 1 -- 172.21.15.67:0/352451238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c1ee130 con 0x7f0c5c06ac20 2024-02-26T10:40:29.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c5c06b000 0x7f0c5c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c5b400640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 0x7f0c5c0c52a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c007ef0 tx=0x7f0c4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c014620 con 0x7f0c5c06ac20 2024-02-26T10:40:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c019070 con 0x7f0c5c06ac20 2024-02-26T10:40:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c1ee410 con 0x7f0c5c06ac20 2024-02-26T10:40:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c014a00 con 0x7f0c5c06ac20 2024-02-26T10:40:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.881+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c1ee9d0 con 0x7f0c5c06ac20 2024-02-26T10:40:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.883+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c074a30 con 0x7f0c5c06ac20 2024-02-26T10:40:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.883+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c4c014ba0 con 0x7f0c5c06ac20 2024-02-26T10:40:29.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.884+0000 7f0c53400640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c3806b500 0x7f0c3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:29.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.884+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f0c4c08acf0 con 0x7f0c5c06ac20 2024-02-26T10:40:29.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.885+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c3806b500 0x7f0c3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:29.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.886+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c3806b500 0x7f0c3806d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c153cf0 tx=0x7f0c44009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:29.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:29.888+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c4c059a30 con 0x7f0c5c06ac20 2024-02-26T10:40:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.126+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c5c0c6cc0 con 0x7f0c5c06ac20 2024-02-26T10:40:30.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.127+0000 7f0c53400640 1 -- 172.21.15.67:0/352451238 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f0c4c059330 con 0x7f0c5c06ac20 2024-02-26T10:40:30.127 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:30.127 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/224163452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":224163452},{"type":"v1","addr":"172.21.15.82:6837","nonce":224163452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:39:55.613510+0000","modified":"2024-02-26T10:39:58.656811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.67:6833/41192939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":41192939},{"type":"v1","addr":"172.21.15.67:6833","nonce":41192939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-02-26T10:40:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 24 2024-02-26T10:40:30.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c3806b500 msgr2=0x7f0c3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c3806b500 0x7f0c3806d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c153cf0 tx=0x7f0c44009040 comp rx=0 tx=0).stop 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 msgr2=0x7f0c5c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 0x7f0c5c0c52a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c007ef0 tx=0x7f0c4c00e4b0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 shutdown_connections 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c5c06db00 0x7f0c5c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c3806b500 0x7f0c3806d9c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c5c06b000 0x7f0c5c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 --2- 172.21.15.67:0/352451238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5c06ac20 0x7f0c5c0c52a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.130+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 >> 172.21.15.67:0/352451238 conn(0x7f0c5c0c63f0 msgr2=0x7f0c5c14de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.131+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 shutdown_connections 2024-02-26T10:40:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.131+0000 7f0c60a00640 1 -- 172.21.15.67:0/352451238 wait complete. 2024-02-26T10:40:30.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:30.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:40:30.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:40:30.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.364+0000 7f5c97800640 1 Processor -- start 2024-02-26T10:40:30.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.364+0000 7f5c97800640 1 -- start start 2024-02-26T10:40:30.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.365+0000 7f5c97800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5c90152420 0x7f5c900c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.365+0000 7f5c97800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c9015f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.365+0000 7f5c97800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c900c15e0 con 0x7f5c900bfa30 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.365+0000 7f5c97800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9006e660 con 0x7f5c90152420 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.365+0000 7f5c97800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9006e920 con 0x7f5c900c35d0 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5c90152420 0x7f5c900c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5c90152420 0x7f5c900c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56722/0 (socket says 172.21.15.67:56722) 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 -- 172.21.15.67:0/503877387 learned_addr learned my addr 172.21.15.67:0/503877387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c95800640 1 --2- 172.21.15.67:0/503877387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c9015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 -- 172.21.15.67:0/503877387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 msgr2=0x7f5c9015f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 --2- 172.21.15.67:0/503877387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c9015f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 -- 172.21.15.67:0/503877387 >> v1:172.21.15.67:6789/0 conn(0x7f5c900bfa30 legacy=0x7f5c90158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.366+0000 7f5c94e00640 1 -- 172.21.15.67:0/503877387 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900c1880 con 0x7f5c90152420 2024-02-26T10:40:30.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.368+0000 7f5c94e00640 1 --2- 172.21.15.67:0/503877387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c900c3090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009870 tx=0x7f5c8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9febbd83dc5341f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.369+0000 7f5c8f400640 1 -- 172.21.15.67:0/503877387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84035020 con 0x7f5c90152420 2024-02-26T10:40:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.369+0000 7f5c8f400640 1 -- 172.21.15.67:0/503877387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c84031640 con 0x7f5c90152420 2024-02-26T10:40:30.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.369+0000 7f5c8f400640 1 -- 172.21.15.67:0/503877387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8402d040 con 0x7f5c90152420 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 -- 172.21.15.67:0/503877387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 msgr2=0x7f5c900c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 --2- 172.21.15.67:0/503877387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c900c3090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009870 tx=0x7f5c8402b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 -- 172.21.15.67:0/503877387 shutdown_connections 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 --2- 172.21.15.67:0/503877387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c9015f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 --2- 172.21.15.67:0/503877387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c900c3090 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 -- 172.21.15.67:0/503877387 >> 172.21.15.67:0/503877387 conn(0x7f5c900a84a0 msgr2=0x7f5c900a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 -- 172.21.15.67:0/503877387 shutdown_connections 2024-02-26T10:40:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.370+0000 7f5c97800640 1 -- 172.21.15.67:0/503877387 wait complete. 2024-02-26T10:40:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.371+0000 7f5c97800640 1 Processor -- start 2024-02-26T10:40:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 -- start start 2024-02-26T10:40:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c900bfa30 0x7f5c901eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c901eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c90072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90152800 con 0x7f5c90152420 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90158400 con 0x7f5c900bfa30 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.372+0000 7f5c97800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90154e10 con 0x7f5c900c35d0 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c90072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c90072260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56728/0 (socket says 172.21.15.67:56728) 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 -- 172.21.15.67:0/3377776687 learned_addr learned my addr 172.21.15.67:0/3377776687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c94e00640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c900bfa30 0x7f5c901eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c8fe00640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c901eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 -- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 msgr2=0x7f5c901eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c901eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 -- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c900bfa30 msgr2=0x7f5c901eb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c900bfa30 0x7f5c901eb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c95800640 1 -- 172.21.15.67:0/3377776687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900aa580 con 0x7f5c90152420 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.373+0000 7f5c8fe00640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c901eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.374+0000 7f5c95800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c90072260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009d30 tx=0x7f5c8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.374+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88018020 con 0x7f5c90152420 2024-02-26T10:40:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.374+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c88014480 con 0x7f5c90152420 2024-02-26T10:40:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.374+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c90157390 con 0x7f5c90152420 2024-02-26T10:40:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.374+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88014870 con 0x7f5c90152420 2024-02-26T10:40:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.374+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c901ee5a0 con 0x7f5c90152420 2024-02-26T10:40:30.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.375+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c90152a70 con 0x7f5c90152420 2024-02-26T10:40:30.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.376+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c88005d10 con 0x7f5c90152420 2024-02-26T10:40:30.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.377+0000 7f5c8cc00640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5c7006b4d0 0x7f5c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.377+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f5c8808ac40 con 0x7f5c90152420 2024-02-26T10:40:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.378+0000 7f5c94e00640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5c7006b4d0 0x7f5c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.379+0000 7f5c94e00640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5c7006b4d0 0x7f5c7006d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5c84036000 tx=0x7f5c8403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:30.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.386+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c90152a70 con 0x7f5c90152420 2024-02-26T10:40:30.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.588+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c90074a30 con 0x7f5c90152420 2024-02-26T10:40:30.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.590+0000 7f5c8cc00640 1 -- 172.21.15.67:0/3377776687 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16768 (secure 0 0 0) 0x7f5c88022070 con 0x7f5c90152420 2024-02-26T10:40:30.590 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:30.590 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":43,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:26.324828+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:39:53.146836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:39:54.623593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/155874470":"2024-02-27T10:40:22.920450+0000","172.21.15.82:6835/155874470":"2024-02-27T10:40:22.920450+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5c7006b4d0 msgr2=0x7f5c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5c7006b4d0 0x7f5c7006d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5c84036000 tx=0x7f5c8403c040 comp rx=0 tx=0).stop 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 msgr2=0x7f5c90072260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c90072260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009d30 tx=0x7f5c8800c490 comp rx=0 tx=0).stop 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 shutdown_connections 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c90152420 0x7f5c90072260 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c900c35d0 0x7f5c901eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5c7006b4d0 0x7f5c7006d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 --2- 172.21.15.67:0/3377776687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c900bfa30 0x7f5c901eb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 >> 172.21.15.67:0/3377776687 conn(0x7f5c900a84a0 msgr2=0x7f5c90150630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.594+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 shutdown_connections 2024-02-26T10:40:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.595+0000 7f5c97800640 1 -- 172.21.15.67:0/3377776687 wait complete. 2024-02-26T10:40:30.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-02-26T10:40:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.832+0000 7fb2d8e00640 1 Processor -- start 2024-02-26T10:40:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.832+0000 7fb2d8e00640 1 -- start start 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2d8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2d4072fe0 0x7fb2d406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2d8e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2d4159210 0x7fb2d4159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2d8e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d415c8a0 con 0x7fb2d4156810 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2d8e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d415cb80 con 0x7fb2d4072fe0 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2d8e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d415ce60 con 0x7fb2d4159210 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2cbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2d4072fe0 0x7fb2d406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2cbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2d4072fe0 0x7fb2d406e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56734/0 (socket says 172.21.15.67:56734) 2024-02-26T10:40:30.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.834+0000 7fb2cbe00640 1 -- 172.21.15.67:0/1945000557 learned_addr learned my addr 172.21.15.67:0/1945000557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:30.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.835+0000 7fb2cbe00640 1 -- 172.21.15.67:0/1945000557 >> v2:172.21.15.82:3300/0 conn(0x7fb2d4159210 msgr2=0x7fb2d4159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:30.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.835+0000 7fb2cbe00640 1 --2- 172.21.15.67:0/1945000557 >> v2:172.21.15.82:3300/0 conn(0x7fb2d4159210 0x7fb2d4159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.835+0000 7fb2cbe00640 1 -- 172.21.15.67:0/1945000557 >> v1:172.21.15.82:6789/0 conn(0x7fb2d4156810 legacy=0x7fb2d40c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:30.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.835+0000 7fb2cbe00640 1 -- 172.21.15.67:0/1945000557 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d415d140 con 0x7fb2d4072fe0 2024-02-26T10:40:30.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.836+0000 7fb2cbe00640 1 --2- 172.21.15.67:0/1945000557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4072fe0 0x7fb2d406e660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb2d406e410 tx=0x7fb2bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a0e8188347f65e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.836+0000 7fb2d1400640 1 -- 172.21.15.67:0/1945000557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc02e070 con 0x7fb2d4072fe0 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.836+0000 7fb2d1400640 1 -- 172.21.15.67:0/1945000557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc037070 con 0x7fb2d4072fe0 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.836+0000 7fb2d1400640 1 -- 172.21.15.67:0/1945000557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc031a00 con 0x7fb2d4072fe0 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 -- 172.21.15.67:0/1945000557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4072fe0 msgr2=0x7fb2d406e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/1945000557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4072fe0 0x7fb2d406e660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb2d406e410 tx=0x7fb2bc02b880 comp rx=0 tx=0).stop 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 -- 172.21.15.67:0/1945000557 shutdown_connections 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/1945000557 >> v2:172.21.15.82:3300/0 conn(0x7fb2d4159210 0x7fb2d4159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/1945000557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4072fe0 0x7fb2d406e660 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 -- 172.21.15.67:0/1945000557 >> 172.21.15.67:0/1945000557 conn(0x7fb2d40a84a0 msgr2=0x7fb2d40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 -- 172.21.15.67:0/1945000557 shutdown_connections 2024-02-26T10:40:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.837+0000 7fb2d8e00640 1 -- 172.21.15.67:0/1945000557 wait complete. 2024-02-26T10:40:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.838+0000 7fb2d8e00640 1 Processor -- start 2024-02-26T10:40:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.839+0000 7fb2d8e00640 1 -- start start 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.839+0000 7fb2d8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d4072fe0 0x7fb2d414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.839+0000 7fb2d8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 0x7fb2d41f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.839+0000 7fb2d8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d4159210 0x7fb2d41f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2d8e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d406ec40 con 0x7fb2d4156810 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2d8e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d415d660 con 0x7fb2d4072fe0 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2d8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4159ba0 con 0x7fb2d4159210 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 0x7fb2d41f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 0x7fb2d41f2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56742/0 (socket says 172.21.15.67:56742) 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 -- 172.21.15.67:0/4263723277 learned_addr learned my addr 172.21.15.67:0/4263723277 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2d1e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d4072fe0 0x7fb2d414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 -- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d4159210 msgr2=0x7fb2d41f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2d2800640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d4159210 0x7fb2d41f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d4159210 0x7fb2d41f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 -- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d4072fe0 msgr2=0x7fb2d414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d4072fe0 0x7fb2d414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2cbe00640 1 -- 172.21.15.67:0/4263723277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4156000 con 0x7fb2d4156810 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.840+0000 7fb2d1e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d4072fe0 0x7fb2d414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.841+0000 7fb2cbe00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 0x7fb2d41f2480 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc000c00 tx=0x7fb2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.841+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc032880 con 0x7fb2d4156810 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.841+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc032a20 con 0x7fb2d4156810 2024-02-26T10:40:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.841+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc032e10 con 0x7fb2d4156810 2024-02-26T10:40:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.841+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d40c41f0 con 0x7fb2d4156810 2024-02-26T10:40:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.841+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d40c4490 con 0x7fb2d4156810 2024-02-26T10:40:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.842+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d40c3f60 con 0x7fb2d4156810 2024-02-26T10:40:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.843+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2bc0453f0 con 0x7fb2d4156810 2024-02-26T10:40:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.844+0000 7fb2ca000640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2b006b4d0 0x7fb2b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.844+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5857+0+0 (secure 0 0 0) 0x7fb2bc0ac410 con 0x7fb2d4156810 2024-02-26T10:40:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.845+0000 7fb2d1e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2b006b4d0 0x7fb2b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.846+0000 7fb2d1e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2b006b4d0 0x7fb2b006d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40a95c0 tx=0x7fb2c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:30.849+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2d40c3f60 con 0x7fb2d4156810 2024-02-26T10:40:31.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.089+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7fb2d4074a30 con 0x7fb2d4156810 2024-02-26T10:40:31.090 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:40:31.090+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:40:31.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.094+0000 7fb2ca000640 1 -- 172.21.15.67:0/4263723277 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 163+0+0 (secure 0 0 0) 0x7fb2bc040070 con 0x7fb2d4156810 2024-02-26T10:40:31.094 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.096+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2b006b4d0 msgr2=0x7fb2b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.096+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2b006b4d0 0x7fb2b006d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40a95c0 tx=0x7fb2c4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.096+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 msgr2=0x7fb2d41f2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.096+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 0x7fb2d41f2480 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc000c00 tx=0x7fb2bc0046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 shutdown_connections 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d4159210 0x7fb2d41f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2d4156810 0x7fb2d41f2480 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb2b006b4d0 0x7fb2b006d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 --2- 172.21.15.67:0/4263723277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d4072fe0 0x7fb2d414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 >> 172.21.15.67:0/4263723277 conn(0x7fb2d40a84a0 msgr2=0x7fb2d40c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 shutdown_connections 2024-02-26T10:40:31.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.097+0000 7fb2d8e00640 1 -- 172.21.15.67:0/4263723277 wait complete. 2024-02-26T10:40:31.108 INFO:tasks.ceph.mds.c.smithi067.stderr: -2467> 2024-02-26T10:39:57.823+0000 7f5b3a200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:40:31.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-02-26T10:40:31.174 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:40:31.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.312+0000 7f9545000640 1 Processor -- start 2024-02-26T10:40:31.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.312+0000 7f9545000640 1 -- start start 2024-02-26T10:40:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.313+0000 7f9545000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f954013c3e0 0x7f9540154310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.313+0000 7f9545000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95400aacd0 0x7f9540153730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.313+0000 7f9545000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9540156410 0x7f954006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.313+0000 7f9545000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95401532a0 con 0x7f954013c3e0 2024-02-26T10:40:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.313+0000 7f9545000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9540157340 con 0x7f9540156410 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.313+0000 7f9545000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9540156d80 con 0x7f95400aacd0 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f954013c3e0 0x7f9540154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f954013c3e0 0x7f9540154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56756/0 (socket says 172.21.15.67:56756) 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 -- 172.21.15.67:0/117592295 learned_addr learned my addr 172.21.15.67:0/117592295 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f9537e00640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95400aacd0 0x7f9540153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 -- 172.21.15.67:0/117592295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95400aacd0 msgr2=0x7f9540153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953e800640 1 --2- 172.21.15.67:0/117592295 >> v2:172.21.15.82:3300/0 conn(0x7f9540156410 0x7f954006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95400aacd0 0x7f9540153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 -- 172.21.15.67:0/117592295 >> v2:172.21.15.82:3300/0 conn(0x7f9540156410 msgr2=0x7f954006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 --2- 172.21.15.67:0/117592295 >> v2:172.21.15.82:3300/0 conn(0x7f9540156410 0x7f954006e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953de00640 1 -- 172.21.15.67:0/117592295 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540157060 con 0x7f954013c3e0 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f9537e00640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95400aacd0 0x7f9540153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:31.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.314+0000 7f953e800640 1 --2- 172.21.15.67:0/117592295 >> v2:172.21.15.82:3300/0 conn(0x7f9540156410 0x7f954006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.315+0000 7f953de00640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954013c3e0 0x7f9540154310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f954006efb0 tx=0x7f953002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a69aa1ab4d0893 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.316+0000 7f953d400640 1 -- 172.21.15.67:0/117592295 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300317d0 con 0x7f954013c3e0 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.316+0000 7f953d400640 1 -- 172.21.15.67:0/117592295 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9530037070 con 0x7f954013c3e0 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.316+0000 7f953d400640 1 -- 172.21.15.67:0/117592295 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530031b90 con 0x7f954013c3e0 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.316+0000 7f9545000640 1 -- 172.21.15.67:0/117592295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954013c3e0 msgr2=0x7f9540154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.316+0000 7f9545000640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954013c3e0 0x7f9540154310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f954006efb0 tx=0x7f953002b870 comp rx=0 tx=0).stop 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 -- 172.21.15.67:0/117592295 shutdown_connections 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 --2- 172.21.15.67:0/117592295 >> v2:172.21.15.82:3300/0 conn(0x7f9540156410 0x7f954006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95400aacd0 0x7f9540153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 --2- 172.21.15.67:0/117592295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954013c3e0 0x7f9540154310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 -- 172.21.15.67:0/117592295 >> 172.21.15.67:0/117592295 conn(0x7f95400a84a0 msgr2=0x7f95400a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 -- 172.21.15.67:0/117592295 shutdown_connections 2024-02-26T10:40:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.317+0000 7f9545000640 1 -- 172.21.15.67:0/117592295 wait complete. 2024-02-26T10:40:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.318+0000 7f9545000640 1 Processor -- start 2024-02-26T10:40:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 -- start start 2024-02-26T10:40:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 0x7f95400c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f954013c3e0 0x7f95400c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9540156410 0x7f95401eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540153df0 con 0x7f95400aacd0 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540153c70 con 0x7f954013c3e0 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.319+0000 7f9545000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95401568c0 con 0x7f9540156410 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 0x7f95400c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 0x7f95400c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56762/0 (socket says 172.21.15.67:56762) 2024-02-26T10:40:31.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 -- 172.21.15.67:0/2519216903 learned_addr learned my addr 172.21.15.67:0/2519216903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f9537e00640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f954013c3e0 0x7f95400c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953e800640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9540156410 0x7f95401eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 -- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9540156410 msgr2=0x7f95401eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:31.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9540156410 0x7f95401eaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 -- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f954013c3e0 msgr2=0x7f95400c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f954013c3e0 0x7f95400c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.320+0000 7f953de00640 1 -- 172.21.15.67:0/2519216903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954014d870 con 0x7f95400aacd0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.321+0000 7f953de00640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 0x7f95400c3ba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9530000c00 tx=0x7f95300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.321+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530032870 con 0x7f95400aacd0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.321+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9530032a10 con 0x7f95400aacd0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.321+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530032e00 con 0x7f95400aacd0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.321+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95401ee7c0 con 0x7f95400aacd0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.321+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95401ea720 con 0x7f95400aacd0 2024-02-26T10:40:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.322+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95400a8ac0 con 0x7f95400aacd0 2024-02-26T10:40:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.323+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95300453f0 con 0x7f95400aacd0 2024-02-26T10:40:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.324+0000 7f9536000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f951c06b4d0 0x7f951c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.324+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f95300ac680 con 0x7f95400aacd0 2024-02-26T10:40:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.325+0000 7f9537e00640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f951c06b4d0 0x7f951c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:31.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.326+0000 7f9537e00640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f951c06b4d0 0x7f951c06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9528006fd0 tx=0x7f9528008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.329+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95400a8ac0 con 0x7f95400aacd0 2024-02-26T10:40:31.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:31.569+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9540074a30 con 0x7f95400aacd0 2024-02-26T10:40:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.097+0000 7f9536000640 1 -- 172.21.15.67:0/2519216903 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 105+0+0 (secure 0 0 0) 0x7f9530040070 con 0x7f95400aacd0 2024-02-26T10:40:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.100+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f951c06b4d0 msgr2=0x7f951c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.100+0000 7f9545000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f951c06b4d0 0x7f951c06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9528006fd0 tx=0x7f9528008040 comp rx=0 tx=0).stop 2024-02-26T10:40:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.100+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 msgr2=0x7f95400c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.100+0000 7f9545000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 0x7f95400c3ba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9530000c00 tx=0x7f95300046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 shutdown_connections 2024-02-26T10:40:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9540156410 0x7f95401eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f951c06b4d0 0x7f951c06d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f954013c3e0 0x7f95400c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 --2- 172.21.15.67:0/2519216903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95400aacd0 0x7f95400c3ba0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 >> 172.21.15.67:0/2519216903 conn(0x7f95400a84a0 msgr2=0x7f9540155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 shutdown_connections 2024-02-26T10:40:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.101+0000 7f9545000640 1 -- 172.21.15.67:0/2519216903 wait complete. 2024-02-26T10:40:32.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-02-26T10:40:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.351+0000 7fd266600640 1 Processor -- start 2024-02-26T10:40:32.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.351+0000 7fd266600640 1 -- start start 2024-02-26T10:40:32.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd266600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 0x7fd260153650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:32.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd266600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd26006dd80 0x7fd2600c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd266600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd260153b90 con 0x7fd2600c35d0 2024-02-26T10:40:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd266600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26014d060 con 0x7fd26006dd80 2024-02-26T10:40:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd266600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601562f0 con 0x7fd26014cc80 2024-02-26T10:40:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd25f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 0x7fd260153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd25f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 0x7fd260153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:40490/0 (socket says 172.21.15.67:40490) 2024-02-26T10:40:32.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd25f400640 1 -- 172.21.15.67:0/2524377673 learned_addr learned my addr 172.21.15.67:0/2524377673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:32.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.353+0000 7fd25ea00640 1 --2- 172.21.15.67:0/2524377673 >> v2:172.21.15.82:3300/0 conn(0x7fd26006dd80 0x7fd2600c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:32.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.354+0000 7fd25f400640 1 -- 172.21.15.67:0/2524377673 >> v2:172.21.15.82:3300/0 conn(0x7fd26006dd80 msgr2=0x7fd2600c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:32.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.354+0000 7fd25f400640 1 --2- 172.21.15.67:0/2524377673 >> v2:172.21.15.82:3300/0 conn(0x7fd26006dd80 0x7fd2600c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.354+0000 7fd25f400640 1 -- 172.21.15.67:0/2524377673 >> v1:172.21.15.82:6789/0 conn(0x7fd2600c35d0 legacy=0x7fd2600aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:32.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.354+0000 7fd25f400640 1 -- 172.21.15.67:0/2524377673 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2601565d0 con 0x7fd26014cc80 2024-02-26T10:40:32.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.354+0000 7fd25e000640 1 -- 172.21.15.67:0/2524377673 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320307950 0 0) 0x7fd258001490 con 0x7fd2600c35d0 2024-02-26T10:40:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.355+0000 7fd25f400640 1 --2- 172.21.15.67:0/2524377673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 0x7fd260153650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd250009870 tx=0x7fd25002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20bb9bc7772f067 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.356+0000 7fd25e000640 1 -- 172.21.15.67:0/2524377673 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500317c0 con 0x7fd26014cc80 2024-02-26T10:40:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.356+0000 7fd25e000640 1 -- 172.21.15.67:0/2524377673 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250037070 con 0x7fd26014cc80 2024-02-26T10:40:32.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.356+0000 7fd25e000640 1 -- 172.21.15.67:0/2524377673 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250031b80 con 0x7fd26014cc80 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 -- 172.21.15.67:0/2524377673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 msgr2=0x7fd260153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 --2- 172.21.15.67:0/2524377673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 0x7fd260153650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd250009870 tx=0x7fd25002b860 comp rx=0 tx=0).stop 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 -- 172.21.15.67:0/2524377673 shutdown_connections 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 --2- 172.21.15.67:0/2524377673 >> v2:172.21.15.82:3300/0 conn(0x7fd26006dd80 0x7fd2600c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 --2- 172.21.15.67:0/2524377673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd26014cc80 0x7fd260153650 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 -- 172.21.15.67:0/2524377673 >> 172.21.15.67:0/2524377673 conn(0x7fd2600a84a0 msgr2=0x7fd2600a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 -- 172.21.15.67:0/2524377673 shutdown_connections 2024-02-26T10:40:32.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.357+0000 7fd266600640 1 -- 172.21.15.67:0/2524377673 wait complete. 2024-02-26T10:40:32.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.358+0000 7fd266600640 1 Processor -- start 2024-02-26T10:40:32.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.359+0000 7fd266600640 1 -- start start 2024-02-26T10:40:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.359+0000 7fd266600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 0x7fd260147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.359+0000 7fd266600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2600c35d0 0x7fd260151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.359+0000 7fd266600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd26014cc80 0x7fd260152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:32.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.359+0000 7fd266600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600ab2d0 con 0x7fd26006dd80 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.359+0000 7fd266600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260156b90 con 0x7fd26014cc80 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd266600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600ab450 con 0x7fd2600c35d0 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 0x7fd260147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 0x7fd260147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56766/0 (socket says 172.21.15.67:56766) 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 -- 172.21.15.67:0/1368923516 learned_addr learned my addr 172.21.15.67:0/1368923516 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25ea00640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2600c35d0 0x7fd260151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25fe00640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd26014cc80 0x7fd260152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 -- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2600c35d0 msgr2=0x7fd260151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2600c35d0 0x7fd260151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 -- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd26014cc80 msgr2=0x7fd260152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd26014cc80 0x7fd260152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.360+0000 7fd25f400640 1 -- 172.21.15.67:0/1368923516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2601480e0 con 0x7fd26006dd80 2024-02-26T10:40:32.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.361+0000 7fd25f400640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 0x7fd260147570 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd2500090b0 tx=0x7fd2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.361+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250032650 con 0x7fd26006dd80 2024-02-26T10:40:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.361+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500327f0 con 0x7fd26006dd80 2024-02-26T10:40:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.361+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250032be0 con 0x7fd26006dd80 2024-02-26T10:40:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.361+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd260147e00 con 0x7fd26006dd80 2024-02-26T10:40:32.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.361+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2601e8d50 con 0x7fd26006dd80 2024-02-26T10:40:32.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.362+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2600ab7e0 con 0x7fd26006dd80 2024-02-26T10:40:32.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.364+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd25004c060 con 0x7fd26006dd80 2024-02-26T10:40:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.364+0000 7fd257400640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd24006b4d0 0x7fd24006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.365+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5943+0+0 (secure 0 0 0) 0x7fd2500abad0 con 0x7fd26006dd80 2024-02-26T10:40:32.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.366+0000 7fd25ea00640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd24006b4d0 0x7fd24006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:32.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.367+0000 7fd25ea00640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd24006b4d0 0x7fd24006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd2600c5580 tx=0x7fd24c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:32.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.371+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2600ab7e0 con 0x7fd26006dd80 2024-02-26T10:40:32.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:32.570+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd260074a30 con 0x7fd26006dd80 2024-02-26T10:40:33.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.393+0000 7fd257400640 1 -- 172.21.15.67:0/1368923516 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd250040070 con 0x7fd26006dd80 2024-02-26T10:40:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_metadata' removed 2024-02-26T10:40:33.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.398+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd24006b4d0 msgr2=0x7fd24006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:33.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.398+0000 7fd266600640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd24006b4d0 0x7fd24006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd2600c5580 tx=0x7fd24c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:33.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.398+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 msgr2=0x7fd260147570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:33.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.398+0000 7fd266600640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 0x7fd260147570 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd2500090b0 tx=0x7fd2500046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 shutdown_connections 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd24006b4d0 0x7fd24006d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd26014cc80 0x7fd260152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2600c35d0 0x7fd260151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 --2- 172.21.15.67:0/1368923516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd26006dd80 0x7fd260147570 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 >> 172.21.15.67:0/1368923516 conn(0x7fd2600a84a0 msgr2=0x7fd2600aabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 shutdown_connections 2024-02-26T10:40:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.400+0000 7fd266600640 1 -- 172.21.15.67:0/1368923516 wait complete. 2024-02-26T10:40:33.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-02-26T10:40:33.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.637+0000 7f3090400640 1 Processor -- start 2024-02-26T10:40:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.638+0000 7f3090400640 1 -- start start 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f3090400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 0x7f308806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f3090400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f308806e390 0x7f30881533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f3090400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30880a6eb0 con 0x7f30880c2950 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f3090400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30880a7190 con 0x7f308806e390 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f3090400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30880a7470 con 0x7f30881503d0 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f308e400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f30880c2950 0x7f3088156320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56664/0 (socket says 172.21.15.67:56664) 2024-02-26T10:40:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.639+0000 7f308e400640 1 -- 172.21.15.67:0/2739362553 learned_addr learned my addr 172.21.15.67:0/2739362553 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:33.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f308da00640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 0x7f308806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f308d000640 1 --2- 172.21.15.67:0/2739362553 >> v2:172.21.15.82:3300/0 conn(0x7f308806e390 0x7f30881533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407896318 0 0) 0x7f30880a6eb0 con 0x7f30880c2950 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3070003520 con 0x7f30880c2950 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608294387 0 0) 0x7f3070003520 con 0x7f30880c2950 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 msgr2=0x7f308806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 0x7f308806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 >> v2:172.21.15.82:3300/0 conn(0x7f308806e390 msgr2=0x7f30881533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 --2- 172.21.15.67:0/2739362553 >> v2:172.21.15.82:3300/0 conn(0x7f308806e390 0x7f30881533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30880a7750 con 0x7f30880c2950 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f30800019f0 con 0x7f30880c2950 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.640+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 >> v1:172.21.15.67:6789/0 conn(0x7f30880c2950 legacy=0x7f3088156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f3087e00640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 0x7f30700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f3087e00640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3070003f30 0x7f3070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f3087e00640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3070004d40 0x7f30700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30880a7190 con 0x7f3070004d40 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30880a7470 con 0x7f3070003f30 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307000baa0 con 0x7f30881503d0 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308e400640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3070004d40 0x7f30700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308d000640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3070003f30 0x7f3070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308da00640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 0x7f30700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308e400640 1 -- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 msgr2=0x7f30700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308e400640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30881503d0 0x7f30700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308e400640 1 -- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3070003f30 msgr2=0x7f3070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308e400640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3070003f30 0x7f3070004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.641+0000 7f308e400640 1 -- 172.21.15.67:0/2739362553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070003520 con 0x7f3070004d40 2024-02-26T10:40:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.642+0000 7f308d000640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3070003f30 0x7f3070004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.643+0000 7f308e400640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3070004d40 0x7f30700051c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f308000a880 tx=0x7f3070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cddc04931a3677 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.643+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000a4a0 con 0x7f3070004d40 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.643+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080034650 con 0x7f3070004d40 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.643+0000 7f3087e00640 1 -- 172.21.15.67:0/2739362553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080034a80 con 0x7f3070004d40 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.643+0000 7f3090400640 1 -- 172.21.15.67:0/2739362553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3070004d40 msgr2=0x7f30700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.643+0000 7f3090400640 1 --2- 172.21.15.67:0/2739362553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3070004d40 0x7f30700051c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f308000a880 tx=0x7f3070005830 comp rx=0 tx=0).stop 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.644+0000 7f308e400640 1 -- 172.21.15.67:0/2739362553 reap_dead start 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.644+0000 7f3090400640 1 -- 172.21.15.67:0/2739362553 shutdown_connections 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.644+0000 7f3090400640 1 -- 172.21.15.67:0/2739362553 >> 172.21.15.67:0/2739362553 conn(0x7f30880c6da0 msgr2=0x7f30880c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.644+0000 7f3090400640 1 -- 172.21.15.67:0/2739362553 shutdown_connections 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.644+0000 7f3090400640 1 -- 172.21.15.67:0/2739362553 wait complete. 2024-02-26T10:40:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.645+0000 7f3090400640 1 Processor -- start 2024-02-26T10:40:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.646+0000 7f3090400640 1 -- start start 2024-02-26T10:40:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f3090400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30880a9ce0 0x7f30880ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f3090400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30880aa0c0 0x7f308814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f3090400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 0x7f3088149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f3090400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881507b0 con 0x7f308814c2c0 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f3090400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308806ef00 con 0x7f30880aa0c0 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f3090400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30880c0ef0 con 0x7f30880a9ce0 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 0x7f3088149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 0x7f3088149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56788/0 (socket says 172.21.15.67:56788) 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 -- 172.21.15.67:0/1911473638 learned_addr learned my addr 172.21.15.67:0/1911473638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308da00640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30880a9ce0 0x7f30880ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308d000640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30880aa0c0 0x7f308814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 -- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30880a9ce0 msgr2=0x7f30880ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30880a9ce0 0x7f30880ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 -- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30880aa0c0 msgr2=0x7f308814bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30880aa0c0 0x7f308814bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308e400640 1 -- 172.21.15.67:0/1911473638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30880aa520 con 0x7f308814c2c0 2024-02-26T10:40:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308d000640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30880aa0c0 0x7f308814bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.647+0000 7f308da00640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30880a9ce0 0x7f30880ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.648+0000 7f308e400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 0x7f3088149400 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f308000a9a0 tx=0x7f3088152db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:33.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.648+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000a4a0 con 0x7f308814c2c0 2024-02-26T10:40:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.648+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080045070 con 0x7f308814c2c0 2024-02-26T10:40:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.648+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080001ff0 con 0x7f308814c2c0 2024-02-26T10:40:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.648+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30880ac2a0 con 0x7f308814c2c0 2024-02-26T10:40:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.648+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30880ac860 con 0x7f308814c2c0 2024-02-26T10:40:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.650+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3088074a30 con 0x7f308814c2c0 2024-02-26T10:40:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.650+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3080034c00 con 0x7f308814c2c0 2024-02-26T10:40:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.652+0000 7f3085600640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f306c06b500 0x7f306c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.652+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5503+0+0 (secure 0 0 0) 0x7f30800ac790 con 0x7f308814c2c0 2024-02-26T10:40:33.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.652+0000 7f308da00640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f306c06b500 0x7f306c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.654+0000 7f308da00640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f306c06b500 0x7f306c06d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f307c009a00 tx=0x7f307c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:33.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.656+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f308007b6d0 con 0x7f308814c2c0 2024-02-26T10:40:33.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:33.859+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3088157150 con 0x7f308814c2c0 2024-02-26T10:40:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.397+0000 7f3085600640 1 -- 172.21.15.67:0/1911473638 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v46) v1 ==== 172+0+0 (secure 0 0 0) 0x7f308007afd0 con 0x7f308814c2c0 2024-02-26T10:40:34.397 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_data' removed 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f306c06b500 msgr2=0x7f306c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f306c06b500 0x7f306c06d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f307c009a00 tx=0x7f307c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 msgr2=0x7f3088149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 0x7f3088149400 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f308000a9a0 tx=0x7f3088152db0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 shutdown_connections 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308814c2c0 0x7f3088149400 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f306c06b500 0x7f306c06d9c0 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f307c009a00 tx=0x7f307c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30880aa0c0 0x7f308814bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 --2- 172.21.15.67:0/1911473638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30880a9ce0 0x7f30880ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.404+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 >> 172.21.15.67:0/1911473638 conn(0x7f30880c6da0 msgr2=0x7f3088152470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:34.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.408+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 shutdown_connections 2024-02-26T10:40:34.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.408+0000 7f3090400640 1 -- 172.21.15.67:0/1911473638 wait complete. 2024-02-26T10:40:34.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-26T10:40:34.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.640+0000 7fbff4200640 1 Processor -- start 2024-02-26T10:40:34.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.641+0000 7fbff4200640 1 -- start start 2024-02-26T10:40:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.642+0000 7fbff4200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbfec154310 0x7fbfec06e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.642+0000 7fbff4200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec154780 con 0x7fbfec13c3e0 2024-02-26T10:40:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.642+0000 7fbff4200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec157340 con 0x7fbfec0aacd0 2024-02-26T10:40:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.642+0000 7fbff4200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec153e40 con 0x7fbfec154310 2024-02-26T10:40:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.642+0000 7fbff1800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbfec13c3e0 0x7fbfec153730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56674/0 (socket says 172.21.15.67:56674) 2024-02-26T10:40:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbff1800640 1 -- 172.21.15.67:0/3262601060 learned_addr learned my addr 172.21.15.67:0/3262601060 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbff2200640 1 --2- 172.21.15.67:0/3262601060 >> v2:172.21.15.82:3300/0 conn(0x7fbfec154310 0x7fbfec06e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069291867 0 0) 0x7fbfec154780 con 0x7fbfec13c3e0 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd4003520 con 0x7fbfec13c3e0 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 388220012 0 0) 0x7fbfd4003520 con 0x7fbfec13c3e0 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 >> v2:172.21.15.82:3300/0 conn(0x7fbfec154310 msgr2=0x7fbfec06e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 --2- 172.21.15.67:0/3262601060 >> v2:172.21.15.82:3300/0 conn(0x7fbfec154310 0x7fbfec06e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 >> v1:172.21.15.82:6789/0 conn(0x7fbfec0aacd0 legacy=0x7fbfec156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec156d80 con 0x7fbfec13c3e0 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fbfe0001670 con 0x7fbfec13c3e0 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.643+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 >> v1:172.21.15.67:6789/0 conn(0x7fbfec13c3e0 legacy=0x7fbfec153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbfebe00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfd40039b0 0x7fbfd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbfebe00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbfebe00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd40051c0 0x7fbfd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec153e40 con 0x7fbfd4004800 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd400c160 con 0x7fbfd40051c0 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd400c360 con 0x7fbfd40039b0 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff0e00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff1800640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfd40039b0 0x7fbfd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff0e00640 1 -- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfd40039b0 msgr2=0x7fbfd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff0e00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfd40039b0 0x7fbfd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff2200640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd40051c0 0x7fbfd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff0e00640 1 -- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd40051c0 msgr2=0x7fbfd400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff0e00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd40051c0 0x7fbfd400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff0e00640 1 -- 172.21.15.67:0/3262601060 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4003520 con 0x7fbfd4004800 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff2200640 1 -- 172.21.15.67:0/3262601060 reap_dead start 2024-02-26T10:40:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.644+0000 7fbff1800640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfd40039b0 0x7fbfd4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.646+0000 7fbff0e00640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfd4004c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8001720 tx=0x7fbfd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d69b721997febe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.646+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8001370 con 0x7fbfd4004800 2024-02-26T10:40:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.646+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80052d0 con 0x7fbfd4004800 2024-02-26T10:40:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.646+0000 7fbfebe00640 1 -- 172.21.15.67:0/3262601060 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8003830 con 0x7fbfd4004800 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.646+0000 7fbff4200640 1 -- 172.21.15.67:0/3262601060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 msgr2=0x7fbfd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.646+0000 7fbff4200640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfd4004c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8001720 tx=0x7fbfd4005830 comp rx=0 tx=0).stop 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.647+0000 7fbff4200640 1 -- 172.21.15.67:0/3262601060 shutdown_connections 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.647+0000 7fbff4200640 1 --2- 172.21.15.67:0/3262601060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfd4004c80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.647+0000 7fbff4200640 1 -- 172.21.15.67:0/3262601060 >> 172.21.15.67:0/3262601060 conn(0x7fbfec0a84a0 msgr2=0x7fbfec0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.647+0000 7fbff4200640 1 -- 172.21.15.67:0/3262601060 shutdown_connections 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.647+0000 7fbff4200640 1 -- 172.21.15.67:0/3262601060 wait complete. 2024-02-26T10:40:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.648+0000 7fbff4200640 1 Processor -- start 2024-02-26T10:40:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.648+0000 7fbff4200640 1 -- start start 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.649+0000 7fbff4200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfec1eb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.649+0000 7fbff4200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfec132ab0 0x7fbfec134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.649+0000 7fbff4200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfec13b2c0 0x7fbfec0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.649+0000 7fbff4200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec152f30 con 0x7fbfd4004800 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.649+0000 7fbff4200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1532a0 con 0x7fbfec13b2c0 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.649+0000 7fbff4200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec153730 con 0x7fbfec132ab0 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfec1eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfec1eb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56814/0 (socket says 172.21.15.67:56814) 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 -- 172.21.15.67:0/3487398111 learned_addr learned my addr 172.21.15.67:0/3487398111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 -- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfec132ab0 msgr2=0x7fbfec134970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfec132ab0 0x7fbfec134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 -- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfec13b2c0 msgr2=0x7fbfec0c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfec13b2c0 0x7fbfec0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 -- 172.21.15.67:0/3487398111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec1327b0 con 0x7fbfd4004800 2024-02-26T10:40:34.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.650+0000 7fbff1800640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfec1eb170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000d7a0 tx=0x7fbfe000dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:34.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.651+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0019070 con 0x7fbfd4004800 2024-02-26T10:40:34.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.651+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00072d0 con 0x7fbfd4004800 2024-02-26T10:40:34.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.651+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00047e0 con 0x7fbfd4004800 2024-02-26T10:40:34.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.651+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec1ea970 con 0x7fbfd4004800 2024-02-26T10:40:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.651+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec134eb0 con 0x7fbfd4004800 2024-02-26T10:40:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.652+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec074a30 con 0x7fbfd4004800 2024-02-26T10:40:34.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.653+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfe001c070 con 0x7fbfd4004800 2024-02-26T10:40:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.654+0000 7fbfe9600640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbfcc06b500 0x7fbfcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.654+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fbfe00897d0 con 0x7fbfd4004800 2024-02-26T10:40:34.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.655+0000 7fbff0e00640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbfcc06b500 0x7fbfcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:34.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.656+0000 7fbff0e00640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbfcc06b500 0x7fbfcc06d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8033000 tx=0x7fbfd802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:34.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.659+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfe00582a0 con 0x7fbfd4004800 2024-02-26T10:40:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.858+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbfec06e260 con 0x7fbfd4004800 2024-02-26T10:40:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.864+0000 7fbfe9600640 1 -- 172.21.15.67:0/3487398111 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbfe005c340 con 0x7fbfd4004800 2024-02-26T10:40:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbfcc06b500 msgr2=0x7fbfcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbfcc06b500 0x7fbfcc06d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8033000 tx=0x7fbfd802f000 comp rx=0 tx=0).stop 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 msgr2=0x7fbfec1eb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfec1eb170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000d7a0 tx=0x7fbfe000dba0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 shutdown_connections 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbfcc06b500 0x7fbfcc06d9c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfec13b2c0 0x7fbfec0c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfec132ab0 0x7fbfec134970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 --2- 172.21.15.67:0/3487398111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd4004800 0x7fbfec1eb170 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 >> 172.21.15.67:0/3487398111 conn(0x7fbfec0a84a0 msgr2=0x7fbfec0ab120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 shutdown_connections 2024-02-26T10:40:34.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:34.867+0000 7fbff4200640 1 -- 172.21.15.67:0/3487398111 wait complete. 2024-02-26T10:40:34.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent' 2024-02-26T10:40:35.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.109+0000 7fe371200640 1 Processor -- start 2024-02-26T10:40:35.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.109+0000 7fe371200640 1 -- start start 2024-02-26T10:40:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe371200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 0x7fe36c06af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe371200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe36c0ac830 0x7fe36c0acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe371200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c156780 con 0x7fe36c0705b0 2024-02-26T10:40:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe371200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c073210 con 0x7fe36c0ac830 2024-02-26T10:40:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe371200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c072c70 con 0x7fe36c14fff0 2024-02-26T10:40:35.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe36aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe36c0705b0 0x7fe36c156070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56678/0 (socket says 172.21.15.67:56678) 2024-02-26T10:40:35.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.111+0000 7fe36aa00640 1 -- 172.21.15.67:0/2858225214 learned_addr learned my addr 172.21.15.67:0/2858225214 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:35.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36b400640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 0x7fe36c06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275614317 0 0) 0x7fe36c156780 con 0x7fe36c0705b0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36be00640 1 --2- 172.21.15.67:0/2858225214 >> v2:172.21.15.82:3300/0 conn(0x7fe36c0ac830 0x7fe36c0acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe350003520 con 0x7fe36c0705b0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756551948 0 0) 0x7fe350003520 con 0x7fe36c0705b0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 msgr2=0x7fe36c06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 0x7fe36c06af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 >> v2:172.21.15.82:3300/0 conn(0x7fe36c0ac830 msgr2=0x7fe36c0acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 --2- 172.21.15.67:0/2858225214 >> v2:172.21.15.82:3300/0 conn(0x7fe36c0ac830 0x7fe36c0acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c06ef00 con 0x7fe36c0705b0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fe358001670 con 0x7fe36c0705b0 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36be00640 1 --2- 172.21.15.67:0/2858225214 >> v2:172.21.15.82:3300/0 conn(0x7fe36c0ac830 0x7fe36c0acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.112+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 >> v1:172.21.15.67:6789/0 conn(0x7fe36c0705b0 legacy=0x7fe36c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36a000640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3500039b0 0x7fe350003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36a000640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 0x7fe350004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36a000640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe350004d40 0x7fe3500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c073210 con 0x7fe350004d40 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c072c70 con 0x7fe3500039b0 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000b9c0 con 0x7fe36c14fff0 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36be00640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe350004d40 0x7fe3500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36b400640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3500039b0 0x7fe350003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36aa00640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 0x7fe350004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36be00640 1 -- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 msgr2=0x7fe350004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36be00640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c14fff0 0x7fe350004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36be00640 1 -- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3500039b0 msgr2=0x7fe350003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36be00640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3500039b0 0x7fe350003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.113+0000 7fe36be00640 1 -- 172.21.15.67:0/2858225214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350003520 con 0x7fe350004d40 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.114+0000 7fe36be00640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe350004d40 0x7fe3500051c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe364007410 tx=0x7fe350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4076913ece9a4049 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364039070 con 0x7fe350004d40 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36402cdc0 con 0x7fe350004d40 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe36a000640 1 -- 172.21.15.67:0/2858225214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364009c80 con 0x7fe350004d40 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe371200640 1 -- 172.21.15.67:0/2858225214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe350004d40 msgr2=0x7fe3500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe371200640 1 --2- 172.21.15.67:0/2858225214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe350004d40 0x7fe3500051c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe364007410 tx=0x7fe350005830 comp rx=0 tx=0).stop 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe36be00640 1 -- 172.21.15.67:0/2858225214 reap_dead start 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe371200640 1 -- 172.21.15.67:0/2858225214 shutdown_connections 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe371200640 1 -- 172.21.15.67:0/2858225214 >> 172.21.15.67:0/2858225214 conn(0x7fe36c06bd30 msgr2=0x7fe36c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.115+0000 7fe371200640 1 -- 172.21.15.67:0/2858225214 shutdown_connections 2024-02-26T10:40:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.116+0000 7fe371200640 1 -- 172.21.15.67:0/2858225214 wait complete. 2024-02-26T10:40:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.117+0000 7fe371200640 1 Processor -- start 2024-02-26T10:40:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.117+0000 7fe371200640 1 -- start start 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.118+0000 7fe371200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe350004d40 0x7fe36c1ec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.118+0000 7fe371200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 0x7fe36c0c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.118+0000 7fe371200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c1e95b0 0x7fe36c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.118+0000 7fe371200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1503d0 con 0x7fe36c155790 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.118+0000 7fe371200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c4dc0 con 0x7fe350004d40 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.118+0000 7fe371200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1ec600 con 0x7fe36c1e95b0 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 0x7fe36c0c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 0x7fe36c0c36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56826/0 (socket says 172.21.15.67:56826) 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 -- 172.21.15.67:0/2078727142 learned_addr learned my addr 172.21.15.67:0/2078727142 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36b400640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe350004d40 0x7fe36c1ec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36be00640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c1e95b0 0x7fe36c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 -- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c1e95b0 msgr2=0x7fe36c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c1e95b0 0x7fe36c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 -- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe350004d40 msgr2=0x7fe36c1ec0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe350004d40 0x7fe36c1ec0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 -- 172.21.15.67:0/2078727142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c155c90 con 0x7fe36c155790 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36be00640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c1e95b0 0x7fe36c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.119+0000 7fe36aa00640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 0x7fe36c0c36c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe35800d7c0 tx=0x7fe35800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.120+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358019070 con 0x7fe36c155790 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.120+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580074e0 con 0x7fe36c155790 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.120+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358004a00 con 0x7fe36c155790 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.120+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c155f70 con 0x7fe36c155790 2024-02-26T10:40:35.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.120+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c153c40 con 0x7fe36c155790 2024-02-26T10:40:35.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.121+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c074a30 con 0x7fe36c155790 2024-02-26T10:40:35.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.122+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe358004ba0 con 0x7fe36c155790 2024-02-26T10:40:35.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.123+0000 7fe363400640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe34c06b500 0x7fe34c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.123+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe3580899b0 con 0x7fe36c155790 2024-02-26T10:40:35.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.124+0000 7fe36b400640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe34c06b500 0x7fe34c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.125+0000 7fe36b400640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe34c06b500 0x7fe34c06d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009990 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.128+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe358058ac0 con 0x7fe36c155790 2024-02-26T10:40:35.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.405+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]} v 0) v1 -- 0x7fe36c153a20 con 0x7fe36c155790 2024-02-26T10:40:35.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.413+0000 7fe363400640 1 -- 172.21.15.67:0/2078727142 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]}]=0 v131) v1 ==== 159+0+0 (secure 0 0 0) 0x7fe3580583c0 con 0x7fe36c155790 2024-02-26T10:40:35.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe34c06b500 msgr2=0x7fe34c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe34c06b500 0x7fe34c06d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009990 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:35.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 msgr2=0x7fe36c0c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 0x7fe36c0c36c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe35800d7c0 tx=0x7fe35800dc80 comp rx=0 tx=0).stop 2024-02-26T10:40:35.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 shutdown_connections 2024-02-26T10:40:35.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe36c1e95b0 0x7fe36c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe36c155790 0x7fe36c0c36c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe34c06b500 0x7fe34c06d9c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 --2- 172.21.15.67:0/2078727142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe350004d40 0x7fe36c1ec0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 >> 172.21.15.67:0/2078727142 conn(0x7fe36c06bd30 msgr2=0x7fe36c14b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:35.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 shutdown_connections 2024-02-26T10:40:35.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.416+0000 7fe371200640 1 -- 172.21.15.67:0/2078727142 wait complete. 2024-02-26T10:40:35.435 INFO:tasks.cephfs_test_runner:test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) ... ok 2024-02-26T10:40:35.435 INFO:tasks.cephfs_test_runner:Starting test: test_mount_mon_and_osd_caps_present_mds_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2024-02-26T10:40:35.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:35.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.656+0000 7f44a3200640 1 Processor -- start 2024-02-26T10:40:35.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.656+0000 7f44a3200640 1 -- start start 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44a40c2310 0x7f44a4153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a4155f10 con 0x7f44a406b000 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a3200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a40c28d0 con 0x7f44a406ac20 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a406b430 con 0x7f44a40c2310 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a2800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f44a406b000 0x7f44a415c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56694/0 (socket says 172.21.15.67:56694) 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a2800640 1 -- 172.21.15.67:0/4119905962 learned_addr learned my addr 172.21.15.67:0/4119905962 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:35.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.658+0000 7f44a1400640 1 --2- 172.21.15.67:0/4119905962 >> v2:172.21.15.67:3300/0 conn(0x7f44a40c2310 0x7f44a4153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.659+0000 7f44a0a00640 1 -- 172.21.15.67:0/4119905962 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294029272 0 0) 0x7f44a4155f10 con 0x7f44a406b000 2024-02-26T10:40:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.659+0000 7f44a0a00640 1 -- 172.21.15.67:0/4119905962 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4488003520 con 0x7f44a406b000 2024-02-26T10:40:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.659+0000 7f44a1400640 1 -- 172.21.15.67:0/4119905962 >> v1:172.21.15.82:6789/0 conn(0x7f44a406ac20 legacy=0x7f44a4152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.659+0000 7f44a1400640 1 -- 172.21.15.67:0/4119905962 >> v1:172.21.15.67:6789/0 conn(0x7f44a406b000 legacy=0x7f44a415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.659+0000 7f44a1400640 1 -- 172.21.15.67:0/4119905962 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4155150 con 0x7f44a40c2310 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.660+0000 7f44a1400640 1 --2- 172.21.15.67:0/4119905962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a40c2310 0x7f44a4153630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f448c0097a0 tx=0x7f448c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d833f62115fb76cc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.660+0000 7f44a0a00640 1 -- 172.21.15.67:0/4119905962 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0315a0 con 0x7f44a40c2310 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.660+0000 7f44a0a00640 1 -- 172.21.15.67:0/4119905962 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c037070 con 0x7f44a40c2310 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.660+0000 7f44a0a00640 1 -- 172.21.15.67:0/4119905962 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c031960 con 0x7f44a40c2310 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 -- 172.21.15.67:0/4119905962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a40c2310 msgr2=0x7f44a4153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 --2- 172.21.15.67:0/4119905962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a40c2310 0x7f44a4153630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f448c0097a0 tx=0x7f448c02b840 comp rx=0 tx=0).stop 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 -- 172.21.15.67:0/4119905962 shutdown_connections 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 --2- 172.21.15.67:0/4119905962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a40c2310 0x7f44a4153630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 -- 172.21.15.67:0/4119905962 >> 172.21.15.67:0/4119905962 conn(0x7f44a40c63f0 msgr2=0x7f44a40c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 -- 172.21.15.67:0/4119905962 shutdown_connections 2024-02-26T10:40:35.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.661+0000 7f44a3200640 1 -- 172.21.15.67:0/4119905962 wait complete. 2024-02-26T10:40:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.662+0000 7f44a3200640 1 Processor -- start 2024-02-26T10:40:35.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 -- start start 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 0x7f44a40724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a406b000 0x7f44a4071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a40c2310 0x7f44a413dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41559f0 con 0x7f44a406ac20 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a4155710 con 0x7f44a406b000 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.663+0000 7f44a3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a4148850 con 0x7f44a40c2310 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 0x7f44a40724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 0x7f44a40724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56836/0 (socket says 172.21.15.67:56836) 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 -- 172.21.15.67:0/1161812426 learned_addr learned my addr 172.21.15.67:0/1161812426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:35.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1400640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a406b000 0x7f44a4071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a2800640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a40c2310 0x7f44a413dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 -- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a40c2310 msgr2=0x7f44a413dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a40c2310 0x7f44a413dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 -- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a406b000 msgr2=0x7f44a4071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a406b000 0x7f44a4071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1e00640 1 -- 172.21.15.67:0/1161812426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40c4d00 con 0x7f44a406ac20 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a2800640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a40c2310 0x7f44a413dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.664+0000 7f44a1400640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a406b000 0x7f44a4071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.665+0000 7f44a1e00640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 0x7f44a40724e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f449400c730 tx=0x7f449400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.665+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494016020 con 0x7f44a406ac20 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.665+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494006d50 con 0x7f44a406ac20 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.665+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494003bb0 con 0x7f44a406ac20 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.665+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a40c5a40 con 0x7f44a406ac20 2024-02-26T10:40:35.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.665+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a40c6000 con 0x7f44a406ac20 2024-02-26T10:40:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.666+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a4074a30 con 0x7f44a406ac20 2024-02-26T10:40:35.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.668+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44940050d0 con 0x7f44a406ac20 2024-02-26T10:40:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.668+0000 7f449a000640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f448006b4d0 0x7f448006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:35.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.668+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4494089da0 con 0x7f44a406ac20 2024-02-26T10:40:35.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.669+0000 7f44a1400640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f448006b4d0 0x7f448006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:35.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.670+0000 7f44a1400640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f448006b4d0 0x7f448006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f448c036000 tx=0x7f448c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:35.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.673+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4494020020 con 0x7f44a406ac20 2024-02-26T10:40:35.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.870+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44a40c20f0 con 0x7f44a406ac20 2024-02-26T10:40:35.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.871+0000 7f449a000640 1 -- 172.21.15.67:0/1161812426 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13072 (secure 0 0 0) 0x7f4494058eb0 con 0x7f44a406ac20 2024-02-26T10:40:35.871 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:35.871 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":46,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:34.394188+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6833/41192939":"2024-02-27T10:40:31.090888+0000","172.21.15.67:6832/41192939":"2024-02-27T10:40:31.090888+0000","172.21.15.82:6837/155874470":"2024-02-27T10:40:22.920450+0000","172.21.15.82:6835/155874470":"2024-02-27T10:40:22.920450+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:35.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f448006b4d0 msgr2=0x7f448006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f448006b4d0 0x7f448006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f448c036000 tx=0x7f448c008030 comp rx=0 tx=0).stop 2024-02-26T10:40:35.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 msgr2=0x7f44a40724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:35.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 0x7f44a40724e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f449400c730 tx=0x7f449400cbf0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 shutdown_connections 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44a40c2310 0x7f44a413dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f448006b4d0 0x7f448006d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44a406b000 0x7f44a4071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 --2- 172.21.15.67:0/1161812426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44a406ac20 0x7f44a40724e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 >> 172.21.15.67:0/1161812426 conn(0x7f44a40c63f0 msgr2=0x7f44a4144ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 shutdown_connections 2024-02-26T10:40:35.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:35.874+0000 7f44a3200640 1 -- 172.21.15.67:0/1161812426 wait complete. 2024-02-26T10:40:35.893 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:35.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.113+0000 7f3108a00640 1 Processor -- start 2024-02-26T10:40:36.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.114+0000 7f3108a00640 1 -- start start 2024-02-26T10:40:36.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.115+0000 7f3108a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f31040ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.115+0000 7f3108a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31041547c0 con 0x7f31040ab3f0 2024-02-26T10:40:36.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.115+0000 7f3108a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3104154aa0 con 0x7f31040abca0 2024-02-26T10:40:36.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.115+0000 7f3108a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3104154d80 con 0x7f31040c1490 2024-02-26T10:40:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3102800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f31040ab3f0 0x7f310406da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56702/0 (socket says 172.21.15.67:56702) 2024-02-26T10:40:36.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3102800640 1 -- 172.21.15.67:0/1860963170 learned_addr learned my addr 172.21.15.67:0/1860963170 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3103200640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f31040ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886792358 0 0) 0x7f31041547c0 con 0x7f31040ab3f0 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e8004630 con 0x7f31040ab3f0 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2332768893 0 0) 0x7f30e8004630 con 0x7f31040ab3f0 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 msgr2=0x7f31040ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3101e00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f31040ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.116+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 >> v1:172.21.15.82:6789/0 conn(0x7f31040abca0 legacy=0x7f31040c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310406e140 con 0x7f31040ab3f0 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f30f4001670 con 0x7f31040ab3f0 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3103200640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f31040ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 >> v1:172.21.15.67:6789/0 conn(0x7f31040ab3f0 legacy=0x7f310406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8004af0 0x7f30e8004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e8005910 0x7f30e8005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f30e8003440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3104154d80 con 0x7f30e8005910 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e80065f0 con 0x7f30e8004af0 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8003980 con 0x7f31040c1490 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f30fbe00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e8005910 0x7f30e8005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3103200640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f30e8003440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.117+0000 7f3102800640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8004af0 0x7f30e8004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.118+0000 7f30fbe00640 1 -- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 msgr2=0x7f30e8003440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.118+0000 7f30fbe00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31040c1490 0x7f30e8003440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.118+0000 7f30fbe00640 1 -- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8004af0 msgr2=0x7f30e8004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.118+0000 7f30fbe00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8004af0 0x7f30e8004f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.118+0000 7f30fbe00640 1 -- 172.21.15.67:0/1860963170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e8004630 con 0x7f30e8005910 2024-02-26T10:40:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.119+0000 7f30fbe00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e8005910 0x7f30e8005d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f30ec001750 tx=0x7f30e8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db76d710c06b4f85 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.119+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec001370 con 0x7f30e8005910 2024-02-26T10:40:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.119+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec0052d0 con 0x7f30e8005910 2024-02-26T10:40:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.119+0000 7f3101e00640 1 -- 172.21.15.67:0/1860963170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec003830 con 0x7f30e8005910 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3108a00640 1 -- 172.21.15.67:0/1860963170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e8005910 msgr2=0x7f30e8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3108a00640 1 --2- 172.21.15.67:0/1860963170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30e8005910 0x7f30e8005d90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f30ec001750 tx=0x7f30e8006940 comp rx=0 tx=0).stop 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3103200640 1 -- 172.21.15.67:0/1860963170 reap_dead start 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3108a00640 1 -- 172.21.15.67:0/1860963170 shutdown_connections 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3108a00640 1 -- 172.21.15.67:0/1860963170 >> 172.21.15.67:0/1860963170 conn(0x7f31040c6380 msgr2=0x7f31040c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3108a00640 1 -- 172.21.15.67:0/1860963170 shutdown_connections 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.120+0000 7f3108a00640 1 -- 172.21.15.67:0/1860963170 wait complete. 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.121+0000 7f3108a00640 1 Processor -- start 2024-02-26T10:40:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 -- start start 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8005910 0x7f3104145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31041466f0 0x7f31040a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 0x7f31040bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310406e440 con 0x7f310414d3c0 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31040ab7d0 con 0x7f30e8005910 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.122+0000 7f3108a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31040c29e0 con 0x7f31041466f0 2024-02-26T10:40:36.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 0x7f31040bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 0x7f31040bf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56868/0 (socket says 172.21.15.67:56868) 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 -- 172.21.15.67:0/321542200 learned_addr learned my addr 172.21.15.67:0/321542200 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3102800640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8005910 0x7f3104145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f30fbe00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31041466f0 0x7f31040a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 -- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31041466f0 msgr2=0x7f31040a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31041466f0 0x7f31040a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 -- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8005910 msgr2=0x7f3104145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8005910 0x7f3104145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f3103200640 1 -- 172.21.15.67:0/321542200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31041ea620 con 0x7f310414d3c0 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.123+0000 7f30fbe00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31041466f0 0x7f31040a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.124+0000 7f3103200640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 0x7f31040bf7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f310406ccc0 tx=0x7f30fc004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.124+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc007cf0 con 0x7f310414d3c0 2024-02-26T10:40:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.124+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30fc019070 con 0x7f310414d3c0 2024-02-26T10:40:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.124+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc014650 con 0x7f310414d3c0 2024-02-26T10:40:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.124+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31041ea900 con 0x7f310414d3c0 2024-02-26T10:40:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.124+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31041eaec0 con 0x7f310414d3c0 2024-02-26T10:40:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.125+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31040678e0 con 0x7f310414d3c0 2024-02-26T10:40:36.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.126+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30fc0147f0 con 0x7f310414d3c0 2024-02-26T10:40:36.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.127+0000 7f30faa00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30e006b4d0 0x7f30e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.127+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f30fc08a5c0 con 0x7f310414d3c0 2024-02-26T10:40:36.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.128+0000 7f3102800640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30e006b4d0 0x7f30e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.129+0000 7f3102800640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30e006b4d0 0x7f30e006d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f30f4001cd0 tx=0x7f30f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.131+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30fc0596d0 con 0x7f310414d3c0 2024-02-26T10:40:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.332+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3104154500 con 0x7f310414d3c0 2024-02-26T10:40:36.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.333+0000 7f30faa00640 1 -- 172.21.15.67:0/321542200 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30fc058fd0 con 0x7f310414d3c0 2024-02-26T10:40:36.333 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30e006b4d0 msgr2=0x7f30e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30e006b4d0 0x7f30e006d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f30f4001cd0 tx=0x7f30f4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 msgr2=0x7f31040bf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 0x7f31040bf7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f310406ccc0 tx=0x7f30fc004780 comp rx=0 tx=0).stop 2024-02-26T10:40:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 shutdown_connections 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f310414d3c0 0x7f31040bf7b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f31041466f0 0x7f31040a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30e006b4d0 0x7f30e006d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 --2- 172.21.15.67:0/321542200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30e8005910 0x7f3104145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 >> 172.21.15.67:0/321542200 conn(0x7f31040c6380 msgr2=0x7f31040c2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 shutdown_connections 2024-02-26T10:40:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.336+0000 7f3108a00640 1 -- 172.21.15.67:0/321542200 wait complete. 2024-02-26T10:40:36.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent' 2024-02-26T10:40:36.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.577+0000 7fee4b800640 1 Processor -- start 2024-02-26T10:40:36.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.577+0000 7fee4b800640 1 -- start start 2024-02-26T10:40:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee4b800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee440bfa30 0x7fee44158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee4b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee440c1090 0x7fee44158e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee4b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee440c15e0 con 0x7fee44152420 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee4b800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee441593a0 con 0x7fee440bfa30 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee4b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4406e660 con 0x7fee440c1090 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee43e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee440bfa30 0x7fee44158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee43e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee440bfa30 0x7fee44158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56870/0 (socket says 172.21.15.67:56870) 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee43e00640 1 -- 172.21.15.67:0/195734999 learned_addr learned my addr 172.21.15.67:0/195734999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.579+0000 7fee49800640 1 --2- 172.21.15.67:0/195734999 >> v2:172.21.15.82:3300/0 conn(0x7fee440c1090 0x7fee44158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.580+0000 7fee43e00640 1 -- 172.21.15.67:0/195734999 >> v2:172.21.15.82:3300/0 conn(0x7fee440c1090 msgr2=0x7fee44158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.580+0000 7fee43e00640 1 --2- 172.21.15.67:0/195734999 >> v2:172.21.15.82:3300/0 conn(0x7fee440c1090 0x7fee44158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.580+0000 7fee43e00640 1 -- 172.21.15.67:0/195734999 >> v1:172.21.15.82:6789/0 conn(0x7fee44152420 legacy=0x7fee440c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.580+0000 7fee43e00640 1 -- 172.21.15.67:0/195734999 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4406e920 con 0x7fee440bfa30 2024-02-26T10:40:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.581+0000 7fee43e00640 1 --2- 172.21.15.67:0/195734999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440bfa30 0x7fee44158920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fee44159520 tx=0x7fee3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b078ade743e27da0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.581+0000 7fee43400640 1 -- 172.21.15.67:0/195734999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3002e070 con 0x7fee440bfa30 2024-02-26T10:40:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.581+0000 7fee43400640 1 -- 172.21.15.67:0/195734999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30037070 con 0x7fee440bfa30 2024-02-26T10:40:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.581+0000 7fee43400640 1 -- 172.21.15.67:0/195734999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30031a00 con 0x7fee440bfa30 2024-02-26T10:40:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 -- 172.21.15.67:0/195734999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440bfa30 msgr2=0x7fee44158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 --2- 172.21.15.67:0/195734999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440bfa30 0x7fee44158920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fee44159520 tx=0x7fee3002b880 comp rx=0 tx=0).stop 2024-02-26T10:40:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 -- 172.21.15.67:0/195734999 shutdown_connections 2024-02-26T10:40:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 --2- 172.21.15.67:0/195734999 >> v2:172.21.15.82:3300/0 conn(0x7fee440c1090 0x7fee44158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 --2- 172.21.15.67:0/195734999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440bfa30 0x7fee44158920 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 -- 172.21.15.67:0/195734999 >> 172.21.15.67:0/195734999 conn(0x7fee440a84a0 msgr2=0x7fee440a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 -- 172.21.15.67:0/195734999 shutdown_connections 2024-02-26T10:40:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.582+0000 7fee4b800640 1 -- 172.21.15.67:0/195734999 wait complete. 2024-02-26T10:40:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.584+0000 7fee4b800640 1 Processor -- start 2024-02-26T10:40:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.584+0000 7fee4b800640 1 -- start start 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee4b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee440bfa30 0x7fee440aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee4b800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 0x7fee441eb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee4b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fee44152420 0x7fee441eb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee4b800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee44158400 con 0x7fee440c1090 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee4b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee440c1b60 con 0x7fee440bfa30 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee4b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee44152800 con 0x7fee44152420 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 0x7fee441eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 0x7fee441eb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56884/0 (socket says 172.21.15.67:56884) 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.585+0000 7fee43e00640 1 -- 172.21.15.67:0/1015821756 learned_addr learned my addr 172.21.15.67:0/1015821756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee48e00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee440bfa30 0x7fee440aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee49800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fee44152420 0x7fee441eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee43e00640 1 -- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fee44152420 msgr2=0x7fee441eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee43e00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fee44152420 0x7fee441eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee43e00640 1 -- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee440bfa30 msgr2=0x7fee440aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee43e00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee440bfa30 0x7fee440aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee43e00640 1 -- 172.21.15.67:0/1015821756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee441ebab0 con 0x7fee440c1090 2024-02-26T10:40:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee43e00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 0x7fee441eb030 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fee30000c00 tx=0x7fee300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.586+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30032670 con 0x7fee440c1090 2024-02-26T10:40:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.587+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30032810 con 0x7fee440c1090 2024-02-26T10:40:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.587+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee441eab30 con 0x7fee440c1090 2024-02-26T10:40:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.587+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30032be0 con 0x7fee440c1090 2024-02-26T10:40:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.587+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee441ee860 con 0x7fee440c1090 2024-02-26T10:40:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.588+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee440c29f0 con 0x7fee440c1090 2024-02-26T10:40:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.589+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee3004c060 con 0x7fee440c1090 2024-02-26T10:40:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.590+0000 7fee40c00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fee2406b500 0x7fee2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.590+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fee300ab4f0 con 0x7fee440c1090 2024-02-26T10:40:36.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.590+0000 7fee48e00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fee2406b500 0x7fee2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.591+0000 7fee48e00640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fee2406b500 0x7fee2406d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee38006fd0 tx=0x7fee38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:36.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.594+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee440c29f0 con 0x7fee440c1090 2024-02-26T10:40:36.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:36.869+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]} v 0) v1 -- 0x7fee44074a30 con 0x7fee440c1090 2024-02-26T10:40:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.425+0000 7fee40c00640 1 -- 172.21.15.67:0/1015821756 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]}]=0 v133) v1 ==== 187+0+0 (secure 0 0 0) 0x7fee441ebab0 con 0x7fee440c1090 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fee2406b500 msgr2=0x7fee2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fee2406b500 0x7fee2406d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee38006fd0 tx=0x7fee38008040 comp rx=0 tx=0).stop 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 msgr2=0x7fee441eb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 0x7fee441eb030 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fee30000c00 tx=0x7fee300046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 shutdown_connections 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fee44152420 0x7fee441eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee440c1090 0x7fee441eb030 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fee2406b500 0x7fee2406d9c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 --2- 172.21.15.67:0/1015821756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee440bfa30 0x7fee440aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 >> 172.21.15.67:0/1015821756 conn(0x7fee440a84a0 msgr2=0x7fee44150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:37.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 shutdown_connections 2024-02-26T10:40:37.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.429+0000 7fee4b800640 1 -- 172.21.15.67:0/1015821756 wait complete. 2024-02-26T10:40:37.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-26T10:40:37.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.678+0000 7f96c8600640 1 Processor -- start 2024-02-26T10:40:37.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.678+0000 7f96c8600640 1 -- start start 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.679+0000 7f96c8600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f96c00c2980 0x7f96c0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.680+0000 7f96c8600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c00c1ec0 con 0x7f96c0070ed0 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.680+0000 7f96c8600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c00c1100 con 0x7f96c01569a0 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.680+0000 7f96c8600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00c1a30 con 0x7f96c00c2980 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.680+0000 7f96c6600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f96c0070ed0 0x7f96c0071340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56708/0 (socket says 172.21.15.67:56708) 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.680+0000 7f96c6600640 1 -- 172.21.15.67:0/491259768 learned_addr learned my addr 172.21.15.67:0/491259768 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:37.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.680+0000 7f96c5200640 1 --2- 172.21.15.67:0/491259768 >> v2:172.21.15.82:3300/0 conn(0x7f96c00c2980 0x7f96c0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.681+0000 7f96bfe00640 1 -- 172.21.15.67:0/491259768 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972005757 0 0) 0x7f96c00c1ec0 con 0x7f96c0070ed0 2024-02-26T10:40:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.681+0000 7f96bfe00640 1 -- 172.21.15.67:0/491259768 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a8003520 con 0x7f96c0070ed0 2024-02-26T10:40:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.681+0000 7f96c5200640 1 -- 172.21.15.67:0/491259768 >> v1:172.21.15.82:6789/0 conn(0x7f96c01569a0 legacy=0x7f96c006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.681+0000 7f96bfe00640 1 -- 172.21.15.67:0/491259768 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759555673 0 0) 0x7f96c00c1100 con 0x7f96c01569a0 2024-02-26T10:40:37.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.681+0000 7f96c5200640 1 -- 172.21.15.67:0/491259768 >> v1:172.21.15.67:6789/0 conn(0x7f96c0070ed0 legacy=0x7f96c0071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.681+0000 7f96c5200640 1 -- 172.21.15.67:0/491259768 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00c13e0 con 0x7f96c00c2980 2024-02-26T10:40:37.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.683+0000 7f96c5200640 1 --2- 172.21.15.67:0/491259768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c00c2980 0x7f96c0159da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0097a0 tx=0x7f96ac02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50e27d6347d607a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.683+0000 7f96bfe00640 1 -- 172.21.15.67:0/491259768 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0315a0 con 0x7f96c00c2980 2024-02-26T10:40:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.683+0000 7f96bfe00640 1 -- 172.21.15.67:0/491259768 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac037070 con 0x7f96c00c2980 2024-02-26T10:40:37.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.683+0000 7f96bfe00640 1 -- 172.21.15.67:0/491259768 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac031960 con 0x7f96c00c2980 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 -- 172.21.15.67:0/491259768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c00c2980 msgr2=0x7f96c0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 --2- 172.21.15.67:0/491259768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c00c2980 0x7f96c0159da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0097a0 tx=0x7f96ac02b840 comp rx=0 tx=0).stop 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 -- 172.21.15.67:0/491259768 shutdown_connections 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 --2- 172.21.15.67:0/491259768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c00c2980 0x7f96c0159da0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 -- 172.21.15.67:0/491259768 >> 172.21.15.67:0/491259768 conn(0x7f96c00c6da0 msgr2=0x7f96c00c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 -- 172.21.15.67:0/491259768 shutdown_connections 2024-02-26T10:40:37.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.684+0000 7f96c8600640 1 -- 172.21.15.67:0/491259768 wait complete. 2024-02-26T10:40:37.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.687+0000 7f96c8600640 1 Processor -- start 2024-02-26T10:40:37.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.687+0000 7f96c8600640 1 -- start start 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96c0070ed0 0x7f96c01f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c8600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 0x7f96c01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c01569a0 0x7f96c0152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c8600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00a6140 con 0x7f96c00c2980 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0156d80 con 0x7f96c01569a0 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c006e560 con 0x7f96c0070ed0 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.688+0000 7f96c5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 0x7f96c01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 0x7f96c01526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56894/0 (socket says 172.21.15.67:56894) 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 -- 172.21.15.67:0/2130003173 learned_addr learned my addr 172.21.15.67:0/2130003173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5c00640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96c0070ed0 0x7f96c01f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c6600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c01569a0 0x7f96c0152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 -- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96c0070ed0 msgr2=0x7f96c01f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96c0070ed0 0x7f96c01f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 -- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c01569a0 msgr2=0x7f96c0152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c01569a0 0x7f96c0152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5200640 1 -- 172.21.15.67:0/2130003173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c01f33d0 con 0x7f96c00c2980 2024-02-26T10:40:37.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.689+0000 7f96c5c00640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96c0070ed0 0x7f96c01f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.690+0000 7f96c5200640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 0x7f96c01526d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0098c0 tx=0x7f96ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.690+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac04a070 con 0x7f96c00c2980 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.690+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac032430 con 0x7f96c00c2980 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.690+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c00a2580 con 0x7f96c00c2980 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.690+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac032800 con 0x7f96c00c2980 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.690+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c015b4b0 con 0x7f96c00c2980 2024-02-26T10:40:37.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.691+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c015b2c0 con 0x7f96c00c2980 2024-02-26T10:40:37.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.692+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96ac0329a0 con 0x7f96c00c2980 2024-02-26T10:40:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.693+0000 7f96bd600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96a006b4d0 0x7f96a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.693+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f96ac02f020 con 0x7f96c00c2980 2024-02-26T10:40:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.694+0000 7f96c5c00640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96a006b4d0 0x7f96a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.695+0000 7f96c5c00640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96a006b4d0 0x7f96a006d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96b4000f60 tx=0x7f96b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:37.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.698+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96ac040070 con 0x7f96c00c2980 2024-02-26T10:40:37.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.897+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f96c0074a30 con 0x7f96c00c2980 2024-02-26T10:40:37.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.905+0000 7f96bd600640 1 -- 172.21.15.67:0/2130003173 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f96ac07aa40 con 0x7f96c00c2980 2024-02-26T10:40:37.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96a006b4d0 msgr2=0x7f96a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96a006b4d0 0x7f96a006d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96b4000f60 tx=0x7f96b4009040 comp rx=0 tx=0).stop 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 msgr2=0x7f96c01526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 0x7f96c01526d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0098c0 tx=0x7f96ac0046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 shutdown_connections 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f96a006b4d0 0x7f96a006d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c01569a0 0x7f96c0152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c00c2980 0x7f96c01526d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 --2- 172.21.15.67:0/2130003173 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96c0070ed0 0x7f96c01f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.908+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 >> 172.21.15.67:0/2130003173 conn(0x7f96c00c6da0 msgr2=0x7f96c00c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.909+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 shutdown_connections 2024-02-26T10:40:37.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:37.909+0000 7f96c8600640 1 -- 172.21.15.67:0/2130003173 wait complete. 2024-02-26T10:40:37.928 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-26T10:40:37.928 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:40:38.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:40:38.152 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-26T10:40:38.152 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:38.152 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:38.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:40:38.213 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:38.213 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:38.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:40:38.270 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-02-26T10:40:38.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:38.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.508+0000 7ff263200640 1 Processor -- start 2024-02-26T10:40:38.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.509+0000 7ff263200640 1 -- start start 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff263200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2640c1c10 0x7ff2641557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff263200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff26406e180 0x7ff26406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff263200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff264155cf0 con 0x7ff2640c0850 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff263200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26406dca0 con 0x7ff26406e180 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff263200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640c2a70 con 0x7ff2640c1c10 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff25be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff2640c0850 0x7ff26406ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56724/0 (socket says 172.21.15.67:56724) 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.510+0000 7ff25be00640 1 -- 172.21.15.67:0/3336337472 learned_addr learned my addr 172.21.15.67:0/3336337472 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff262800640 1 --2- 172.21.15.67:0/3336337472 >> v2:172.21.15.67:3300/0 conn(0x7ff26406e180 0x7ff26406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff261e00640 1 --2- 172.21.15.67:0/3336337472 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2640c1c10 0x7ff2641557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:38.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff261400640 1 -- 172.21.15.67:0/3336337472 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841548866 0 0) 0x7ff24c001110 con 0x7ff2640c0850 2024-02-26T10:40:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff261400640 1 -- 172.21.15.67:0/3336337472 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff248003520 con 0x7ff2640c0850 2024-02-26T10:40:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff262800640 1 -- 172.21.15.67:0/3336337472 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2640c1c10 msgr2=0x7ff2641557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff262800640 1 --2- 172.21.15.67:0/3336337472 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2640c1c10 0x7ff2641557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff262800640 1 -- 172.21.15.67:0/3336337472 >> v1:172.21.15.67:6789/0 conn(0x7ff2640c0850 legacy=0x7ff26406ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.511+0000 7ff262800640 1 -- 172.21.15.67:0/3336337472 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2640c21d0 con 0x7ff26406e180 2024-02-26T10:40:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.513+0000 7ff262800640 1 --2- 172.21.15.67:0/3336337472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff26406e180 0x7ff26406e630 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009d90 tx=0x7ff25c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125e986d37586ca6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.513+0000 7ff261400640 1 -- 172.21.15.67:0/3336337472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c039070 con 0x7ff26406e180 2024-02-26T10:40:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.513+0000 7ff261400640 1 -- 172.21.15.67:0/3336337472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c02cb30 con 0x7ff26406e180 2024-02-26T10:40:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.513+0000 7ff261400640 1 -- 172.21.15.67:0/3336337472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c034440 con 0x7ff26406e180 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 -- 172.21.15.67:0/3336337472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff26406e180 msgr2=0x7ff26406e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 --2- 172.21.15.67:0/3336337472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff26406e180 0x7ff26406e630 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009d90 tx=0x7ff25c02bd80 comp rx=0 tx=0).stop 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 -- 172.21.15.67:0/3336337472 shutdown_connections 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 --2- 172.21.15.67:0/3336337472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff26406e180 0x7ff26406e630 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 --2- 172.21.15.67:0/3336337472 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2640c1c10 0x7ff2641557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 -- 172.21.15.67:0/3336337472 >> 172.21.15.67:0/3336337472 conn(0x7ff2640c63f0 msgr2=0x7ff2640c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 -- 172.21.15.67:0/3336337472 shutdown_connections 2024-02-26T10:40:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.514+0000 7ff263200640 1 -- 172.21.15.67:0/3336337472 wait complete. 2024-02-26T10:40:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.515+0000 7ff263200640 1 Processor -- start 2024-02-26T10:40:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.516+0000 7ff263200640 1 -- start start 2024-02-26T10:40:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.516+0000 7ff263200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff26406e180 0x7ff2641ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff263200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2640c0850 0x7ff26414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff263200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 0x7ff26414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff263200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26406b330 con 0x7ff2640c1c10 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff263200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640c2790 con 0x7ff2640c0850 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff263200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26406b4b0 con 0x7ff26406e180 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 0x7ff26414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 0x7ff26414b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56914/0 (socket says 172.21.15.67:56914) 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 -- 172.21.15.67:0/1988289467 learned_addr learned my addr 172.21.15.67:0/1988289467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff261e00640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff26406e180 0x7ff2641ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff25be00640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2640c0850 0x7ff26414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 -- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff26406e180 msgr2=0x7ff2641ebce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff26406e180 0x7ff2641ebce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 -- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2640c0850 msgr2=0x7ff26414d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2640c0850 0x7ff26414d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff262800640 1 -- 172.21.15.67:0/1988289467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26414cf50 con 0x7ff2640c1c10 2024-02-26T10:40:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.517+0000 7ff25be00640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2640c0850 0x7ff26414d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.518+0000 7ff262800640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 0x7ff26414b180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009eb0 tx=0x7ff25c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.518+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c039050 con 0x7ff2640c1c10 2024-02-26T10:40:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.518+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c045070 con 0x7ff2640c1c10 2024-02-26T10:40:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.518+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c03e850 con 0x7ff2640c1c10 2024-02-26T10:40:38.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.518+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2640a9430 con 0x7ff2640c1c10 2024-02-26T10:40:38.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.518+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26414d1d0 con 0x7ff2640c1c10 2024-02-26T10:40:38.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.520+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff264074a30 con 0x7ff2640c1c10 2024-02-26T10:40:38.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.520+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff25c03e9f0 con 0x7ff2640c1c10 2024-02-26T10:40:38.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.521+0000 7ff25a000640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff24006b4d0 0x7ff24006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:38.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.521+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff25c0acdc0 con 0x7ff2640c1c10 2024-02-26T10:40:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.522+0000 7ff261e00640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff24006b4d0 0x7ff24006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.523+0000 7ff261e00640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff24006b4d0 0x7ff24006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff254006fd0 tx=0x7ff254008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.526+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff26414cf50 con 0x7ff2640c1c10 2024-02-26T10:40:38.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.764+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2640ac540 con 0x7ff2640c1c10 2024-02-26T10:40:38.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.765+0000 7ff25a000640 1 -- 172.21.15.67:0/1988289467 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3696 (secure 0 0 0) 0x7ff25c032e20 con 0x7ff2640c1c10 2024-02-26T10:40:38.765 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:38.765 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1727063027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":1727063027},{"type":"v1","addr":"172.21.15.67:6835","nonce":1727063027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4747,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2288847923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2288847923},{"type":"v1","addr":"172.21.15.82:6836","nonce":2288847923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/224163452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":224163452},{"type":"v1","addr":"172.21.15.82:6837","nonce":224163452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6833/4169139190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":4169139190},{"type":"v1","addr":"172.21.15.67:6833","nonce":4169139190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[]} 2024-02-26T10:40:38.765 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 26 2024-02-26T10:40:38.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff24006b4d0 msgr2=0x7ff24006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff24006b4d0 0x7ff24006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff254006fd0 tx=0x7ff254008040 comp rx=0 tx=0).stop 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 msgr2=0x7ff26414b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 0x7ff26414b180 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009eb0 tx=0x7ff25c004a40 comp rx=0 tx=0).stop 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 shutdown_connections 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2640c1c10 0x7ff26414b180 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff24006b4d0 0x7ff24006d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2640c0850 0x7ff26414d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 --2- 172.21.15.67:0/1988289467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff26406e180 0x7ff2641ebce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 >> 172.21.15.67:0/1988289467 conn(0x7ff2640c63f0 msgr2=0x7ff2640c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.767+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 shutdown_connections 2024-02-26T10:40:38.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:38.768+0000 7ff263200640 1 -- 172.21.15.67:0/1988289467 wait complete. 2024-02-26T10:40:38.785 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-26T10:40:38.786 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-26T10:40:38.786 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-26T10:40:38.786 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:40:38.786 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-26T10:40:38.786 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-26T10:40:38.787 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-26T10:40:38.787 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:40:38.787 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-26T10:40:38.787 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-26T10:40:38.787 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-26T10:40:38.787 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:40:38.787 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-26T10:40:38.788 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-26T10:40:38.788 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-26T10:40:38.788 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:40:38.811 INFO:tasks.ceph.mds.a:Stopped 2024-02-26T10:40:38.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-26T10:40:38.815 INFO:tasks.ceph.mds.c:Stopped 2024-02-26T10:40:38.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-26T10:40:38.818 INFO:tasks.ceph.mds.b:Stopped 2024-02-26T10:40:38.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-26T10:40:38.819 INFO:tasks.ceph.mds.d:Stopped 2024-02-26T10:40:38.819 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-26T10:40:38.861 INFO:tasks.ceph.mds.a:Started 2024-02-26T10:40:38.862 DEBUG:teuthology.parallel:result is None 2024-02-26T10:40:38.865 INFO:tasks.ceph.mds.b:Started 2024-02-26T10:40:38.865 DEBUG:teuthology.parallel:result is None 2024-02-26T10:40:38.867 INFO:tasks.ceph.mds.c:Started 2024-02-26T10:40:38.868 DEBUG:teuthology.parallel:result is None 2024-02-26T10:40:38.871 INFO:tasks.ceph.mds.d:Started 2024-02-26T10:40:38.871 DEBUG:teuthology.parallel:result is None 2024-02-26T10:40:38.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-26T10:40:39.025 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-02-26T10:40:39.025 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:40:39.026 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-02-26T10:40:39.029 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-02-26T10:40:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.068+0000 7f24fb800640 1 Processor -- start 2024-02-26T10:40:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.069+0000 7f24fb800640 1 -- start start 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24fb800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4072ea0 0x7f24f4154e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24fb800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f40c0a40 con 0x7f24f4072a30 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24fb800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f4155ac0 con 0x7f24f406ae90 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24fb800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f4073330 con 0x7f24f4072ea0 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24f9800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f24f4072a30 0x7f24f41553b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56730/0 (socket says 172.21.15.67:56730) 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24f9800640 1 -- 172.21.15.67:0/1589507979 learned_addr learned my addr 172.21.15.67:0/1589507979 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.070+0000 7f24f3e00640 1 --2- 172.21.15.67:0/1589507979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4072ea0 0x7f24f4154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609527925 0 0) 0x7f24f40c0a40 con 0x7f24f4072a30 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24dc003520 con 0x7f24f4072a30 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747654898 0 0) 0x7f24f4155ac0 con 0x7f24f406ae90 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f40c0a40 con 0x7f24f406ae90 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3e00640 1 -- 172.21.15.67:0/1589507979 >> v1:172.21.15.82:6789/0 conn(0x7f24f406ae90 legacy=0x7f24f41521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3e00640 1 -- 172.21.15.67:0/1589507979 >> v1:172.21.15.67:6789/0 conn(0x7f24f4072a30 legacy=0x7f24f41553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.071+0000 7f24f3e00640 1 -- 172.21.15.67:0/1589507979 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40c7010 con 0x7f24f4072ea0 2024-02-26T10:40:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.072+0000 7f24f3e00640 1 --2- 172.21.15.67:0/1589507979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4072ea0 0x7f24f4154e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24e00097d0 tx=0x7f24e002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0f74ffe35a474b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.072+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0035020 con 0x7f24f4072ea0 2024-02-26T10:40:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.072+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0031890 con 0x7f24f4072ea0 2024-02-26T10:40:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.072+0000 7f24f3400640 1 -- 172.21.15.67:0/1589507979 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0031c60 con 0x7f24f4072ea0 2024-02-26T10:40:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 -- 172.21.15.67:0/1589507979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4072ea0 msgr2=0x7f24f4154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 --2- 172.21.15.67:0/1589507979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4072ea0 0x7f24f4154e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24e00097d0 tx=0x7f24e002b930 comp rx=0 tx=0).stop 2024-02-26T10:40:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 -- 172.21.15.67:0/1589507979 shutdown_connections 2024-02-26T10:40:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 --2- 172.21.15.67:0/1589507979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4072ea0 0x7f24f4154e70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 -- 172.21.15.67:0/1589507979 >> 172.21.15.67:0/1589507979 conn(0x7f24f40c7c40 msgr2=0x7f24f406bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 -- 172.21.15.67:0/1589507979 shutdown_connections 2024-02-26T10:40:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.073+0000 7f24fb800640 1 -- 172.21.15.67:0/1589507979 wait complete. 2024-02-26T10:40:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.075+0000 7f24fb800640 1 Processor -- start 2024-02-26T10:40:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.075+0000 7f24fb800640 1 -- start start 2024-02-26T10:40:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24fb800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 0x7f24f41ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24fb800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f4072a30 0x7f24f41536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24fb800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4159090 0x7f24f4153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24fb800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f406b270 con 0x7f24f4072a30 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24fb800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c0d70 con 0x7f24f406ae90 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24fb800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f406b3f0 con 0x7f24f4159090 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24f8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 0x7f24f41ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f4072a30 0x7f24f41536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24f8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 0x7f24f41ede10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34372/0 (socket says 172.21.15.67:34372) 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24f8e00640 1 -- 172.21.15.67:0/2202022433 learned_addr learned my addr 172.21.15.67:0/2202022433 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.076+0000 7f24f9800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4159090 0x7f24f4153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f8e00640 1 -- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4159090 msgr2=0x7f24f4153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f8e00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4159090 0x7f24f4153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f8e00640 1 -- 172.21.15.67:0/2202022433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f4072a30 msgr2=0x7f24f41536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f8e00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f4072a30 0x7f24f41536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f8e00640 1 -- 172.21.15.67:0/2202022433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4151260 con 0x7f24f406ae90 2024-02-26T10:40:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f3e00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f4072a30 0x7f24f41536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:40:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f8e00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 0x7f24f41ede10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f24e800c430 tx=0x7f24e800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8017070 con 0x7f24f406ae90 2024-02-26T10:40:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e80052f0 con 0x7f24f406ae90 2024-02-26T10:40:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8003830 con 0x7f24f406ae90 2024-02-26T10:40:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.077+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f4154cb0 con 0x7f24f406ae90 2024-02-26T10:40:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.078+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f4154f50 con 0x7f24f406ae90 2024-02-26T10:40:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.079+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f4074a30 con 0x7f24f406ae90 2024-02-26T10:40:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.080+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24e801a070 con 0x7f24f406ae90 2024-02-26T10:40:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.080+0000 7f24f0c00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24d406b4d0 0x7f24d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.081+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f24e8089260 con 0x7f24f406ae90 2024-02-26T10:40:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.081+0000 7f24f3e00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24d406b4d0 0x7f24d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.082+0000 7f24f3e00640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24d406b4d0 0x7f24d406d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036000 tx=0x7f24e0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:39.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.089+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24f4151260 con 0x7f24f406ae90 2024-02-26T10:40:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:39.299+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24f4155120 con 0x7f24f406ae90 2024-02-26T10:40:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.472+0000 7f24f0c00640 1 -- 172.21.15.67:0/2202022433 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24e8089080 con 0x7f24f406ae90 2024-02-26T10:40:40.473 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24d406b4d0 msgr2=0x7f24d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24d406b4d0 0x7f24d406d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036000 tx=0x7f24e0008030 comp rx=0 tx=0).stop 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 msgr2=0x7f24f41ede10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 0x7f24f41ede10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f24e800c430 tx=0x7f24e800c8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 shutdown_connections 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24f4159090 0x7f24f4153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f4072a30 0x7f24f41536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24d406b4d0 0x7f24d406d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 --2- 172.21.15.67:0/2202022433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f406ae90 0x7f24f41ede10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 >> 172.21.15.67:0/2202022433 conn(0x7f24f40c7c40 msgr2=0x7f24f40c3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:40.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 shutdown_connections 2024-02-26T10:40:40.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.476+0000 7f24fb800640 1 -- 172.21.15.67:0/2202022433 wait complete. 2024-02-26T10:40:40.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-26T10:40:40.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.719+0000 7fe123e00640 1 Processor -- start 2024-02-26T10:40:40.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.719+0000 7fe123e00640 1 -- start start 2024-02-26T10:40:40.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe123e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406ac20 0x7fe1240a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:40.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe123e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe12406b000 0x7fe12406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:40.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe123e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12406b460 con 0x7fe124153610 2024-02-26T10:40:40.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe123e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240a9a80 con 0x7fe12406b000 2024-02-26T10:40:40.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe123e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240a9d60 con 0x7fe12406ac20 2024-02-26T10:40:40.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe11a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe12406b000 0x7fe12406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:40.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe11a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe12406b000 0x7fe12406db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35764/0 (socket says 172.21.15.67:35764) 2024-02-26T10:40:40.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.721+0000 7fe11a000640 1 -- 172.21.15.67:0/1499903106 learned_addr learned my addr 172.21.15.67:0/1499903106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:40.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.722+0000 7fe122a00640 1 --2- 172.21.15.67:0/1499903106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406ac20 0x7fe1240a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:40.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.722+0000 7fe11a000640 1 -- 172.21.15.67:0/1499903106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406ac20 msgr2=0x7fe1240a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:40.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.722+0000 7fe11a000640 1 --2- 172.21.15.67:0/1499903106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406ac20 0x7fe1240a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.722+0000 7fe11a000640 1 -- 172.21.15.67:0/1499903106 >> v1:172.21.15.82:6789/0 conn(0x7fe124153610 legacy=0x7fe1240abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.722+0000 7fe11a000640 1 -- 172.21.15.67:0/1499903106 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1240ab750 con 0x7fe12406b000 2024-02-26T10:40:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.723+0000 7fe11a000640 1 --2- 172.21.15.67:0/1499903106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406b000 0x7fe12406db00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009870 tx=0x7fe10c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91cfa78d69023a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe122000640 1 -- 172.21.15.67:0/1499903106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c035020 con 0x7fe12406b000 2024-02-26T10:40:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe122000640 1 -- 172.21.15.67:0/1499903106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c031640 con 0x7fe12406b000 2024-02-26T10:40:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe122000640 1 -- 172.21.15.67:0/1499903106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c02d040 con 0x7fe12406b000 2024-02-26T10:40:40.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 -- 172.21.15.67:0/1499903106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406b000 msgr2=0x7fe12406db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:40.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 --2- 172.21.15.67:0/1499903106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406b000 0x7fe12406db00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009870 tx=0x7fe10c02b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 -- 172.21.15.67:0/1499903106 shutdown_connections 2024-02-26T10:40:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 --2- 172.21.15.67:0/1499903106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406b000 0x7fe12406db00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 --2- 172.21.15.67:0/1499903106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406ac20 0x7fe1240a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 -- 172.21.15.67:0/1499903106 >> 172.21.15.67:0/1499903106 conn(0x7fe1240c63f0 msgr2=0x7fe1240c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.724+0000 7fe123e00640 1 -- 172.21.15.67:0/1499903106 shutdown_connections 2024-02-26T10:40:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.725+0000 7fe123e00640 1 -- 172.21.15.67:0/1499903106 wait complete. 2024-02-26T10:40:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.726+0000 7fe123e00640 1 Processor -- start 2024-02-26T10:40:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.726+0000 7fe123e00640 1 -- start start 2024-02-26T10:40:40.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.727+0000 7fe123e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 0x7fe12414af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:40.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.727+0000 7fe123e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406b000 0x7fe12414b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:40.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.727+0000 7fe123e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe124153610 0x7fe124156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.727+0000 7fe123e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe124150410 con 0x7fe12406ac20 2024-02-26T10:40:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.727+0000 7fe123e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe124153aa0 con 0x7fe124153610 2024-02-26T10:40:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.727+0000 7fe123e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe124150590 con 0x7fe12406b000 2024-02-26T10:40:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 0x7fe12414af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 0x7fe12414af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35768/0 (socket says 172.21.15.67:35768) 2024-02-26T10:40:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 -- 172.21.15.67:0/722373223 learned_addr learned my addr 172.21.15.67:0/722373223 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe11a000640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406b000 0x7fe12414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 -- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406b000 msgr2=0x7fe12414b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406b000 0x7fe12414b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 -- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe124153610 msgr2=0x7fe124156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe124153610 0x7fe124156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.728+0000 7fe122a00640 1 -- 172.21.15.67:0/722373223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1241e8a90 con 0x7fe12406ac20 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.729+0000 7fe122a00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 0x7fe12414af30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe114007ef0 tx=0x7fe11400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.729+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114014620 con 0x7fe12406ac20 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.729+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114019070 con 0x7fe12406ac20 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.729+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114014a00 con 0x7fe12406ac20 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.729+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12414a770 con 0x7fe12406ac20 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.729+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1241e9030 con 0x7fe12406ac20 2024-02-26T10:40:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.730+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12406bea0 con 0x7fe12406ac20 2024-02-26T10:40:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.731+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe114014ba0 con 0x7fe12406ac20 2024-02-26T10:40:40.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.733+0000 7fe11b400640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:40.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.733+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe11408a760 con 0x7fe12406ac20 2024-02-26T10:40:40.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.733+0000 7fe11a000640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:40.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.734+0000 7fe11a000640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe10006b4d0 0x7fe10006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe10c036000 tx=0x7fe10c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:40.738+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe12406bea0 con 0x7fe12406ac20 2024-02-26T10:40:41.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.010+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe124074a30 con 0x7fe12406ac20 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.011+0000 7fe11b400640 1 -- 172.21.15.67:0/722373223 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5021 (secure 0 0 0) 0x7fe1140599c0 con 0x7fe12406ac20 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlrPwnMBAADC1UqiHSleXCZZ8LRLN/MQ==", 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.011 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlXEpIGBAANii7fywaaL0+vjyqLX6Atw==", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.012 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxl2Z1dBhAAT9aRD1Te0TwHE93ZvNdInA==", 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-02-26T10:40:41.013 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlDx1gJxAAYGDhukySlrxjHaHWn+707g==", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDzadxlki0qBBAAWmQAFvxfJJ5kYMrk0ChwLQ==", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD2adxl4X5OHRAAHtaf00M+kXgvYOn1/ddr7A==", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD5adxlQqxnNxAAWzDQ/1DzO3r1ztTrociQzQ==", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.015 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD9adxlgCXOFRAAHdACX7JyyLLY0KPVHOycGw==", 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-02-26T10:40:41.016 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAEatxlJ8FqFhAA2XFN8d8NUYW1rITUu3tQnA==", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAHatxlO9YRMBAA6z8hYbTwHnU0BV3OMxKSjA==", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.017 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAKatxlzGhIORAANuWy2QCgtFcJRO4vQV4EkQ==", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAOatxlUwuJGBAAYZlIHvgD3suXzh5kyeUWqQ==", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw==", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw==", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-02-26T10:40:41.019 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw pool=cephfs2_data" 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDsadxlvPgjIxAAClmsDQ2hetAdBGNLgj50lg==", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.testuser", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQCHatxl8b54GRAAIFSuHYyYQln561YAeiB2Hg==", 2024-02-26T10:40:41.020 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow rw fsname=cephfs", 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r fsname=cephfs", 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlQMXpIxAA0W1caz5FAL2wDMrNG3j5Mg==", 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:41.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlMa/TFxAAAa7erEK+tXM+k5KOfx2JgA==", 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-02-26T10:40:41.022 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe10006b4d0 msgr2=0x7fe10006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe10006b4d0 0x7fe10006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe10c036000 tx=0x7fe10c03c040 comp rx=0 tx=0).stop 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 msgr2=0x7fe12414af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 0x7fe12414af30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe114007ef0 tx=0x7fe11400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 shutdown_connections 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe124153610 0x7fe124156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe12406b000 0x7fe12414b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 --2- 172.21.15.67:0/722373223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe12406ac20 0x7fe12414af30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 >> 172.21.15.67:0/722373223 conn(0x7fe1240c63f0 msgr2=0x7fe12414dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:41.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 shutdown_connections 2024-02-26T10:40:41.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.014+0000 7fe123e00640 1 -- 172.21.15.67:0/722373223 wait complete. 2024-02-26T10:40:41.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.testuser 2024-02-26T10:40:41.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.258+0000 7fa3a9000640 1 Processor -- start 2024-02-26T10:40:41.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.259+0000 7fa3a9000640 1 -- start start 2024-02-26T10:40:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a9000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3a4150300 0x7fa3a40c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3a406e410 0x7fa3a40c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a9000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a4152e50 con 0x7fa3a4156230 2024-02-26T10:40:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a9000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a41506e0 con 0x7fa3a406e410 2024-02-26T10:40:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a9000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a4153a60 con 0x7fa3a4150300 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3a406e410 0x7fa3a40c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3a406e410 0x7fa3a40c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35784/0 (socket says 172.21.15.67:35784) 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.260+0000 7fa3a1400640 1 -- 172.21.15.67:0/2944543260 learned_addr learned my addr 172.21.15.67:0/2944543260 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.261+0000 7fa3a1e00640 1 --2- 172.21.15.67:0/2944543260 >> v2:172.21.15.82:3300/0 conn(0x7fa3a4150300 0x7fa3a40c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.261+0000 7fa3a1400640 1 -- 172.21.15.67:0/2944543260 >> v2:172.21.15.82:3300/0 conn(0x7fa3a4150300 msgr2=0x7fa3a40c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.261+0000 7fa3a1400640 1 --2- 172.21.15.67:0/2944543260 >> v2:172.21.15.82:3300/0 conn(0x7fa3a4150300 0x7fa3a40c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.261+0000 7fa3a1400640 1 -- 172.21.15.67:0/2944543260 >> v1:172.21.15.82:6789/0 conn(0x7fa3a4156230 legacy=0x7fa3a40aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.261+0000 7fa3a1400640 1 -- 172.21.15.67:0/2944543260 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4153130 con 0x7fa3a406e410 2024-02-26T10:40:41.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.262+0000 7fa3a1400640 1 --2- 172.21.15.67:0/2944543260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40c3530 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa3a406bcc0 tx=0x7fa38c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb649e7d487f11 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a0a00640 1 -- 172.21.15.67:0/2944543260 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c02e070 con 0x7fa3a406e410 2024-02-26T10:40:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a0a00640 1 -- 172.21.15.67:0/2944543260 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c037070 con 0x7fa3a406e410 2024-02-26T10:40:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a0a00640 1 -- 172.21.15.67:0/2944543260 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c031a00 con 0x7fa3a406e410 2024-02-26T10:40:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a9000640 1 -- 172.21.15.67:0/2944543260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 msgr2=0x7fa3a40c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2944543260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40c3530 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa3a406bcc0 tx=0x7fa38c02b880 comp rx=0 tx=0).stop 2024-02-26T10:40:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a9000640 1 -- 172.21.15.67:0/2944543260 shutdown_connections 2024-02-26T10:40:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2944543260 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40c3530 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.263+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2944543260 >> v2:172.21.15.82:3300/0 conn(0x7fa3a4150300 0x7fa3a40c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.264+0000 7fa3a9000640 1 -- 172.21.15.67:0/2944543260 >> 172.21.15.67:0/2944543260 conn(0x7fa3a40a84a0 msgr2=0x7fa3a40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.264+0000 7fa3a9000640 1 -- 172.21.15.67:0/2944543260 shutdown_connections 2024-02-26T10:40:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.264+0000 7fa3a9000640 1 -- 172.21.15.67:0/2944543260 wait complete. 2024-02-26T10:40:41.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.265+0000 7fa3a9000640 1 Processor -- start 2024-02-26T10:40:41.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.265+0000 7fa3a9000640 1 -- start start 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.266+0000 7fa3a9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.266+0000 7fa3a9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3a4150300 0x7fa3a4148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.266+0000 7fa3a9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3a4156230 0x7fa3a4148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.266+0000 7fa3a9000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a406dd80 con 0x7fa3a406e410 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.266+0000 7fa3a9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a41536f0 con 0x7fa3a4150300 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.266+0000 7fa3a9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40ab2b0 con 0x7fa3a4156230 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40bfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35800/0 (socket says 172.21.15.67:35800) 2024-02-26T10:40:41.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 -- 172.21.15.67:0/2958920981 learned_addr learned my addr 172.21.15.67:0/2958920981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1400640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3a4150300 0x7fa3a4148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 -- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3a4156230 msgr2=0x7fa3a4148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3a4156230 0x7fa3a4148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 -- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3a4150300 msgr2=0x7fa3a4148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3a4150300 0x7fa3a4148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1e00640 1 -- 172.21.15.67:0/2958920981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a40725b0 con 0x7fa3a406e410 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.267+0000 7fa3a1400640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3a4150300 0x7fa3a4148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.268+0000 7fa3a1e00640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40bfc10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d800 tx=0x7fa39400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.268+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400b660 con 0x7fa3a406e410 2024-02-26T10:40:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.268+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa394019070 con 0x7fa3a406e410 2024-02-26T10:40:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.268+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400ba20 con 0x7fa3a406e410 2024-02-26T10:40:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.268+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a41eed50 con 0x7fa3a406e410 2024-02-26T10:40:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.268+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a4072b50 con 0x7fa3a406e410 2024-02-26T10:40:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.269+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a4074a30 con 0x7fa3a406e410 2024-02-26T10:40:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.270+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa39400bbc0 con 0x7fa3a406e410 2024-02-26T10:40:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.271+0000 7fa39a000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa38006b500 0x7fa38006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.271+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa39408a300 con 0x7fa3a406e410 2024-02-26T10:40:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.272+0000 7fa3a1400640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa38006b500 0x7fa38006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.273+0000 7fa3a1400640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa38006b500 0x7fa38006d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa38c036000 tx=0x7fa38c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:41.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.275+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa39401c360 con 0x7fa3a406e410 2024-02-26T10:40:41.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.558+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.testuser"} v 0) v1 -- 0x7fa3a406e0d0 con 0x7fa3a406e410 2024-02-26T10:40:41.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.563+0000 7fa39a000640 1 -- 172.21.15.67:0/2958920981 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.testuser"}]=0 v12) v1 ==== 85+0+0 (secure 0 0 0) 0x7fa3940594c0 con 0x7fa3a406e410 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa38006b500 msgr2=0x7fa38006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa38006b500 0x7fa38006d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa38c036000 tx=0x7fa38c008030 comp rx=0 tx=0).stop 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 msgr2=0x7fa3a40bfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40bfc10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d800 tx=0x7fa39400dcc0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 shutdown_connections 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3a4156230 0x7fa3a4148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa38006b500 0x7fa38006d9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3a4150300 0x7fa3a4148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 --2- 172.21.15.67:0/2958920981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3a406e410 0x7fa3a40bfc10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 >> 172.21.15.67:0/2958920981 conn(0x7fa3a40a84a0 msgr2=0x7fa3a40c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 shutdown_connections 2024-02-26T10:40:41.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.566+0000 7fa3a9000640 1 -- 172.21.15.67:0/2958920981 wait complete. 2024-02-26T10:40:41.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.804+0000 7f8933200640 1 Processor -- start 2024-02-26T10:40:41.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.804+0000 7f8933200640 1 -- start start 2024-02-26T10:40:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8933200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89340a94a0 0x7f893406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8933200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 0x7f89340abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8933200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893406df90 con 0x7f893406ac20 2024-02-26T10:40:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8933200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89340a9a80 con 0x7f89340a94a0 2024-02-26T10:40:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8933200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340a9d60 con 0x7f893406db00 2024-02-26T10:40:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8931e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f893406ac20 0x7f8934153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42844/0 (socket says 172.21.15.67:42844) 2024-02-26T10:40:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8931e00640 1 -- 172.21.15.67:0/1013556523 learned_addr learned my addr 172.21.15.67:0/1013556523 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:41.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8931400640 1 --2- 172.21.15.67:0/1013556523 >> v2:172.21.15.82:3300/0 conn(0x7f89340a94a0 0x7f893406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8932800640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 0x7f89340abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029729632 0 0) 0x7f893406df90 con 0x7f893406ac20 2024-02-26T10:40:41.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8918003520 con 0x7f893406ac20 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918217735 0 0) 0x7f8918003520 con 0x7f893406ac20 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 msgr2=0x7f89340abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 0x7f89340abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 >> v2:172.21.15.82:3300/0 conn(0x7f89340a94a0 msgr2=0x7f893406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 --2- 172.21.15.67:0/1013556523 >> v2:172.21.15.82:3300/0 conn(0x7f89340a94a0 0x7f893406b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340ab750 con 0x7f893406ac20 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8924001670 con 0x7f893406ac20 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 >> v1:172.21.15.67:6789/0 conn(0x7f893406ac20 legacy=0x7f8934153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 0x7f89180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918003f30 0x7f8918004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8918004d40 0x7f89180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340a9a80 con 0x7f8918004d40 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340a9d60 con 0x7f8918003f30 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.807+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891800b9c0 con 0x7f893406db00 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8932800640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8918004d40 0x7f89180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8931400640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918003f30 0x7f8918004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8931e00640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 0x7f89180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8932800640 1 -- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 msgr2=0x7f89180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8932800640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f893406db00 0x7f89180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8932800640 1 -- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918003f30 msgr2=0x7f8918004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8932800640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918003f30 0x7f8918004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.808+0000 7f8932800640 1 -- 172.21.15.67:0/1013556523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918003520 con 0x7f8918004d40 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.809+0000 7f8932800640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8918004d40 0x7f89180051c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f892c009fc0 tx=0x7f8918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244c4de589ec626 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.810+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c039070 con 0x7f8918004d40 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.810+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c02cc00 con 0x7f8918004d40 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.810+0000 7f8930a00640 1 -- 172.21.15.67:0/1013556523 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c009c10 con 0x7f8918004d40 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.810+0000 7f8933200640 1 -- 172.21.15.67:0/1013556523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8918004d40 msgr2=0x7f89180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.810+0000 7f8933200640 1 --2- 172.21.15.67:0/1013556523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8918004d40 0x7f89180051c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f892c009fc0 tx=0x7f8918005830 comp rx=0 tx=0).stop 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.810+0000 7f8932800640 1 -- 172.21.15.67:0/1013556523 reap_dead start 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.811+0000 7f8933200640 1 -- 172.21.15.67:0/1013556523 shutdown_connections 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.811+0000 7f8933200640 1 -- 172.21.15.67:0/1013556523 >> 172.21.15.67:0/1013556523 conn(0x7f89340c63f0 msgr2=0x7f89340c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.811+0000 7f8933200640 1 -- 172.21.15.67:0/1013556523 shutdown_connections 2024-02-26T10:40:41.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.811+0000 7f8933200640 1 -- 172.21.15.67:0/1013556523 wait complete. 2024-02-26T10:40:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.813+0000 7f8933200640 1 Processor -- start 2024-02-26T10:40:41.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.813+0000 7f8933200640 1 -- start start 2024-02-26T10:40:41.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8933200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918004d40 0x7f89341dd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8933200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89340c4eb0 0x7f89341e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8933200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 0x7f8934147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8933200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934150590 con 0x7f89341ea1b0 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8933200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934150410 con 0x7f8918004d40 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8933200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340abf50 con 0x7f89340c4eb0 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8932800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 0x7f8934147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8932800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 0x7f8934147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35810/0 (socket says 172.21.15.67:35810) 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8932800640 1 -- 172.21.15.67:0/1745118364 learned_addr learned my addr 172.21.15.67:0/1745118364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8931400640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89340c4eb0 0x7f89341e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.815+0000 7f8931e00640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918004d40 0x7f89341dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8932800640 1 -- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89340c4eb0 msgr2=0x7f89341e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8932800640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89340c4eb0 0x7f89341e9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8932800640 1 -- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918004d40 msgr2=0x7f89341dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8932800640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918004d40 0x7f89341dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8932800640 1 -- 172.21.15.67:0/1745118364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340c5340 con 0x7f89341ea1b0 2024-02-26T10:40:41.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8931e00640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918004d40 0x7f89341dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8932800640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 0x7f8934147280 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f892c037040 tx=0x7f892c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c039030 con 0x7f89341ea1b0 2024-02-26T10:40:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c03a040 con 0x7f89341ea1b0 2024-02-26T10:40:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c003990 con 0x7f89341ea1b0 2024-02-26T10:40:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.816+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89340c5620 con 0x7f89341ea1b0 2024-02-26T10:40:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.817+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89340c5be0 con 0x7f89341ea1b0 2024-02-26T10:40:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.818+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89340c5e20 con 0x7f89341ea1b0 2024-02-26T10:40:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.819+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f892c045030 con 0x7f89341ea1b0 2024-02-26T10:40:41.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.820+0000 7f892a000640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f891406b4d0 0x7f891406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:41.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.820+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f892c0abe90 con 0x7f89341ea1b0 2024-02-26T10:40:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.820+0000 7f8931e00640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f891406b4d0 0x7f891406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.821+0000 7f8931e00640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f891406b4d0 0x7f891406d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f89240059f0 tx=0x7f8924001580 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:41.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:41.824+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f892c04c020 con 0x7f89341ea1b0 2024-02-26T10:40:42.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.023+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8934074a30 con 0x7f89341ea1b0 2024-02-26T10:40:42.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.025+0000 7f892a000640 1 -- 172.21.15.67:0/1745118364 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12819 (secure 0 0 0) 0x7f892c0ad090 con 0x7f89341ea1b0 2024-02-26T10:40:42.025 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:42.025 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":51,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:40.464556+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:42.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f891406b4d0 msgr2=0x7f891406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f891406b4d0 0x7f891406d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f89240059f0 tx=0x7f8924001580 comp rx=0 tx=0).stop 2024-02-26T10:40:42.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 msgr2=0x7f8934147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 0x7f8934147280 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f892c037040 tx=0x7f892c009750 comp rx=0 tx=0).stop 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 shutdown_connections 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89341ea1b0 0x7f8934147280 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89340c4eb0 0x7f89341e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f891406b4d0 0x7f891406d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 --2- 172.21.15.67:0/1745118364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8918004d40 0x7f89341dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.028+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 >> 172.21.15.67:0/1745118364 conn(0x7f89340c63f0 msgr2=0x7f8934150a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.029+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 shutdown_connections 2024-02-26T10:40:42.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.029+0000 7f8933200640 1 -- 172.21.15.67:0/1745118364 wait complete. 2024-02-26T10:40:42.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:42.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.273+0000 7f3844e00640 1 Processor -- start 2024-02-26T10:40:42.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.273+0000 7f3844e00640 1 -- start start 2024-02-26T10:40:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f3844e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38400c25a0 0x7f38400c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f3844e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c1eb0 0x7f384014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f3844e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38400c1be0 con 0x7f384014f260 2024-02-26T10:40:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f3844e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38400c1750 con 0x7f38400c25a0 2024-02-26T10:40:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f3844e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400c3660 con 0x7f38400c1eb0 2024-02-26T10:40:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f383f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38400c25a0 0x7f38400c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f383f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38400c25a0 0x7f38400c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35824/0 (socket says 172.21.15.67:35824) 2024-02-26T10:40:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f383fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f384014f260 0x7f3840151980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42848/0 (socket says 172.21.15.67:42848) 2024-02-26T10:40:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.274+0000 7f383f400640 1 -- 172.21.15.67:0/3388671169 learned_addr learned my addr 172.21.15.67:0/3388671169 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.275+0000 7f383ea00640 1 --2- 172.21.15.67:0/3388671169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c1eb0 0x7f384014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.275+0000 7f383f400640 1 -- 172.21.15.67:0/3388671169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c1eb0 msgr2=0x7f384014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.275+0000 7f383f400640 1 --2- 172.21.15.67:0/3388671169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c1eb0 0x7f384014ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.275+0000 7f383f400640 1 -- 172.21.15.67:0/3388671169 >> v1:172.21.15.67:6789/0 conn(0x7f384014f260 legacy=0x7f3840151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.275+0000 7f383f400640 1 -- 172.21.15.67:0/3388671169 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400c0e20 con 0x7f38400c25a0 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.275+0000 7f383e000640 1 -- 172.21.15.67:0/3388671169 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489464639 0 0) 0x7f38400c1be0 con 0x7f384014f260 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.276+0000 7f383f400640 1 --2- 172.21.15.67:0/3388671169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c25a0 0x7f38400c2e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3830009870 tx=0x7f383002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e646fa1ee28ee20 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.276+0000 7f383e000640 1 -- 172.21.15.67:0/3388671169 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830035020 con 0x7f38400c25a0 2024-02-26T10:40:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.276+0000 7f383e000640 1 -- 172.21.15.67:0/3388671169 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3830031640 con 0x7f38400c25a0 2024-02-26T10:40:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.276+0000 7f383e000640 1 -- 172.21.15.67:0/3388671169 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383002d040 con 0x7f38400c25a0 2024-02-26T10:40:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 -- 172.21.15.67:0/3388671169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c25a0 msgr2=0x7f38400c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 --2- 172.21.15.67:0/3388671169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c25a0 0x7f38400c2e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3830009870 tx=0x7f383002b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 -- 172.21.15.67:0/3388671169 shutdown_connections 2024-02-26T10:40:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 --2- 172.21.15.67:0/3388671169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c1eb0 0x7f384014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 --2- 172.21.15.67:0/3388671169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c25a0 0x7f38400c2e10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 -- 172.21.15.67:0/3388671169 >> 172.21.15.67:0/3388671169 conn(0x7f38400c6380 msgr2=0x7f38400c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 -- 172.21.15.67:0/3388671169 shutdown_connections 2024-02-26T10:40:42.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.277+0000 7f3844e00640 1 -- 172.21.15.67:0/3388671169 wait complete. 2024-02-26T10:40:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.279+0000 7f3844e00640 1 Processor -- start 2024-02-26T10:40:42.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.279+0000 7f3844e00640 1 -- start start 2024-02-26T10:40:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f3844e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 0x7f38401eb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f3844e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c25a0 0x7f384014bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f3844e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f384014f260 0x7f384014a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f3844e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400c3350 con 0x7f38400c1eb0 2024-02-26T10:40:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f3844e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400c13e0 con 0x7f384014f260 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f3844e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400c2980 con 0x7f38400c25a0 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f383f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 0x7f38401eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f383f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 0x7f38401eb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35836/0 (socket says 172.21.15.67:35836) 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f383f400640 1 -- 172.21.15.67:0/854700707 learned_addr learned my addr 172.21.15.67:0/854700707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f383fe00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f384014f260 0x7f384014a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f383f400640 1 -- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c25a0 msgr2=0x7f384014bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.280+0000 7f383ea00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c25a0 0x7f384014bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f383f400640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c25a0 0x7f384014bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f383f400640 1 -- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f384014f260 msgr2=0x7f384014a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f383f400640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f384014f260 0x7f384014a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f383f400640 1 -- 172.21.15.67:0/854700707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840149230 con 0x7f38400c1eb0 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f383fe00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f384014f260 0x7f384014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f383f400640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 0x7f38401eb200 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f383002b930 tx=0x7f38300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830040070 con 0x7f38400c1eb0 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383003e070 con 0x7f38400c1eb0 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830035040 con 0x7f38400c1eb0 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3840149510 con 0x7f38400c1eb0 2024-02-26T10:40:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.281+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3840149ad0 con 0x7f38400c1eb0 2024-02-26T10:40:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.283+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38400678e0 con 0x7f38400c1eb0 2024-02-26T10:40:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.284+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3830038020 con 0x7f38400c1eb0 2024-02-26T10:40:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.285+0000 7f3837400640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f382006b4d0 0x7f382006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.285+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38300ab8f0 con 0x7f38400c1eb0 2024-02-26T10:40:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.285+0000 7f383ea00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f382006b4d0 0x7f382006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.286+0000 7f383ea00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f382006b4d0 0x7f382006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3840155cc0 tx=0x7f382c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:42.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.289+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f383007aab0 con 0x7f38400c1eb0 2024-02-26T10:40:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.491+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f384014e740 con 0x7f38400c1eb0 2024-02-26T10:40:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.492+0000 7f3837400640 1 -- 172.21.15.67:0/854700707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f383007a3b0 con 0x7f38400c1eb0 2024-02-26T10:40:42.491 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f382006b4d0 msgr2=0x7f382006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f382006b4d0 0x7f382006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3840155cc0 tx=0x7f382c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 msgr2=0x7f38401eb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 0x7f38401eb200 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f383002b930 tx=0x7f38300046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 shutdown_connections 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f382006b4d0 0x7f382006d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f384014f260 0x7f384014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38400c25a0 0x7f384014bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 --2- 172.21.15.67:0/854700707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38400c1eb0 0x7f38401eb200 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 >> 172.21.15.67:0/854700707 conn(0x7f38400c6380 msgr2=0x7f38400bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 shutdown_connections 2024-02-26T10:40:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.495+0000 7f3844e00640 1 -- 172.21.15.67:0/854700707 wait complete. 2024-02-26T10:40:42.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:40:42.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.736+0000 7fd58a200640 1 Processor -- start 2024-02-26T10:40:42.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.737+0000 7fd58a200640 1 -- start start 2024-02-26T10:40:42.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd58a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd584152380 0x7fd5840c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd58a200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5840c7020 0x7fd584155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd58a200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd58406d290 0x7fd5840c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd58a200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd584152760 con 0x7fd5840c7020 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd58a200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd584155cd0 con 0x7fd58406d290 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd58a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840c7550 con 0x7fd584152380 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.738+0000 7fd583e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5840c7020 0x7fd584155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5840c7020 0x7fd584155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35852/0 (socket says 172.21.15.67:35852) 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 -- 172.21.15.67:0/2128578629 learned_addr learned my addr 172.21.15.67:0/2128578629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd588e00640 1 --2- 172.21.15.67:0/2128578629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd584152380 0x7fd5840c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd589800640 1 --2- 172.21.15.67:0/2128578629 >> v2:172.21.15.82:3300/0 conn(0x7fd58406d290 0x7fd5840c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 -- 172.21.15.67:0/2128578629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd584152380 msgr2=0x7fd5840c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 --2- 172.21.15.67:0/2128578629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd584152380 0x7fd5840c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 -- 172.21.15.67:0/2128578629 >> v2:172.21.15.82:3300/0 conn(0x7fd58406d290 msgr2=0x7fd5840c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 --2- 172.21.15.67:0/2128578629 >> v2:172.21.15.82:3300/0 conn(0x7fd58406d290 0x7fd5840c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.739+0000 7fd583e00640 1 -- 172.21.15.67:0/2128578629 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840c7830 con 0x7fd5840c7020 2024-02-26T10:40:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.740+0000 7fd583e00640 1 --2- 172.21.15.67:0/2128578629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5840c7020 0x7fd584155790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd58406dd70 tx=0x7fd57402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2700c86eae899d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd583400640 1 -- 172.21.15.67:0/2128578629 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57402c9f0 con 0x7fd5840c7020 2024-02-26T10:40:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd583400640 1 -- 172.21.15.67:0/2128578629 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57402cb90 con 0x7fd5840c7020 2024-02-26T10:40:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd583400640 1 -- 172.21.15.67:0/2128578629 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740343f0 con 0x7fd5840c7020 2024-02-26T10:40:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 -- 172.21.15.67:0/2128578629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5840c7020 msgr2=0x7fd584155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 --2- 172.21.15.67:0/2128578629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5840c7020 0x7fd584155790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd58406dd70 tx=0x7fd57402be50 comp rx=0 tx=0).stop 2024-02-26T10:40:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 -- 172.21.15.67:0/2128578629 shutdown_connections 2024-02-26T10:40:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 --2- 172.21.15.67:0/2128578629 >> v2:172.21.15.82:3300/0 conn(0x7fd58406d290 0x7fd5840c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 --2- 172.21.15.67:0/2128578629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5840c7020 0x7fd584155790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 --2- 172.21.15.67:0/2128578629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd584152380 0x7fd5840c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.741+0000 7fd58a200640 1 -- 172.21.15.67:0/2128578629 >> 172.21.15.67:0/2128578629 conn(0x7fd5840ac1a0 msgr2=0x7fd5840ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.742+0000 7fd58a200640 1 -- 172.21.15.67:0/2128578629 shutdown_connections 2024-02-26T10:40:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.742+0000 7fd58a200640 1 -- 172.21.15.67:0/2128578629 wait complete. 2024-02-26T10:40:42.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.743+0000 7fd58a200640 1 Processor -- start 2024-02-26T10:40:42.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.743+0000 7fd58a200640 1 -- start start 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.744+0000 7fd58a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 0x7fd5840c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.744+0000 7fd58a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5840c7020 0x7fd5840c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.744+0000 7fd58a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd584152380 0x7fd5840c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.744+0000 7fd58a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58406d720 con 0x7fd58406d290 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.744+0000 7fd58a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58406cd70 con 0x7fd584152380 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.744+0000 7fd58a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58406d8a0 con 0x7fd5840c7020 2024-02-26T10:40:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 0x7fd5840c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 0x7fd5840c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35858/0 (socket says 172.21.15.67:35858) 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 -- 172.21.15.67:0/1060362667 learned_addr learned my addr 172.21.15.67:0/1060362667 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd589800640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd584152380 0x7fd5840c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd583e00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5840c7020 0x7fd5840c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 -- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5840c7020 msgr2=0x7fd5840c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5840c7020 0x7fd5840c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 -- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd584152380 msgr2=0x7fd5840c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd584152380 0x7fd5840c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 -- 172.21.15.67:0/1060362667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840c18f0 con 0x7fd58406d290 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd589800640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd584152380 0x7fd5840c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.745+0000 7fd588e00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 0x7fd5840c3400 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00d830 tx=0x7fd56c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.746+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00b6c0 con 0x7fd58406d290 2024-02-26T10:40:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.746+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c019070 con 0x7fd58406d290 2024-02-26T10:40:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.746+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58414ba10 con 0x7fd58406d290 2024-02-26T10:40:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.746+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00ba80 con 0x7fd58406d290 2024-02-26T10:40:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.746+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5840c09e0 con 0x7fd58406d290 2024-02-26T10:40:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.747+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd584074a30 con 0x7fd58406d290 2024-02-26T10:40:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.748+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd56c01e460 con 0x7fd58406d290 2024-02-26T10:40:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.749+0000 7fd580c00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd56006b4d0 0x7fd56006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:42.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.749+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd56c08a600 con 0x7fd58406d290 2024-02-26T10:40:42.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.750+0000 7fd583e00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd56006b4d0 0x7fd56006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.751+0000 7fd583e00640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd56006b4d0 0x7fd56006d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd574035000 tx=0x7fd574030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:42.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.753+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd56c059730 con 0x7fd58406d290 2024-02-26T10:40:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.954+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd5840c81a0 con 0x7fd58406d290 2024-02-26T10:40:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.955+0000 7fd580c00640 1 -- 172.21.15.67:0/1060362667 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd56c059030 con 0x7fd58406d290 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:42.954 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:40:42.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd56006b4d0 msgr2=0x7fd56006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd56006b4d0 0x7fd56006d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd574035000 tx=0x7fd574030000 comp rx=0 tx=0).stop 2024-02-26T10:40:42.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 msgr2=0x7fd5840c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:42.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 0x7fd5840c3400 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00d830 tx=0x7fd56c00dcf0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 shutdown_connections 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd56006b4d0 0x7fd56006d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd584152380 0x7fd5840c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5840c7020 0x7fd5840c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 --2- 172.21.15.67:0/1060362667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd58406d290 0x7fd5840c3400 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.957+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 >> 172.21.15.67:0/1060362667 conn(0x7fd5840ac1a0 msgr2=0x7fd58414cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.958+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 shutdown_connections 2024-02-26T10:40:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:42.958+0000 7fd58a200640 1 -- 172.21.15.67:0/1060362667 wait complete. 2024-02-26T10:40:42.974 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T10:40:42.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T10:40:43.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.202+0000 7f5907e00640 1 Processor -- start 2024-02-26T10:40:43.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.202+0000 7f5907e00640 1 -- start start 2024-02-26T10:40:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59080c25a0 0x7f59080c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f59080c1eb0 0x7f590814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080c1be0 con 0x7f590814f260 2024-02-26T10:40:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080c1750 con 0x7f59080c1eb0 2024-02-26T10:40:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080c3660 con 0x7f59080c25a0 2024-02-26T10:40:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f590814f260 0x7f5908151980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42856/0 (socket says 172.21.15.67:42856) 2024-02-26T10:40:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5907400640 1 -- 172.21.15.67:0/728672636 learned_addr learned my addr 172.21.15.67:0/728672636 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5906000640 1 --2- 172.21.15.67:0/728672636 >> v2:172.21.15.82:3300/0 conn(0x7f59080c1eb0 0x7f590814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.204+0000 7f5906a00640 1 --2- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59080c25a0 0x7f59080c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5905600640 1 -- 172.21.15.67:0/728672636 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95195699 0 0) 0x7f59080c1be0 con 0x7f590814f260 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5905600640 1 -- 172.21.15.67:0/728672636 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ec003520 con 0x7f590814f260 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5906000640 1 -- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59080c25a0 msgr2=0x7f59080c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5906000640 1 --2- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59080c25a0 0x7f59080c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5905600640 1 -- 172.21.15.67:0/728672636 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3489396947 0 0) 0x7f58ec003520 con 0x7f590814f260 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5906000640 1 -- 172.21.15.67:0/728672636 >> v1:172.21.15.67:6789/0 conn(0x7f590814f260 legacy=0x7f5908151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.205+0000 7f5906000640 1 -- 172.21.15.67:0/728672636 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080c0e20 con 0x7f59080c1eb0 2024-02-26T10:40:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.206+0000 7f5906000640 1 --2- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f590814ed20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097a0 tx=0x7f58f002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ac7c12755c6953 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5905600640 1 -- 172.21.15.67:0/728672636 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0037070 con 0x7f59080c1eb0 2024-02-26T10:40:43.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5905600640 1 -- 172.21.15.67:0/728672636 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0030510 con 0x7f59080c1eb0 2024-02-26T10:40:43.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5905600640 1 -- 172.21.15.67:0/728672636 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900001670 con 0x7f59080c1eb0 2024-02-26T10:40:43.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5907e00640 1 -- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 msgr2=0x7f590814ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5907e00640 1 --2- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f590814ed20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58f00097a0 tx=0x7f58f002b710 comp rx=0 tx=0).stop 2024-02-26T10:40:43.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5907e00640 1 -- 172.21.15.67:0/728672636 shutdown_connections 2024-02-26T10:40:43.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5907e00640 1 --2- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f590814ed20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5907e00640 1 --2- 172.21.15.67:0/728672636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59080c25a0 0x7f59080c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.207+0000 7f5907e00640 1 -- 172.21.15.67:0/728672636 >> 172.21.15.67:0/728672636 conn(0x7f59080c6380 msgr2=0x7f59080c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.208+0000 7f5907e00640 1 -- 172.21.15.67:0/728672636 shutdown_connections 2024-02-26T10:40:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.208+0000 7f5907e00640 1 -- 172.21.15.67:0/728672636 wait complete. 2024-02-26T10:40:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.209+0000 7f5907e00640 1 Processor -- start 2024-02-26T10:40:43.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.209+0000 7f5907e00640 1 -- start start 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.210+0000 7f5907e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f5908143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.210+0000 7f5907e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 0x7f59081edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.210+0000 7f5907e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f590814f260 0x7f59081ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.210+0000 7f5907e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080c3350 con 0x7f59080c25a0 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.210+0000 7f5907e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080c13e0 con 0x7f59080c1eb0 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.210+0000 7f5907e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080c2980 con 0x7f590814f260 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 0x7f59081edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 0x7f59081edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35872/0 (socket says 172.21.15.67:35872) 2024-02-26T10:40:43.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 -- 172.21.15.67:0/2282688661 learned_addr learned my addr 172.21.15.67:0/2282688661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906a00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f5908143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5907400640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f590814f260 0x7f59081ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 -- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f590814f260 msgr2=0x7f59081ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f590814f260 0x7f59081ee110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 -- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 msgr2=0x7f5908143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f5908143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 -- 172.21.15.67:0/2282688661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59081ee800 con 0x7f59080c25a0 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906a00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f5908143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.211+0000 7f5906000640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 0x7f59081edbd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f58f002b750 tx=0x7f58f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.212+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0041020 con 0x7f59080c25a0 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.212+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f003c890 con 0x7f59080c25a0 2024-02-26T10:40:43.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.212+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5908152ef0 con 0x7f59080c25a0 2024-02-26T10:40:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.212+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0037040 con 0x7f59080c25a0 2024-02-26T10:40:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.212+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59081534b0 con 0x7f59080c25a0 2024-02-26T10:40:43.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.213+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5908074a30 con 0x7f59080c25a0 2024-02-26T10:40:43.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.214+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f0044020 con 0x7f59080c25a0 2024-02-26T10:40:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.216+0000 7f58fea00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58e406b4d0 0x7f58e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.216+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f58f00ab910 con 0x7f59080c25a0 2024-02-26T10:40:43.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.216+0000 7f5906a00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58e406b4d0 0x7f58e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.217+0000 7f5906a00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58e406b4d0 0x7f58e406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5908070ce0 tx=0x7f58f800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:43.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.220+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58f004b030 con 0x7f59080c25a0 2024-02-26T10:40:43.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.420+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f59080a96b0 con 0x7f59080c25a0 2024-02-26T10:40:43.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.571+0000 7f58fea00640 1 -- 172.21.15.67:0/2282688661 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f58f003ae20 con 0x7f59080c25a0 2024-02-26T10:40:43.570 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-02-26T10:40:43.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58e406b4d0 msgr2=0x7f58e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58e406b4d0 0x7f58e406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5908070ce0 tx=0x7f58f800b040 comp rx=0 tx=0).stop 2024-02-26T10:40:43.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 msgr2=0x7f59081edbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 0x7f59081edbd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f58f002b750 tx=0x7f58f00046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 shutdown_connections 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f590814f260 0x7f59081ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59080c25a0 0x7f59081edbd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f58e406b4d0 0x7f58e406d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 --2- 172.21.15.67:0/2282688661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59080c1eb0 0x7f5908143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.576+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 >> 172.21.15.67:0/2282688661 conn(0x7f59080c6380 msgr2=0x7f5908071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:43.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.577+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 shutdown_connections 2024-02-26T10:40:43.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.577+0000 7f5907e00640 1 -- 172.21.15.67:0/2282688661 wait complete. 2024-02-26T10:40:43.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:40:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.825+0000 7f65c8200640 1 Processor -- start 2024-02-26T10:40:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.825+0000 7f65c8200640 1 -- start start 2024-02-26T10:40:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.827+0000 7f65c8200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65c014cf30 0x7f65c01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.827+0000 7f65c8200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c006ea10 0x7f65c006ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.827+0000 7f65c8200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c01542d0 con 0x7f65c006b160 2024-02-26T10:40:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.827+0000 7f65c8200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c0154570 con 0x7f65c014cf30 2024-02-26T10:40:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.827+0000 7f65c8200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0154850 con 0x7f65c006ea10 2024-02-26T10:40:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.827+0000 7f65c6200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f65c006b160 0x7f65c0155050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42860/0 (socket says 172.21.15.67:42860) 2024-02-26T10:40:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65c6200640 1 -- 172.21.15.67:0/2055407115 learned_addr learned my addr 172.21.15.67:0/2055407115 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65c4e00640 1 --2- 172.21.15.67:0/2055407115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c006ea10 0x7f65c006ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65c5800640 1 --2- 172.21.15.67:0/2055407115 >> v2:172.21.15.82:3300/0 conn(0x7f65c014cf30 0x7f65c01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65bfe00640 1 -- 172.21.15.67:0/2055407115 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613414898 0 0) 0x7f65c01542d0 con 0x7f65c006b160 2024-02-26T10:40:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65bfe00640 1 -- 172.21.15.67:0/2055407115 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a8003520 con 0x7f65c006b160 2024-02-26T10:40:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65c4e00640 1 -- 172.21.15.67:0/2055407115 >> v2:172.21.15.82:3300/0 conn(0x7f65c014cf30 msgr2=0x7f65c01538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65c4e00640 1 --2- 172.21.15.67:0/2055407115 >> v2:172.21.15.82:3300/0 conn(0x7f65c014cf30 0x7f65c01538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.828+0000 7f65c4e00640 1 -- 172.21.15.67:0/2055407115 >> v1:172.21.15.67:6789/0 conn(0x7f65c006b160 legacy=0x7f65c0155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.829+0000 7f65bfe00640 1 -- 172.21.15.67:0/2055407115 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1443684941 0 0) 0x7f65a8003520 con 0x7f65c006b160 2024-02-26T10:40:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.829+0000 7f65c4e00640 1 -- 172.21.15.67:0/2055407115 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0154b30 con 0x7f65c006ea10 2024-02-26T10:40:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.830+0000 7f65c4e00640 1 --2- 172.21.15.67:0/2055407115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c006ea10 0x7f65c006ac20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0097a0 tx=0x7f65ac02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1448ec67ae0da2e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.830+0000 7f65bfe00640 1 -- 172.21.15.67:0/2055407115 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac037070 con 0x7f65c006ea10 2024-02-26T10:40:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.830+0000 7f65bfe00640 1 -- 172.21.15.67:0/2055407115 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac030510 con 0x7f65c006ea10 2024-02-26T10:40:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.830+0000 7f65bfe00640 1 -- 172.21.15.67:0/2055407115 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8001670 con 0x7f65c006ea10 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 -- 172.21.15.67:0/2055407115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c006ea10 msgr2=0x7f65c006ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 --2- 172.21.15.67:0/2055407115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c006ea10 0x7f65c006ac20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0097a0 tx=0x7f65ac02b710 comp rx=0 tx=0).stop 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 -- 172.21.15.67:0/2055407115 shutdown_connections 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 --2- 172.21.15.67:0/2055407115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c006ea10 0x7f65c006ac20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 --2- 172.21.15.67:0/2055407115 >> v2:172.21.15.82:3300/0 conn(0x7f65c014cf30 0x7f65c01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 -- 172.21.15.67:0/2055407115 >> 172.21.15.67:0/2055407115 conn(0x7f65c00c7160 msgr2=0x7f65c00a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 -- 172.21.15.67:0/2055407115 shutdown_connections 2024-02-26T10:40:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.831+0000 7f65c8200640 1 -- 172.21.15.67:0/2055407115 wait complete. 2024-02-26T10:40:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.833+0000 7f65c8200640 1 Processor -- start 2024-02-26T10:40:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.833+0000 7f65c8200640 1 -- start start 2024-02-26T10:40:43.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c8200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c006b160 0x7f65c0151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c8200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 0x7f65c00c2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c8200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c014cf30 0x7f65c00a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c8200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c006e790 con 0x7f65c006ea10 2024-02-26T10:40:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c8200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c006e610 con 0x7f65c006b160 2024-02-26T10:40:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c8200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0155760 con 0x7f65c014cf30 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 0x7f65c00c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 0x7f65c00c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35882/0 (socket says 172.21.15.67:35882) 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c4e00640 1 -- 172.21.15.67:0/2227925985 learned_addr learned my addr 172.21.15.67:0/2227925985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c6200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c014cf30 0x7f65c00a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.834+0000 7f65c5800640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c006b160 0x7f65c0151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.835+0000 7f65c4e00640 1 -- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c014cf30 msgr2=0x7f65c00a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.835+0000 7f65c4e00640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c014cf30 0x7f65c00a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.835+0000 7f65c4e00640 1 -- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c006b160 msgr2=0x7f65c0151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:43.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.835+0000 7f65c4e00640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c006b160 0x7f65c0151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:43.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.835+0000 7f65c4e00640 1 -- 172.21.15.67:0/2227925985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0150b80 con 0x7f65c006ea10 2024-02-26T10:40:43.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.835+0000 7f65c4e00640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 0x7f65c00c2760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:43.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.836+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac041020 con 0x7f65c006ea10 2024-02-26T10:40:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.836+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac03c870 con 0x7f65c006ea10 2024-02-26T10:40:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.836+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac037040 con 0x7f65c006ea10 2024-02-26T10:40:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.836+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0150e60 con 0x7f65c006ea10 2024-02-26T10:40:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.836+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c00c2ca0 con 0x7f65c006ea10 2024-02-26T10:40:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.837+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c0074a30 con 0x7f65c006ea10 2024-02-26T10:40:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.839+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65ac044020 con 0x7f65c006ea10 2024-02-26T10:40:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.839+0000 7f65bd600640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65a006b500 0x7f65a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.840+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f65ac0abec0 con 0x7f65c006ea10 2024-02-26T10:40:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.840+0000 7f65c5800640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65a006b500 0x7f65a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.841+0000 7f65c5800640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65a006b500 0x7f65a006d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65b40073f0 tx=0x7f65b400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:43.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:43.844+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65ac04b070 con 0x7f65c006ea10 2024-02-26T10:40:44.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.042+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f65c00abaf0 con 0x7f65c006ea10 2024-02-26T10:40:44.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.578+0000 7f65bd600640 1 -- 172.21.15.67:0/2227925985 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7f65ac07af90 con 0x7f65c006ea10 2024-02-26T10:40:44.577 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-02-26T10:40:44.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65a006b500 msgr2=0x7f65a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:44.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65a006b500 0x7f65a006d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65b40073f0 tx=0x7f65b400b040 comp rx=0 tx=0).stop 2024-02-26T10:40:44.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 msgr2=0x7f65c00c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 0x7f65c00c2760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac0046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 shutdown_connections 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c014cf30 0x7f65c00a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65c006ea10 0x7f65c00c2760 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f65a006b500 0x7f65a006d9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 --2- 172.21.15.67:0/2227925985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c006b160 0x7f65c0151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 >> 172.21.15.67:0/2227925985 conn(0x7f65c00c7160 msgr2=0x7f65c00a7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:44.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 shutdown_connections 2024-02-26T10:40:44.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.582+0000 7f65c8200640 1 -- 172.21.15.67:0/2227925985 wait complete. 2024-02-26T10:40:44.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T10:40:44.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.806+0000 7f4ae8800640 1 Processor -- start 2024-02-26T10:40:44.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.806+0000 7f4ae8800640 1 -- start start 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae8800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ae014ce20 0x7f4ae006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae8800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae00c25c0 con 0x7f4ae00c2110 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae8800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae0152ee0 con 0x7f4ae01563c0 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae8800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae01531c0 con 0x7f4ae014ce20 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ae014ce20 0x7f4ae006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ae014ce20 0x7f4ae006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35894/0 (socket says 172.21.15.67:35894) 2024-02-26T10:40:44.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.808+0000 7f4ae5e00640 1 -- 172.21.15.67:0/2568173325 learned_addr learned my addr 172.21.15.67:0/2568173325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.809+0000 7f4ae5e00640 1 -- 172.21.15.67:0/2568173325 >> v1:172.21.15.82:6789/0 conn(0x7f4ae01563c0 legacy=0x7f4ae0156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.809+0000 7f4ae5e00640 1 -- 172.21.15.67:0/2568173325 >> v1:172.21.15.67:6789/0 conn(0x7f4ae00c2110 legacy=0x7f4ae00c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:44.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.809+0000 7f4ae5e00640 1 -- 172.21.15.67:0/2568173325 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae01534a0 con 0x7f4ae014ce20 2024-02-26T10:40:44.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.811+0000 7f4ae5e00640 1 --2- 172.21.15.67:0/2568173325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae006fd20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40098a0 tx=0x7f4ad402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6ee6e9c43a9589 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.811+0000 7f4ae4a00640 1 -- 172.21.15.67:0/2568173325 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4035020 con 0x7f4ae014ce20 2024-02-26T10:40:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.811+0000 7f4ae4a00640 1 -- 172.21.15.67:0/2568173325 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4031640 con 0x7f4ae014ce20 2024-02-26T10:40:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.811+0000 7f4ae4a00640 1 -- 172.21.15.67:0/2568173325 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad402d040 con 0x7f4ae014ce20 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 -- 172.21.15.67:0/2568173325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 msgr2=0x7f4ae006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 --2- 172.21.15.67:0/2568173325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae006fd20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40098a0 tx=0x7f4ad402b8f0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 -- 172.21.15.67:0/2568173325 shutdown_connections 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 --2- 172.21.15.67:0/2568173325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae006fd20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 -- 172.21.15.67:0/2568173325 >> 172.21.15.67:0/2568173325 conn(0x7f4ae00c7b00 msgr2=0x7f4ae00c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 -- 172.21.15.67:0/2568173325 shutdown_connections 2024-02-26T10:40:44.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.812+0000 7f4ae8800640 1 -- 172.21.15.67:0/2568173325 wait complete. 2024-02-26T10:40:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.814+0000 7f4ae8800640 1 Processor -- start 2024-02-26T10:40:44.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.814+0000 7f4ae8800640 1 -- start start 2024-02-26T10:40:44.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae8800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ae00c2110 0x7f4ae0147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:44.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae8800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:44.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae8800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ae01563c0 0x7f4ae01eac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae8800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0153ca0 con 0x7f4ae014ce20 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae8800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae014d200 con 0x7f4ae00c2110 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae8800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae00c2880 con 0x7f4ae01563c0 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae0151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35902/0 (socket says 172.21.15.67:35902) 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae5400640 1 -- 172.21.15.67:0/238027678 learned_addr learned my addr 172.21.15.67:0/238027678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.815+0000 7f4ae5e00640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ae00c2110 0x7f4ae0147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae6800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ae01563c0 0x7f4ae01eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae5400640 1 -- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ae01563c0 msgr2=0x7f4ae01eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae5400640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ae01563c0 0x7f4ae01eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae5400640 1 -- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ae00c2110 msgr2=0x7f4ae0147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae5400640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ae00c2110 0x7f4ae0147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae5400640 1 -- 172.21.15.67:0/238027678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae01482f0 con 0x7f4ae014ce20 2024-02-26T10:40:44.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.816+0000 7f4ae5400640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae0151f00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c480 tx=0x7f4acc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.817+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0052d0 con 0x7f4ae014ce20 2024-02-26T10:40:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.817+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc005470 con 0x7f4ae014ce20 2024-02-26T10:40:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.817+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae00ab860 con 0x7f4ae014ce20 2024-02-26T10:40:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.817+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc012870 con 0x7f4ae014ce20 2024-02-26T10:40:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.817+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae0155750 con 0x7f4ae014ce20 2024-02-26T10:40:44.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.818+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae0074a30 con 0x7f4ae014ce20 2024-02-26T10:40:44.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.819+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4acc012a10 con 0x7f4ae014ce20 2024-02-26T10:40:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.820+0000 7f4ada000640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ac006b4d0 0x7f4ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.820+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4acc089da0 con 0x7f4ae014ce20 2024-02-26T10:40:44.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.821+0000 7f4ae5e00640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ac006b4d0 0x7f4ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:44.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.822+0000 7f4ae5e00640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ac006b4d0 0x7f4ac006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4036000 tx=0x7f4ad403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:44.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:44.825+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4acc058d10 con 0x7f4ae014ce20 2024-02-26T10:40:45.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.063+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4ae006f140 con 0x7f4ae014ce20 2024-02-26T10:40:45.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.600+0000 7f4ada000640 1 -- 172.21.15.67:0/238027678 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4acc058610 con 0x7f4ae014ce20 2024-02-26T10:40:45.601 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 8 and data pool 9 2024-02-26T10:40:45.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ac006b4d0 msgr2=0x7f4ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ac006b4d0 0x7f4ac006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4036000 tx=0x7f4ad403c040 comp rx=0 tx=0).stop 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 msgr2=0x7f4ae0151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae0151f00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c480 tx=0x7f4acc00c940 comp rx=0 tx=0).stop 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 shutdown_connections 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ae01563c0 0x7f4ae01eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ae014ce20 0x7f4ae0151f00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4ac006b4d0 0x7f4ac006d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 --2- 172.21.15.67:0/238027678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ae00c2110 0x7f4ae0147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.604+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 >> 172.21.15.67:0/238027678 conn(0x7f4ae00c7b00 msgr2=0x7f4ae00c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:45.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.605+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 shutdown_connections 2024-02-26T10:40:45.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.605+0000 7f4ae8800640 1 -- 172.21.15.67:0/238027678 wait complete. 2024-02-26T10:40:45.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.825+0000 7f0da6a00640 1 Processor -- start 2024-02-26T10:40:45.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.825+0000 7f0da6a00640 1 -- start start 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0da6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0da0156230 0x7f0da00ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0da6a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da0152e50 con 0x7f0da006e410 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0da6a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da0153130 con 0x7f0da0150300 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0da6a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da01506e0 con 0x7f0da0156230 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0d97400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0da006e410 0x7f0da00aaba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42868/0 (socket says 172.21.15.67:42868) 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0d97400640 1 -- 172.21.15.67:0/28865831 learned_addr learned my addr 172.21.15.67:0/28865831 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:45.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0da4a00640 1 --2- 172.21.15.67:0/28865831 >> v2:172.21.15.67:3300/0 conn(0x7f0da0156230 0x7f0da00ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:45.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0d9f400640 1 -- 172.21.15.67:0/28865831 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489346227 0 0) 0x7f0da0152e50 con 0x7f0da006e410 2024-02-26T10:40:45.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.827+0000 7f0d9f400640 1 -- 172.21.15.67:0/28865831 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d84003520 con 0x7f0da006e410 2024-02-26T10:40:45.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.828+0000 7f0da4a00640 1 -- 172.21.15.67:0/28865831 >> v1:172.21.15.82:6789/0 conn(0x7f0da0150300 legacy=0x7f0da00c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.828+0000 7f0da4a00640 1 -- 172.21.15.67:0/28865831 >> v1:172.21.15.67:6789/0 conn(0x7f0da006e410 legacy=0x7f0da00aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.828+0000 7f0da4a00640 1 -- 172.21.15.67:0/28865831 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0153410 con 0x7f0da0156230 2024-02-26T10:40:45.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.829+0000 7f0da4a00640 1 --2- 172.21.15.67:0/28865831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da0156230 0x7f0da00ab2b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009d90 tx=0x7f0d9802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb371d309209f5b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.829+0000 7f0d9f400640 1 -- 172.21.15.67:0/28865831 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98039070 con 0x7f0da0156230 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.829+0000 7f0d9f400640 1 -- 172.21.15.67:0/28865831 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9802cbd0 con 0x7f0da0156230 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.829+0000 7f0d9f400640 1 -- 172.21.15.67:0/28865831 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98034440 con 0x7f0da0156230 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 -- 172.21.15.67:0/28865831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da0156230 msgr2=0x7f0da00ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 --2- 172.21.15.67:0/28865831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da0156230 0x7f0da00ab2b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009d90 tx=0x7f0d9802be20 comp rx=0 tx=0).stop 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 -- 172.21.15.67:0/28865831 shutdown_connections 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 --2- 172.21.15.67:0/28865831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da0156230 0x7f0da00ab2b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 -- 172.21.15.67:0/28865831 >> 172.21.15.67:0/28865831 conn(0x7f0da00a84a0 msgr2=0x7f0da00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 -- 172.21.15.67:0/28865831 shutdown_connections 2024-02-26T10:40:45.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.830+0000 7f0da6a00640 1 -- 172.21.15.67:0/28865831 wait complete. 2024-02-26T10:40:45.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.832+0000 7f0da6a00640 1 Processor -- start 2024-02-26T10:40:45.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.832+0000 7f0da6a00640 1 -- start start 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 0x7f0da0149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da6a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0da0150300 0x7f0da0072b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0da0156230 0x7f0da00bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da6a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00ab7f0 con 0x7f0da006e410 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da6a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0153a60 con 0x7f0da0156230 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da6a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da006dd80 con 0x7f0da0150300 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 0x7f0da0149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 0x7f0da0149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35914/0 (socket says 172.21.15.67:35914) 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 -- 172.21.15.67:0/3817317675 learned_addr learned my addr 172.21.15.67:0/3817317675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da4a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0da0156230 0x7f0da00bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d97400640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0da0150300 0x7f0da0072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 -- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0da0150300 msgr2=0x7f0da0072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0da0150300 0x7f0da0072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 -- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0da0156230 msgr2=0x7f0da00bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0da0156230 0x7f0da00bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0d9fe00640 1 -- 172.21.15.67:0/3817317675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0149450 con 0x7f0da006e410 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.833+0000 7f0da4a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0da0156230 0x7f0da00bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.834+0000 7f0d9fe00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 0x7f0da0149ca0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c480 tx=0x7f0d9000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.834+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900052d0 con 0x7f0da006e410 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.834+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d90005470 con 0x7f0da006e410 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.834+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90012870 con 0x7f0da006e410 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.834+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da0149730 con 0x7f0da006e410 2024-02-26T10:40:45.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.834+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da00c01e0 con 0x7f0da006e410 2024-02-26T10:40:45.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.835+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da00714b0 con 0x7f0da006e410 2024-02-26T10:40:45.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.836+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d90012a10 con 0x7f0da006e410 2024-02-26T10:40:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.837+0000 7f0d9cc00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.837+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d90089e10 con 0x7f0da006e410 2024-02-26T10:40:45.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.838+0000 7f0d97400640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:45.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.839+0000 7f0d97400640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0da01e8150 tx=0x7f0d88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:45.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:45.842+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d90058d00 con 0x7f0da006e410 2024-02-26T10:40:46.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.037+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0da00c0bd0 con 0x7f0da006e410 2024-02-26T10:40:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.040+0000 7f0d9cc00640 1 -- 172.21.15.67:0/3817317675 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16606 (secure 0 0 0) 0x7f0da00c0bd0 con 0x7f0da006e410 2024-02-26T10:40:46.041 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":54,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:45.064500+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d7c06b4d0 msgr2=0x7f0d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0da01e8150 tx=0x7f0d88009040 comp rx=0 tx=0).stop 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 msgr2=0x7f0da0149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 0x7f0da0149ca0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c480 tx=0x7f0d9000c940 comp rx=0 tx=0).stop 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 shutdown_connections 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0da0156230 0x7f0da00bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0da0150300 0x7f0da0072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 --2- 172.21.15.67:0/3817317675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0da006e410 0x7f0da0149ca0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 >> 172.21.15.67:0/3817317675 conn(0x7f0da00a84a0 msgr2=0x7f0da00aaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 shutdown_connections 2024-02-26T10:40:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.043+0000 7f0da6a00640 1 -- 172.21.15.67:0/3817317675 wait complete. 2024-02-26T10:40:46.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:46.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.260+0000 7f7033000640 1 Processor -- start 2024-02-26T10:40:46.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.261+0000 7f7033000640 1 -- start start 2024-02-26T10:40:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.262+0000 7f7033000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f702c0c6b50 0x7f702c0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.262+0000 7f7033000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c158b20 con 0x7f702c152670 2024-02-26T10:40:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.262+0000 7f7033000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c0c07d0 con 0x7f702c0c4be0 2024-02-26T10:40:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.262+0000 7f7033000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702c0c5c60 con 0x7f702c0c6b50 2024-02-26T10:40:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.262+0000 7f702be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f702c152670 0x7f702c158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42884/0 (socket says 172.21.15.67:42884) 2024-02-26T10:40:46.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.262+0000 7f702be00640 1 -- 172.21.15.67:0/1982146623 learned_addr learned my addr 172.21.15.67:0/1982146623 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924724347 0 0) 0x7f702c158b20 con 0x7f702c152670 2024-02-26T10:40:46.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f700c003520 con 0x7f702c152670 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f7031000640 1 --2- 172.21.15.67:0/1982146623 >> v2:172.21.15.82:3300/0 conn(0x7f702c0c6b50 0x7f702c0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3548029423 0 0) 0x7f700c003520 con 0x7f702c152670 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 >> v2:172.21.15.82:3300/0 conn(0x7f702c0c6b50 msgr2=0x7f702c0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 --2- 172.21.15.67:0/1982146623 >> v2:172.21.15.82:3300/0 conn(0x7f702c0c6b50 0x7f702c0c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 >> v1:172.21.15.82:6789/0 conn(0x7f702c0c4be0 legacy=0x7f702c0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0c0ab0 con 0x7f702c152670 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f701c001670 con 0x7f702c152670 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.263+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 >> v1:172.21.15.67:6789/0 conn(0x7f702c152670 legacy=0x7f702c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702aa00640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c0039b0 0x7f700c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702aa00640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c004800 0x7f700c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702aa00640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c0052e0 0x7f700c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0c5c60 con 0x7f700c004800 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00c1b0 con 0x7f700c0052e0 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00c3b0 con 0x7f700c0039b0 2024-02-26T10:40:46.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702b400640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c004800 0x7f700c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702be00640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c0039b0 0x7f700c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702b400640 1 -- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c0039b0 msgr2=0x7f700c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702b400640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f700c0039b0 0x7f700c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702b400640 1 -- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c0052e0 msgr2=0x7f700c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702b400640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c0052e0 0x7f700c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f702b400640 1 -- 172.21.15.67:0/1982146623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c003520 con 0x7f700c004800 2024-02-26T10:40:46.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.264+0000 7f7031000640 1 -- 172.21.15.67:0/1982146623 reap_dead start 2024-02-26T10:40:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.265+0000 7f702b400640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c004800 0x7f700c004c80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7014001e20 tx=0x7f700c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668ea6d2249b6025 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.265+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014001a70 con 0x7f700c004800 2024-02-26T10:40:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.265+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014003b70 con 0x7f700c004800 2024-02-26T10:40:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.265+0000 7f702aa00640 1 -- 172.21.15.67:0/1982146623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140050d0 con 0x7f700c004800 2024-02-26T10:40:46.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 -- 172.21.15.67:0/1982146623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c004800 msgr2=0x7f700c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c004800 0x7f700c004c80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7014001e20 tx=0x7f700c005830 comp rx=0 tx=0).stop 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 -- 172.21.15.67:0/1982146623 shutdown_connections 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 --2- 172.21.15.67:0/1982146623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f700c004800 0x7f700c004c80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 -- 172.21.15.67:0/1982146623 >> 172.21.15.67:0/1982146623 conn(0x7f702c072350 msgr2=0x7f702c072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 -- 172.21.15.67:0/1982146623 shutdown_connections 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.266+0000 7f7033000640 1 -- 172.21.15.67:0/1982146623 wait complete. 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.268+0000 7f7033000640 1 Processor -- start 2024-02-26T10:40:46.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.268+0000 7f7033000640 1 -- start start 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7033000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c004800 0x7f702c1e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7033000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f702c1536b0 0x7f702c14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7033000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 0x7f702c1f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7033000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0c0480 con 0x7f702c0c2fe0 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7033000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0c5f40 con 0x7f700c004800 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7033000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0c63d0 con 0x7f702c1536b0 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7031000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 0x7f702c1f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7031000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 0x7f702c1f2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35932/0 (socket says 172.21.15.67:35932) 2024-02-26T10:40:46.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f7031000640 1 -- 172.21.15.67:0/2477977315 learned_addr learned my addr 172.21.15.67:0/2477977315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f702be00640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c004800 0x7f702c1e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.269+0000 7f702b400640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f702c1536b0 0x7f702c14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7031000640 1 -- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f702c1536b0 msgr2=0x7f702c14d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7031000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f702c1536b0 0x7f702c14d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7031000640 1 -- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c004800 msgr2=0x7f702c1e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7031000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c004800 0x7f702c1e0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7031000640 1 -- 172.21.15.67:0/2477977315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c1eec30 con 0x7f702c0c2fe0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7031000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 0x7f702c1f2110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f702c0cb550 tx=0x7f702400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024017020 con 0x7f702c0c2fe0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7024004030 con 0x7f702c0c2fe0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c1eeed0 con 0x7f702c0c2fe0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024004420 con 0x7f702c0c2fe0 2024-02-26T10:40:46.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.270+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c1ef490 con 0x7f702c0c2fe0 2024-02-26T10:40:46.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.271+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c074a30 con 0x7f702c0c2fe0 2024-02-26T10:40:46.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.272+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f702401a020 con 0x7f702c0c2fe0 2024-02-26T10:40:46.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.274+0000 7f7023e00640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f700806b500 0x7f700806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.274+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7024089fe0 con 0x7f702c0c2fe0 2024-02-26T10:40:46.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.274+0000 7f702be00640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f700806b500 0x7f700806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.275+0000 7f702be00640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f700806b500 0x7f700806d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f702c06a430 tx=0x7f701c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:46.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.278+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7024021030 con 0x7f702c0c2fe0 2024-02-26T10:40:46.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.467+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f702c0c67c0 con 0x7f702c0c2fe0 2024-02-26T10:40:46.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.469+0000 7f7023e00640 1 -- 172.21.15.67:0/2477977315 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16606 (secure 0 0 0) 0x7f7024058e90 con 0x7f702c0c2fe0 2024-02-26T10:40:46.469 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:46.469 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":54,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:45.064500+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f700806b500 msgr2=0x7f700806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f700806b500 0x7f700806d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f702c06a430 tx=0x7f701c009040 comp rx=0 tx=0).stop 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 msgr2=0x7f702c1f2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 0x7f702c1f2110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f702c0cb550 tx=0x7f702400ec00 comp rx=0 tx=0).stop 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 shutdown_connections 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f702c0c2fe0 0x7f702c1f2110 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f702c1536b0 0x7f702c14d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f700806b500 0x7f700806d9c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 --2- 172.21.15.67:0/2477977315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c004800 0x7f702c1e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 >> 172.21.15.67:0/2477977315 conn(0x7f702c072350 msgr2=0x7f702c14fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:46.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 shutdown_connections 2024-02-26T10:40:46.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.471+0000 7f7033000640 1 -- 172.21.15.67:0/2477977315 wait complete. 2024-02-26T10:40:46.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T10:40:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.713+0000 7f69c6200640 1 Processor -- start 2024-02-26T10:40:46.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.713+0000 7f69c6200640 1 -- start start 2024-02-26T10:40:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.715+0000 7f69c6200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f69c0071fd0 0x7f69c0072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.715+0000 7f69c6200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c006dc80 con 0x7f69c0154cf0 2024-02-26T10:40:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.715+0000 7f69c6200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c006df60 con 0x7f69c01523b0 2024-02-26T10:40:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.715+0000 7f69c6200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c0152790 con 0x7f69c0071fd0 2024-02-26T10:40:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.715+0000 7f69bfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f69c0154cf0 0x7f69c00c3300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42892/0 (socket says 172.21.15.67:42892) 2024-02-26T10:40:46.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.715+0000 7f69bfe00640 1 -- 172.21.15.67:0/3687753727 learned_addr learned my addr 172.21.15.67:0/3687753727 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:46.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69c5800640 1 --2- 172.21.15.67:0/3687753727 >> v2:172.21.15.82:3300/0 conn(0x7f69c0071fd0 0x7f69c0072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037633818 0 0) 0x7f69c006dc80 con 0x7f69c0154cf0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69a8003520 con 0x7f69c0154cf0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620691959 0 0) 0x7f69a8003520 con 0x7f69c0154cf0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 >> v2:172.21.15.82:3300/0 conn(0x7f69c0071fd0 msgr2=0x7f69c0072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 --2- 172.21.15.67:0/3687753727 >> v2:172.21.15.82:3300/0 conn(0x7f69c0071fd0 0x7f69c0072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 >> v1:172.21.15.82:6789/0 conn(0x7f69c01523b0 legacy=0x7f69c00c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c006e240 con 0x7f69c0154cf0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f69ac001670 con 0x7f69c0154cf0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 >> v1:172.21.15.67:6789/0 conn(0x7f69c0154cf0 legacy=0x7f69c00c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69a80039b0 0x7f69a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a8004800 0x7f69a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69a80052e0 0x7f69a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c0152790 con 0x7f69a80039b0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a800c1b0 con 0x7f69a80052e0 2024-02-26T10:40:46.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a800c3b0 con 0x7f69a8004800 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.716+0000 7f69c4e00640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69a80039b0 0x7f69a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.717+0000 7f69c4e00640 1 -- 172.21.15.67:0/3687753727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a8004800 msgr2=0x7f69a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.717+0000 7f69c4e00640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a8004800 0x7f69a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.717+0000 7f69c4e00640 1 -- 172.21.15.67:0/3687753727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69a80052e0 msgr2=0x7f69a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.717+0000 7f69c4e00640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69a80052e0 0x7f69a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.717+0000 7f69c4e00640 1 -- 172.21.15.67:0/3687753727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8003520 con 0x7f69a80039b0 2024-02-26T10:40:46.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.717+0000 7f69c5800640 1 -- 172.21.15.67:0/3687753727 reap_dead start 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.718+0000 7f69c4e00640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69a80039b0 0x7f69a8003dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001720 tx=0x7f69a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368e5e829f87236 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.718+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4001370 con 0x7f69a80039b0 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.718+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b40052d0 con 0x7f69a80039b0 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.718+0000 7f69bf400640 1 -- 172.21.15.67:0/3687753727 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4003830 con 0x7f69a80039b0 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 -- 172.21.15.67:0/3687753727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69a80039b0 msgr2=0x7f69a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69a80039b0 0x7f69a8003dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001720 tx=0x7f69a8005830 comp rx=0 tx=0).stop 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 -- 172.21.15.67:0/3687753727 shutdown_connections 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 --2- 172.21.15.67:0/3687753727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69a80039b0 0x7f69a8003dd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 -- 172.21.15.67:0/3687753727 >> 172.21.15.67:0/3687753727 conn(0x7f69c00c6380 msgr2=0x7f69c00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:46.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 -- 172.21.15.67:0/3687753727 shutdown_connections 2024-02-26T10:40:46.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.719+0000 7f69c6200640 1 -- 172.21.15.67:0/3687753727 wait complete. 2024-02-26T10:40:46.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.721+0000 7f69c6200640 1 Processor -- start 2024-02-26T10:40:46.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.721+0000 7f69c6200640 1 -- start start 2024-02-26T10:40:46.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c6200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a80039b0 0x7f69c00c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c6200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69c014f0b0 0x7f69c0143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 0x7f69c01deb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c006e800 con 0x7f69c0143eb0 2024-02-26T10:40:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c6200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c006da30 con 0x7f69c014f0b0 2024-02-26T10:40:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c6200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00c08e0 con 0x7f69a80039b0 2024-02-26T10:40:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 0x7f69c01deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 0x7f69c01deb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35958/0 (socket says 172.21.15.67:35958) 2024-02-26T10:40:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 -- 172.21.15.67:0/44940457 learned_addr learned my addr 172.21.15.67:0/44940457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 -- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a80039b0 msgr2=0x7f69c00c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a80039b0 0x7f69c00c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 -- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69c014f0b0 msgr2=0x7f69c0143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69c014f0b0 0x7f69c0143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.722+0000 7f69c5800640 1 -- 172.21.15.67:0/44940457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c00c5930 con 0x7f69c0143eb0 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.723+0000 7f69c5800640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 0x7f69c01deb10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f69c00c71d0 tx=0x7f69b800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.723+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8018020 con 0x7f69c0143eb0 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.723+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b800bb30 con 0x7f69c0143eb0 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.723+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8014400 con 0x7f69c0143eb0 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.723+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c00c5c10 con 0x7f69c0143eb0 2024-02-26T10:40:46.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.723+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c01e11b0 con 0x7f69c0143eb0 2024-02-26T10:40:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.724+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c00678e0 con 0x7f69c0143eb0 2024-02-26T10:40:46.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.725+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69b801b020 con 0x7f69c0143eb0 2024-02-26T10:40:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.726+0000 7f69bcc00640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f69a006b500 0x7f69a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.727+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69b808a770 con 0x7f69c0143eb0 2024-02-26T10:40:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.727+0000 7f69c4e00640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f69a006b500 0x7f69a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:46.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.728+0000 7f69c4e00640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f69a006b500 0x7f69a006d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f69b4033000 tx=0x7f69b402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:46.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.732+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69b8022020 con 0x7f69c0143eb0 2024-02-26T10:40:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:46.976+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f69c00bf840 con 0x7f69c0143eb0 2024-02-26T10:40:47.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.640+0000 7f69bcc00640 1 -- 172.21.15.67:0/44940457 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f69b8059660 con 0x7f69c0143eb0 2024-02-26T10:40:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f69a006b500 msgr2=0x7f69a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f69a006b500 0x7f69a006d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f69b4033000 tx=0x7f69b402f000 comp rx=0 tx=0).stop 2024-02-26T10:40:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 msgr2=0x7f69c01deb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 0x7f69c01deb10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f69c00c71d0 tx=0x7f69b800eee0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 shutdown_connections 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69c0143eb0 0x7f69c01deb10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f69a006b500 0x7f69a006d9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69c014f0b0 0x7f69c0143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 --2- 172.21.15.67:0/44940457 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69a80039b0 0x7f69c00c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.644+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 >> 172.21.15.67:0/44940457 conn(0x7f69c00c6380 msgr2=0x7f69c0072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.645+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 shutdown_connections 2024-02-26T10:40:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.645+0000 7f69c6200640 1 -- 172.21.15.67:0/44940457 wait complete. 2024-02-26T10:40:47.669 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:40:47.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-26T10:40:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.898+0000 7f7488000640 1 Processor -- start 2024-02-26T10:40:47.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.899+0000 7f7488000640 1 -- start start 2024-02-26T10:40:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.900+0000 7f7488000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f74800a6150 0x7f74800c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.900+0000 7f7488000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f748006e860 0x7f74800a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.900+0000 7f7488000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7480155170 0x7f7480155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.900+0000 7f7488000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7480152b10 con 0x7f7480155170 2024-02-26T10:40:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.900+0000 7f7488000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7480152680 con 0x7f748006e860 2024-02-26T10:40:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.900+0000 7f7488000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7480151d70 con 0x7f74800a6150 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7480155170 0x7f7480155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7480155170 0x7f7480155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35970/0 (socket says 172.21.15.67:35970) 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 -- 172.21.15.67:0/1987545333 learned_addr learned my addr 172.21.15.67:0/1987545333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7485600640 1 --2- 172.21.15.67:0/1987545333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f74800a6150 0x7f74800c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7484c00640 1 --2- 172.21.15.67:0/1987545333 >> v2:172.21.15.82:3300/0 conn(0x7f748006e860 0x7f74800a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 -- 172.21.15.67:0/1987545333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f74800a6150 msgr2=0x7f74800c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:47.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 --2- 172.21.15.67:0/1987545333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f74800a6150 0x7f74800c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 -- 172.21.15.67:0/1987545333 >> v2:172.21.15.82:3300/0 conn(0x7f748006e860 msgr2=0x7f74800a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:47.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 --2- 172.21.15.67:0/1987545333 >> v2:172.21.15.82:3300/0 conn(0x7f748006e860 0x7f74800a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.901+0000 7f7486000640 1 -- 172.21.15.67:0/1987545333 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480152030 con 0x7f7480155170 2024-02-26T10:40:47.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.902+0000 7f7486000640 1 --2- 172.21.15.67:0/1987545333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7480155170 0x7f7480155600 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f748006f010 tx=0x7f747802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cbaa0778e24ce9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:47.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.903+0000 7f747fe00640 1 -- 172.21.15.67:0/1987545333 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747802c9e0 con 0x7f7480155170 2024-02-26T10:40:47.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.903+0000 7f747fe00640 1 -- 172.21.15.67:0/1987545333 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747802cb80 con 0x7f7480155170 2024-02-26T10:40:47.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.903+0000 7f747fe00640 1 -- 172.21.15.67:0/1987545333 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780343f0 con 0x7f7480155170 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 -- 172.21.15.67:0/1987545333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7480155170 msgr2=0x7f7480155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 --2- 172.21.15.67:0/1987545333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7480155170 0x7f7480155600 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f748006f010 tx=0x7f747802be40 comp rx=0 tx=0).stop 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 -- 172.21.15.67:0/1987545333 shutdown_connections 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 --2- 172.21.15.67:0/1987545333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7480155170 0x7f7480155600 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 --2- 172.21.15.67:0/1987545333 >> v2:172.21.15.82:3300/0 conn(0x7f748006e860 0x7f74800a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 --2- 172.21.15.67:0/1987545333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f74800a6150 0x7f74800c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 -- 172.21.15.67:0/1987545333 >> 172.21.15.67:0/1987545333 conn(0x7f74800c6da0 msgr2=0x7f74800c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 -- 172.21.15.67:0/1987545333 shutdown_connections 2024-02-26T10:40:47.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.904+0000 7f7488000640 1 -- 172.21.15.67:0/1987545333 wait complete. 2024-02-26T10:40:47.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.906+0000 7f7488000640 1 Processor -- start 2024-02-26T10:40:47.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.906+0000 7f7488000640 1 -- start start 2024-02-26T10:40:47.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7488000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f748006e860 0x7f7480148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7488000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 0x7f7480151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7488000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7480155170 0x7f74801496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7488000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480155b40 con 0x7f74800a6150 2024-02-26T10:40:47.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7488000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800c2220 con 0x7f7480155170 2024-02-26T10:40:47.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7488000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7480155cc0 con 0x7f748006e860 2024-02-26T10:40:47.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 0x7f7480151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 0x7f7480151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35986/0 (socket says 172.21.15.67:35986) 2024-02-26T10:40:47.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 -- 172.21.15.67:0/3021597688 learned_addr learned my addr 172.21.15.67:0/3021597688 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:47.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7485600640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f748006e860 0x7f7480148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7486000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7480155170 0x7f74801496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 -- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f748006e860 msgr2=0x7f7480148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:47.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f748006e860 0x7f7480148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 -- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7480155170 msgr2=0x7f74801496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7480155170 0x7f74801496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7484c00640 1 -- 172.21.15.67:0/3021597688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480151940 con 0x7f74800a6150 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.907+0000 7f7486000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7480155170 0x7f74801496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.908+0000 7f7484c00640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 0x7f7480151400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d800 tx=0x7f746c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.908+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c018030 con 0x7f74800a6150 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.908+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c0096a0 con 0x7f74800a6150 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.908+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c009870 con 0x7f74800a6150 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.908+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74801eb660 con 0x7f74800a6150 2024-02-26T10:40:47.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.908+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748014b360 con 0x7f74800a6150 2024-02-26T10:40:47.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.909+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7480074a30 con 0x7f74800a6150 2024-02-26T10:40:47.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.910+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f746c01b070 con 0x7f74800a6150 2024-02-26T10:40:47.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.911+0000 7f747d600640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f746006b500 0x7f746006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:47.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.911+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f746c08a3e0 con 0x7f74800a6150 2024-02-26T10:40:47.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.912+0000 7f7485600640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f746006b500 0x7f746006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:47.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.913+0000 7f7485600640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f746006b500 0x7f746006d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7474006fd0 tx=0x7f7474008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:47.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:47.916+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f746c0592f0 con 0x7f74800a6150 2024-02-26T10:40:48.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.159+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7480073280 con 0x7f74800a6150 2024-02-26T10:40:48.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.643+0000 7f747d600640 1 -- 172.21.15.67:0/3021597688 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f746c058bf0 con 0x7f74800a6150 2024-02-26T10:40:48.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.647+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f746006b500 msgr2=0x7f746006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.647+0000 7f7488000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f746006b500 0x7f746006d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7474006fd0 tx=0x7f7474008040 comp rx=0 tx=0).stop 2024-02-26T10:40:48.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.647+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 msgr2=0x7f7480151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.647+0000 7f7488000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 0x7f7480151400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d800 tx=0x7f746c00dcc0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 shutdown_connections 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f746006b500 0x7f746006d9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7480155170 0x7f74801496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74800a6150 0x7f7480151400 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 --2- 172.21.15.67:0/3021597688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f748006e860 0x7f7480148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 >> 172.21.15.67:0/3021597688 conn(0x7f74800c6da0 msgr2=0x7f74800a8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 shutdown_connections 2024-02-26T10:40:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.648+0000 7f7488000640 1 -- 172.21.15.67:0/3021597688 wait complete. 2024-02-26T10:40:48.669 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-26T10:40:48.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.893+0000 7f93ef200640 1 Processor -- start 2024-02-26T10:40:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.893+0000 7f93ef200640 1 -- start start 2024-02-26T10:40:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ef200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93f006ac20 0x7f93f00a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ef200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006db00 0x7f93f00abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ef200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f006b460 con 0x7f93f006b000 2024-02-26T10:40:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ef200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f006dfb0 con 0x7f93f006ac20 2024-02-26T10:40:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ef200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00a9a80 con 0x7f93f006db00 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ede00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93f006ac20 0x7f93f00a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ede00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93f006ac20 0x7f93f00a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35998/0 (socket says 172.21.15.67:35998) 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ede00640 1 -- 172.21.15.67:0/1601520650 learned_addr learned my addr 172.21.15.67:0/1601520650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ee800640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006db00 0x7f93f00abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.895+0000 7f93ede00640 1 -- 172.21.15.67:0/1601520650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006db00 msgr2=0x7f93f00abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.896+0000 7f93ede00640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006db00 0x7f93f00abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.896+0000 7f93ede00640 1 -- 172.21.15.67:0/1601520650 >> v1:172.21.15.67:6789/0 conn(0x7f93f006b000 legacy=0x7f93f0153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.896+0000 7f93ede00640 1 -- 172.21.15.67:0/1601520650 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00a9d60 con 0x7f93f006ac20 2024-02-26T10:40:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.896+0000 7f93ee800640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006db00 0x7f93f00abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.897+0000 7f93ede00640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f00a94a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009870 tx=0x7f93e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9c19557a17dd54 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.897+0000 7f93eca00640 1 -- 172.21.15.67:0/1601520650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0034070 con 0x7f93f006ac20 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.897+0000 7f93eca00640 1 -- 172.21.15.67:0/1601520650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0030840 con 0x7f93f006ac20 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.897+0000 7f93eca00640 1 -- 172.21.15.67:0/1601520650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0030c10 con 0x7f93f006ac20 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 -- 172.21.15.67:0/1601520650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 msgr2=0x7f93f00a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f00a94a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009870 tx=0x7f93e002b980 comp rx=0 tx=0).stop 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 -- 172.21.15.67:0/1601520650 shutdown_connections 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006db00 0x7f93f00abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 --2- 172.21.15.67:0/1601520650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f00a94a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 -- 172.21.15.67:0/1601520650 >> 172.21.15.67:0/1601520650 conn(0x7f93f00c63f0 msgr2=0x7f93f00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 -- 172.21.15.67:0/1601520650 shutdown_connections 2024-02-26T10:40:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.898+0000 7f93ef200640 1 -- 172.21.15.67:0/1601520650 wait complete. 2024-02-26T10:40:48.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.900+0000 7f93ef200640 1 Processor -- start 2024-02-26T10:40:48.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.900+0000 7f93ef200640 1 -- start start 2024-02-26T10:40:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ef200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f0147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ef200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006b000 0x7f93f00c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ef200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93f006db00 0x7f93f00c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ef200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00ac490 con 0x7f93f006ac20 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ef200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00aba30 con 0x7f93f006db00 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ef200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f0150410 con 0x7f93f006b000 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ede00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f0147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ede00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f0147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36000/0 (socket says 172.21.15.67:36000) 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ede00640 1 -- 172.21.15.67:0/1070739550 learned_addr learned my addr 172.21.15.67:0/1070739550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.901+0000 7f93ed400640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006b000 0x7f93f00c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ede00640 1 -- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006b000 msgr2=0x7f93f00c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ede00640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006b000 0x7f93f00c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ede00640 1 -- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93f006db00 msgr2=0x7f93f00c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ede00640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93f006db00 0x7f93f00c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ede00640 1 -- 172.21.15.67:0/1070739550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f01510a0 con 0x7f93f006ac20 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ede00640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f0147a00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0048070 con 0x7f93f006ac20 2024-02-26T10:40:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0030640 con 0x7f93f006ac20 2024-02-26T10:40:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0034040 con 0x7f93f006ac20 2024-02-26T10:40:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.902+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f01eb750 con 0x7f93f006ac20 2024-02-26T10:40:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.903+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f01ea990 con 0x7f93f006ac20 2024-02-26T10:40:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.904+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f0074a30 con 0x7f93f006ac20 2024-02-26T10:40:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.905+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93e004b070 con 0x7f93f006ac20 2024-02-26T10:40:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.905+0000 7f93e6000640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f93d006b500 0x7f93d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.906+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93e00ac340 con 0x7f93f006ac20 2024-02-26T10:40:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.906+0000 7f93ed400640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f93d006b500 0x7f93d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.907+0000 7f93ed400640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f93d006b500 0x7f93d006d9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93f00c2a30 tx=0x7f93dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:48.910+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93e007b120 con 0x7f93f006ac20 2024-02-26T10:40:49.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.149+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93f006b9b0 con 0x7f93f006ac20 2024-02-26T10:40:49.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.150+0000 7f93e6000640 1 -- 172.21.15.67:0/1070739550 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f93e007aa20 con 0x7f93f006ac20 2024-02-26T10:40:49.150 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:49.150 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-26T10:40:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 32 2024-02-26T10:40:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f93d006b500 msgr2=0x7f93d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f93d006b500 0x7f93d006d9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93f00c2a30 tx=0x7f93dc009040 comp rx=0 tx=0).stop 2024-02-26T10:40:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 msgr2=0x7f93f0147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f0147a00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0004730 comp rx=0 tx=0).stop 2024-02-26T10:40:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 shutdown_connections 2024-02-26T10:40:49.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f93d006b500 0x7f93d006d9c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93f006db00 0x7f93f00c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93f006b000 0x7f93f00c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 --2- 172.21.15.67:0/1070739550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93f006ac20 0x7f93f0147a00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 >> 172.21.15.67:0/1070739550 conn(0x7f93f00c63f0 msgr2=0x7f93f014f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 shutdown_connections 2024-02-26T10:40:49.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.153+0000 7f93ef200640 1 -- 172.21.15.67:0/1070739550 wait complete. 2024-02-26T10:40:49.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.394+0000 7fe8be800640 1 Processor -- start 2024-02-26T10:40:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.394+0000 7fe8be800640 1 -- start start 2024-02-26T10:40:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.395+0000 7fe8be800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8b80c6350 0x7fe8b8155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.395+0000 7fe8be800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8b8158a60 0x7fe8b8158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.395+0000 7fe8be800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b8158580 con 0x7fe8b814b950 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.395+0000 7fe8be800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b8155e60 con 0x7fe8b80c6350 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8be800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b80c2f80 con 0x7fe8b8158a60 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8b80c6350 0x7fe8b8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8b80c6350 0x7fe8b8155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40618/0 (socket says 172.21.15.67:40618) 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bca00640 1 -- 172.21.15.67:0/2507831527 learned_addr learned my addr 172.21.15.67:0/2507831527 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bde00640 1 --2- 172.21.15.67:0/2507831527 >> v2:172.21.15.82:3300/0 conn(0x7fe8b8158a60 0x7fe8b8158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bca00640 1 -- 172.21.15.67:0/2507831527 >> v2:172.21.15.82:3300/0 conn(0x7fe8b8158a60 msgr2=0x7fe8b8158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bca00640 1 --2- 172.21.15.67:0/2507831527 >> v2:172.21.15.82:3300/0 conn(0x7fe8b8158a60 0x7fe8b8158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.396+0000 7fe8bca00640 1 -- 172.21.15.67:0/2507831527 >> v1:172.21.15.82:6789/0 conn(0x7fe8b814b950 legacy=0x7fe8b80c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.397+0000 7fe8bca00640 1 -- 172.21.15.67:0/2507831527 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80c3260 con 0x7fe8b80c6350 2024-02-26T10:40:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.398+0000 7fe8bca00640 1 --2- 172.21.15.67:0/2507831527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b80c6350 0x7fe8b8155920 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009870 tx=0x7fe8a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0eeb593ec03966 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.398+0000 7fe8b7e00640 1 -- 172.21.15.67:0/2507831527 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4035020 con 0x7fe8b80c6350 2024-02-26T10:40:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.398+0000 7fe8b7e00640 1 -- 172.21.15.67:0/2507831527 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4031870 con 0x7fe8b80c6350 2024-02-26T10:40:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.398+0000 7fe8b7e00640 1 -- 172.21.15.67:0/2507831527 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4031c40 con 0x7fe8b80c6350 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 -- 172.21.15.67:0/2507831527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b80c6350 msgr2=0x7fe8b8155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 --2- 172.21.15.67:0/2507831527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b80c6350 0x7fe8b8155920 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009870 tx=0x7fe8a402b910 comp rx=0 tx=0).stop 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 -- 172.21.15.67:0/2507831527 shutdown_connections 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 --2- 172.21.15.67:0/2507831527 >> v2:172.21.15.82:3300/0 conn(0x7fe8b8158a60 0x7fe8b8158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 --2- 172.21.15.67:0/2507831527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b80c6350 0x7fe8b8155920 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 -- 172.21.15.67:0/2507831527 >> 172.21.15.67:0/2507831527 conn(0x7fe8b806b040 msgr2=0x7fe8b806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 -- 172.21.15.67:0/2507831527 shutdown_connections 2024-02-26T10:40:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.399+0000 7fe8be800640 1 -- 172.21.15.67:0/2507831527 wait complete. 2024-02-26T10:40:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.401+0000 7fe8be800640 1 Processor -- start 2024-02-26T10:40:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.401+0000 7fe8be800640 1 -- start start 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8be800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b80c6350 0x7fe8b8070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8be800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 0x7fe8b806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8be800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8b8158a60 0x7fe8b81eb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8be800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80c3820 con 0x7fe8b814b950 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8be800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b814bd30 con 0x7fe8b80c6350 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8be800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80c39a0 con 0x7fe8b8158a60 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 0x7fe8b806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 0x7fe8b806fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40620/0 (socket says 172.21.15.67:40620) 2024-02-26T10:40:49.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 -- 172.21.15.67:0/1413127202 learned_addr learned my addr 172.21.15.67:0/1413127202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bd400640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b80c6350 0x7fe8b8070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 -- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8b8158a60 msgr2=0x7fe8b81eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8b8158a60 0x7fe8b81eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 -- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b80c6350 msgr2=0x7fe8b8070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b80c6350 0x7fe8b8070a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.402+0000 7fe8bca00640 1 -- 172.21.15.67:0/1413127202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8070f50 con 0x7fe8b814b950 2024-02-26T10:40:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.403+0000 7fe8bca00640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 0x7fe8b806fd80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402b950 tx=0x7fe8a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.403+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4035040 con 0x7fe8b814b950 2024-02-26T10:40:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.403+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4030d00 con 0x7fe8b814b950 2024-02-26T10:40:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.403+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40456c0 con 0x7fe8b814b950 2024-02-26T10:40:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.403+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b80a88a0 con 0x7fe8b814b950 2024-02-26T10:40:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.403+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b80702c0 con 0x7fe8b814b950 2024-02-26T10:40:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.404+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b8074a30 con 0x7fe8b814b950 2024-02-26T10:40:49.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.406+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8a4038020 con 0x7fe8b814b950 2024-02-26T10:40:49.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.407+0000 7fe8b5600640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe89806b500 0x7fe89806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.407+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a40ac410 con 0x7fe8b814b950 2024-02-26T10:40:49.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.408+0000 7fe8bd400640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe89806b500 0x7fe89806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.409+0000 7fe8bd400640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe89806b500 0x7fe89806d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac007fd0 tx=0x7fe8ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:49.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.412+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8a407b280 con 0x7fe8b814b950 2024-02-26T10:40:49.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.610+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8b80c2910 con 0x7fe8b814b950 2024-02-26T10:40:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.613+0000 7fe8b5600640 1 -- 172.21.15.67:0/1413127202 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fe8a407ab80 con 0x7fe8b814b950 2024-02-26T10:40:49.613 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:49.614 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:46.616937+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe89806b500 msgr2=0x7fe89806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe89806b500 0x7fe89806d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac007fd0 tx=0x7fe8ac009040 comp rx=0 tx=0).stop 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 msgr2=0x7fe8b806fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 0x7fe8b806fd80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402b950 tx=0x7fe8a40046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 shutdown_connections 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe8b8158a60 0x7fe8b81eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8b814b950 0x7fe8b806fd80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe89806b500 0x7fe89806d9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 --2- 172.21.15.67:0/1413127202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8b80c6350 0x7fe8b8070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 >> 172.21.15.67:0/1413127202 conn(0x7fe8b806b040 msgr2=0x7fe8b806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 shutdown_connections 2024-02-26T10:40:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.616+0000 7fe8be800640 1 -- 172.21.15.67:0/1413127202 wait complete. 2024-02-26T10:40:49.639 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:40:49.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:40:49.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.850+0000 7ff456c00640 1 Processor -- start 2024-02-26T10:40:49.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.851+0000 7ff456c00640 1 -- start start 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.852+0000 7ff456c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff45014b8f0 0x7ff4500c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.852+0000 7ff456c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4500c2e50 0x7ff450070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.852+0000 7ff456c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff450155a30 con 0x7ff4500c2840 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.852+0000 7ff456c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4500c6a00 con 0x7ff45014b8f0 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.852+0000 7ff456c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff450155d10 con 0x7ff4500c2e50 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff45014b8f0 0x7ff4500c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff45014b8f0 0x7ff4500c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40624/0 (socket says 172.21.15.67:40624) 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 -- 172.21.15.67:0/109894076 learned_addr learned my addr 172.21.15.67:0/109894076 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:49.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff454e00640 1 --2- 172.21.15.67:0/109894076 >> v2:172.21.15.82:3300/0 conn(0x7ff4500c2e50 0x7ff450070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 -- 172.21.15.67:0/109894076 >> v2:172.21.15.82:3300/0 conn(0x7ff4500c2e50 msgr2=0x7ff450070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:49.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 --2- 172.21.15.67:0/109894076 >> v2:172.21.15.82:3300/0 conn(0x7ff4500c2e50 0x7ff450070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 -- 172.21.15.67:0/109894076 >> v1:172.21.15.67:6789/0 conn(0x7ff4500c2840 legacy=0x7ff450070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.853+0000 7ff455800640 1 -- 172.21.15.67:0/109894076 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4500c63b0 con 0x7ff45014b8f0 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.854+0000 7ff455800640 1 --2- 172.21.15.67:0/109894076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff45014b8f0 0x7ff4500c8460 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff444009870 tx=0x7ff44402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d26058641f311e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff44fe00640 1 -- 172.21.15.67:0/109894076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444034070 con 0x7ff45014b8f0 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff44fe00640 1 -- 172.21.15.67:0/109894076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444030850 con 0x7ff45014b8f0 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff44fe00640 1 -- 172.21.15.67:0/109894076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444030c20 con 0x7ff45014b8f0 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff456c00640 1 -- 172.21.15.67:0/109894076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff45014b8f0 msgr2=0x7ff4500c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff456c00640 1 --2- 172.21.15.67:0/109894076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff45014b8f0 0x7ff4500c8460 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff444009870 tx=0x7ff44402b990 comp rx=0 tx=0).stop 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff456c00640 1 -- 172.21.15.67:0/109894076 shutdown_connections 2024-02-26T10:40:49.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff456c00640 1 --2- 172.21.15.67:0/109894076 >> v2:172.21.15.82:3300/0 conn(0x7ff4500c2e50 0x7ff450070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff456c00640 1 --2- 172.21.15.67:0/109894076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff45014b8f0 0x7ff4500c8460 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.855+0000 7ff456c00640 1 -- 172.21.15.67:0/109894076 >> 172.21.15.67:0/109894076 conn(0x7ff45006ac20 msgr2=0x7ff45006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:49.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.856+0000 7ff456c00640 1 -- 172.21.15.67:0/109894076 shutdown_connections 2024-02-26T10:40:49.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.856+0000 7ff456c00640 1 -- 172.21.15.67:0/109894076 wait complete. 2024-02-26T10:40:49.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.857+0000 7ff456c00640 1 Processor -- start 2024-02-26T10:40:49.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.857+0000 7ff456c00640 1 -- start start 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.858+0000 7ff456c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4500c2840 0x7ff4500ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.858+0000 7ff456c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 0x7ff45014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.858+0000 7ff456c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff45014b8f0 0x7ff45014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.858+0000 7ff456c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500c8090 con 0x7ff4500c2e50 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.858+0000 7ff456c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450070e80 con 0x7ff45014b8f0 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.858+0000 7ff456c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500c8210 con 0x7ff4500c2840 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 0x7ff45014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 0x7ff45014e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40626/0 (socket says 172.21.15.67:40626) 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 -- 172.21.15.67:0/4086355517 learned_addr learned my addr 172.21.15.67:0/4086355517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 -- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4500c2840 msgr2=0x7ff4500ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4500c2840 0x7ff4500ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 -- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff45014b8f0 msgr2=0x7ff45014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:49.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff45014b8f0 0x7ff45014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:49.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 -- 172.21.15.67:0/4086355517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4501ee590 con 0x7ff4500c2e50 2024-02-26T10:40:49.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.859+0000 7ff454e00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 0x7ff45014e0d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff44000d800 tx=0x7ff44000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:49.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.860+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000b6c0 con 0x7ff4500c2e50 2024-02-26T10:40:49.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.860+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff440019070 con 0x7ff4500c2e50 2024-02-26T10:40:49.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.860+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000ba80 con 0x7ff4500c2e50 2024-02-26T10:40:49.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.860+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4501ee870 con 0x7ff4500c2e50 2024-02-26T10:40:49.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.860+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4501eee30 con 0x7ff4500c2e50 2024-02-26T10:40:49.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.861+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff450074a30 con 0x7ff4500c2e50 2024-02-26T10:40:49.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.862+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff44000bc20 con 0x7ff4500c2e50 2024-02-26T10:40:49.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.863+0000 7ff44d600640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff43406b4d0 0x7ff43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:49.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.863+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff44008a6c0 con 0x7ff4500c2e50 2024-02-26T10:40:49.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.864+0000 7ff455800640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff43406b4d0 0x7ff43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:49.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.865+0000 7ff455800640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff43406b4d0 0x7ff43406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff444036000 tx=0x7ff444030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:49.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:49.867+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff440059530 con 0x7ff4500c2e50 2024-02-26T10:40:49.922 INFO:tasks.ceph.mds.c.smithi067.stderr:2024-02-26T10:40:49.923+0000 7f64f2e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:40:50.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.063+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff45006b4e0 con 0x7ff43406b4d0 2024-02-26T10:40:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.080+0000 7ff44d600640 1 -- 172.21.15.67:0/4086355517 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238350 (secure 0 0 0) 0x7ff45006b4e0 con 0x7ff43406b4d0 2024-02-26T10:40:50.082 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:50.084 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:40:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.086+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff43406b4d0 msgr2=0x7ff43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.086+0000 7ff456c00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff43406b4d0 0x7ff43406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff444036000 tx=0x7ff444030000 comp rx=0 tx=0).stop 2024-02-26T10:40:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.086+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 msgr2=0x7ff45014e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.086+0000 7ff456c00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 0x7ff45014e0d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff44000d800 tx=0x7ff44000dcc0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 shutdown_connections 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff43406b4d0 0x7ff43406d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff45014b8f0 0x7ff45014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500c2e50 0x7ff45014e0d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 --2- 172.21.15.67:0/4086355517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4500c2840 0x7ff4500ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 >> 172.21.15.67:0/4086355517 conn(0x7ff45006ac20 msgr2=0x7ff45006d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 shutdown_connections 2024-02-26T10:40:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.087+0000 7ff456c00640 1 -- 172.21.15.67:0/4086355517 wait complete. 2024-02-26T10:40:50.105 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-02-26T10:40:48.372956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":14,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":226304,"kb_used_data":5952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748416,"statfs":{"total":773094113280,"available":772862377984,"internally_reserved":0,"allocated":6094848,"data_stored":3260231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.799254"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809381+0000","last_change":"2024-02-26T10:40:45.640652+0000","last_active":"2024-02-26T10:40:46.809381+0000","last_peered":"2024-02-26T10:40:46.809381+0000","last_clean":"2024-02-26T10:40:46.809381+0000","last_became_active":"2024-02-26T10:40:45.637863+0000","last_became_peered":"2024-02-26T10:40:45.637863+0000","last_unstale":"2024-02-26T10:40:46.809381+0000","last_undegraded":"2024-02-26T10:40:46.809381+0000","last_fullsized":"2024-02-26T10:40:46.809381+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:40.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.808899+0000","last_change":"2024-02-26T10:40:45.645804+0000","last_active":"2024-02-26T10:40:46.808899+0000","last_peered":"2024-02-26T10:40:46.808899+0000","last_clean":"2024-02-26T10:40:46.808899+0000","last_became_active":"2024-02-26T10:40:45.645530+0000","last_became_peered":"2024-02-26T10:40:45.645530+0000","last_unstale":"2024-02-26T10:40:46.808899+0000","last_undegraded":"2024-02-26T10:40:46.808899+0000","last_fullsized":"2024-02-26T10:40:46.808899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:50.576149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809214+0000","last_change":"2024-02-26T10:40:45.626981+0000","last_active":"2024-02-26T10:40:46.809214+0000","last_peered":"2024-02-26T10:40:46.809214+0000","last_clean":"2024-02-26T10:40:46.809214+0000","last_became_active":"2024-02-26T10:40:45.626693+0000","last_became_peered":"2024-02-26T10:40:45.626693+0000","last_unstale":"2024-02-26T10:40:46.809214+0000","last_undegraded":"2024-02-26T10:40:46.809214+0000","last_fullsized":"2024-02-26T10:40:46.809214+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:37.225724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809461+0000","last_change":"2024-02-26T10:40:45.637507+0000","last_active":"2024-02-26T10:40:46.809461+0000","last_peered":"2024-02-26T10:40:46.809461+0000","last_clean":"2024-02-26T10:40:46.809461+0000","last_became_active":"2024-02-26T10:40:45.631723+0000","last_became_peered":"2024-02-26T10:40:45.631723+0000","last_unstale":"2024-02-26T10:40:46.809461+0000","last_undegraded":"2024-02-26T10:40:46.809461+0000","last_fullsized":"2024-02-26T10:40:46.809461+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:07.605390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809049+0000","last_change":"2024-02-26T10:40:45.632119+0000","last_active":"2024-02-26T10:40:46.809049+0000","last_peered":"2024-02-26T10:40:46.809049+0000","last_clean":"2024-02-26T10:40:46.809049+0000","last_became_active":"2024-02-26T10:40:45.631803+0000","last_became_peered":"2024-02-26T10:40:45.631803+0000","last_unstale":"2024-02-26T10:40:46.809049+0000","last_undegraded":"2024-02-26T10:40:46.809049+0000","last_fullsized":"2024-02-26T10:40:46.809049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:42.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.808787+0000","last_change":"2024-02-26T10:40:45.625890+0000","last_active":"2024-02-26T10:40:46.808787+0000","last_peered":"2024-02-26T10:40:46.808787+0000","last_clean":"2024-02-26T10:40:46.808787+0000","last_became_active":"2024-02-26T10:40:45.625334+0000","last_became_peered":"2024-02-26T10:40:45.625334+0000","last_unstale":"2024-02-26T10:40:46.808787+0000","last_undegraded":"2024-02-26T10:40:46.808787+0000","last_fullsized":"2024-02-26T10:40:46.808787+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:38:29.332505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:44.571150+0000","last_change":"2024-02-26T10:40:44.571150+0000","last_active":"2024-02-26T10:40:44.571150+0000","last_peered":"2024-02-26T10:40:44.571150+0000","last_clean":"2024-02-26T10:40:44.571150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:44.571150+0000","last_undegraded":"2024-02-26T10:40:44.571150+0000","last_fullsized":"2024-02-26T10:40:44.571150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:40:43.566605+0000","last_change":"2024-02-26T10:40:43.566605+0000","last_active":"2024-02-26T10:40:43.566605+0000","last_peered":"2024-02-26T10:40:43.566605+0000","last_clean":"2024-02-26T10:40:43.566605+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:40:43.566605+0000","last_undegraded":"2024-02-26T10:40:43.566605+0000","last_fullsized":"2024-02-26T10:40:43.566605+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":226,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-02-26T10:40:40.536284+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:40:40.536284+0000","last_peered":"2024-02-26T10:40:40.536284+0000","last_clean":"2024-02-26T10:40:40.536284+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:40:40.536284+0000","last_undegraded":"2024-02-26T10:40:40.536284+0000","last_fullsized":"2024-02-26T10:40:40.536284+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1200128,"data_stored":847153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":256785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":256785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27620,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":606208,"data_stored":256785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":256785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":282000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":256785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":847153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:40:50.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:40:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.375+0000 7f2dc7200640 1 Processor -- start 2024-02-26T10:40:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.375+0000 7f2dc7200640 1 -- start start 2024-02-26T10:40:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.376+0000 7f2dc7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dc80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.376+0000 7f2dc7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc806b460 con 0x7f2dc806b000 2024-02-26T10:40:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.376+0000 7f2dc7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc80a9a80 con 0x7f2dc806db00 2024-02-26T10:40:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.376+0000 7f2dc7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc806dfb0 con 0x7f2dc806ac20 2024-02-26T10:40:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.377+0000 7f2dbfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2dc806b000 0x7f2dc8153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46432/0 (socket says 172.21.15.67:46432) 2024-02-26T10:40:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.377+0000 7f2dbfe00640 1 -- 172.21.15.67:0/3356724880 learned_addr learned my addr 172.21.15.67:0/3356724880 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.377+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dc80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.377+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877036835 0 0) 0x7f2dc806b460 con 0x7f2dc806b000 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.377+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dac003520 con 0x7f2dc806b000 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4139312686 0 0) 0x7f2dac003520 con 0x7f2dc806b000 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 msgr2=0x7f2dc80a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dc80a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 >> v1:172.21.15.82:6789/0 conn(0x7f2dc806db00 legacy=0x7f2dc80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80a9d60 con 0x7f2dc806b000 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f2db0001670 con 0x7f2dc806b000 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dc80a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 >> v1:172.21.15.67:6789/0 conn(0x7f2dc806b000 legacy=0x7f2dc8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac0039e0 0x7f2dac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc806dfb0 con 0x7f2dac004800 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac00bb50 con 0x7f2dac0039e0 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac00bd00 con 0x7f2dc806ac20 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac0039e0 0x7f2dac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dc6800640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.378+0000 7f2dbfe00640 1 -- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 msgr2=0x7f2dac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.379+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.379+0000 7f2dbfe00640 1 -- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac0039e0 msgr2=0x7f2dac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.379+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac0039e0 0x7f2dac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.379+0000 7f2dbfe00640 1 -- 172.21.15.67:0/3356724880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac003520 con 0x7f2dac004800 2024-02-26T10:40:50.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.379+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac0039e0 0x7f2dac003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:50.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.379+0000 7f2dc6800640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc806ac20 0x7f2dac005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dac004c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c0e0 tx=0x7f2dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939b9986ca8cd8fa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0007c40 con 0x7f2dac004800 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db0030070 con 0x7f2dac004800 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dc5400640 1 -- 172.21.15.67:0/3356724880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0005d30 con 0x7f2dac004800 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dc7200640 1 -- 172.21.15.67:0/3356724880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 msgr2=0x7f2dac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dc7200640 1 --2- 172.21.15.67:0/3356724880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dac004c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c0e0 tx=0x7f2dac005830 comp rx=0 tx=0).stop 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.380+0000 7f2dc6800640 1 -- 172.21.15.67:0/3356724880 reap_dead start 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.381+0000 7f2dc7200640 1 -- 172.21.15.67:0/3356724880 shutdown_connections 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.381+0000 7f2dc7200640 1 -- 172.21.15.67:0/3356724880 >> 172.21.15.67:0/3356724880 conn(0x7f2dc80c63f0 msgr2=0x7f2dc80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.381+0000 7f2dc7200640 1 -- 172.21.15.67:0/3356724880 shutdown_connections 2024-02-26T10:40:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.381+0000 7f2dc7200640 1 -- 172.21.15.67:0/3356724880 wait complete. 2024-02-26T10:40:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.383+0000 7f2dc7200640 1 Processor -- start 2024-02-26T10:40:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.383+0000 7f2dc7200640 1 -- start start 2024-02-26T10:40:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dc81df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc8147620 0x7f2dc8146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc814fcb0 0x7f2dc8146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc8150410 con 0x7f2dac004800 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80aba30 con 0x7f2dc8147620 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc8150590 con 0x7f2dc814fcb0 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dc81df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dc81df8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40638/0 (socket says 172.21.15.67:40638) 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 -- 172.21.15.67:0/2541003578 learned_addr learned my addr 172.21.15.67:0/2541003578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc6800640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc814fcb0 0x7f2dc8146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 -- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc814fcb0 msgr2=0x7f2dc8146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc814fcb0 0x7f2dc8146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 -- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc8147620 msgr2=0x7f2dc8146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc8147620 0x7f2dc8146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc8147620 0x7f2dc8146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc5e00640 1 -- 172.21.15.67:0/2541003578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc814a830 con 0x7f2dac004800 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.384+0000 7f2dc6800640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc814fcb0 0x7f2dc8146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.385+0000 7f2dc5e00640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dc81df8d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2db800f560 tx=0x7f2db800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.385+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8018070 con 0x7f2dac004800 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.385+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db8014890 con 0x7f2dac004800 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.385+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8014c40 con 0x7f2dac004800 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.385+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc8145c20 con 0x7f2dac004800 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.385+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc81dfe10 con 0x7f2dac004800 2024-02-26T10:40:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.387+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc8074a30 con 0x7f2dac004800 2024-02-26T10:40:50.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.387+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2db801b070 con 0x7f2dac004800 2024-02-26T10:40:50.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.388+0000 7f2dbe000640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2da406b500 0x7f2da406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.388+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2db808a520 con 0x7f2dac004800 2024-02-26T10:40:50.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.389+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2da406b500 0x7f2da406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.390+0000 7f2dbfe00640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2da406b500 0x7f2da406d9c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2db0034000 tx=0x7f2db0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:50.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.393+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2db8059300 con 0x7f2dac004800 2024-02-26T10:40:50.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.590+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2dc80c4b30 con 0x7f2da406b500 2024-02-26T10:40:50.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.609+0000 7f2dbe000640 1 -- 172.21.15.67:0/2541003578 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251955 (secure 0 0 0) 0x7f2dc80c4b30 con 0x7f2da406b500 2024-02-26T10:40:50.612 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:50.614 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:40:50.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2da406b500 msgr2=0x7f2da406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2da406b500 0x7f2da406d9c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2db0034000 tx=0x7f2db0009030 comp rx=0 tx=0).stop 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 msgr2=0x7f2dc81df8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dc81df8d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2db800f560 tx=0x7f2db800fa20 comp rx=0 tx=0).stop 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 shutdown_connections 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dc814fcb0 0x7f2dc8146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2da406b500 0x7f2da406d9c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc8147620 0x7f2dc8146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 --2- 172.21.15.67:0/2541003578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac004800 0x7f2dc81df8d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 >> 172.21.15.67:0/2541003578 conn(0x7f2dc80c63f0 msgr2=0x7f2dc80c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 shutdown_connections 2024-02-26T10:40:50.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.615+0000 7f2dc7200640 1 -- 172.21.15.67:0/2541003578 wait complete. 2024-02-26T10:40:50.633 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-02-26T10:40:50.373578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230844,"kb_used_data":6344,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743876,"statfs":{"total":773094113280,"available":772857729024,"internally_reserved":0,"allocated":6496256,"data_stored":3441108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001846"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809381+0000","last_change":"2024-02-26T10:40:45.640652+0000","last_active":"2024-02-26T10:40:46.809381+0000","last_peered":"2024-02-26T10:40:46.809381+0000","last_clean":"2024-02-26T10:40:46.809381+0000","last_became_active":"2024-02-26T10:40:45.637863+0000","last_became_peered":"2024-02-26T10:40:45.637863+0000","last_unstale":"2024-02-26T10:40:46.809381+0000","last_undegraded":"2024-02-26T10:40:46.809381+0000","last_fullsized":"2024-02-26T10:40:46.809381+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:40.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732737+0000","last_change":"2024-02-26T10:40:45.635299+0000","last_active":"2024-02-26T10:40:46.732737+0000","last_peered":"2024-02-26T10:40:46.732737+0000","last_clean":"2024-02-26T10:40:46.732737+0000","last_became_active":"2024-02-26T10:40:45.634986+0000","last_became_peered":"2024-02-26T10:40:45.634986+0000","last_unstale":"2024-02-26T10:40:46.732737+0000","last_undegraded":"2024-02-26T10:40:46.732737+0000","last_fullsized":"2024-02-26T10:40:46.732737+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:53.114125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810807+0000","last_change":"2024-02-26T10:40:45.652997+0000","last_active":"2024-02-26T10:40:46.810807+0000","last_peered":"2024-02-26T10:40:46.810807+0000","last_clean":"2024-02-26T10:40:46.810807+0000","last_became_active":"2024-02-26T10:40:45.652579+0000","last_became_peered":"2024-02-26T10:40:45.652579+0000","last_unstale":"2024-02-26T10:40:46.810807+0000","last_undegraded":"2024-02-26T10:40:46.810807+0000","last_fullsized":"2024-02-26T10:40:46.810807+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:14.395806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637525+0000","last_change":"2024-02-26T10:40:45.625722+0000","last_active":"2024-02-26T10:40:46.637525+0000","last_peered":"2024-02-26T10:40:46.637525+0000","last_clean":"2024-02-26T10:40:46.637525+0000","last_became_active":"2024-02-26T10:40:45.625252+0000","last_became_peered":"2024-02-26T10:40:45.625252+0000","last_unstale":"2024-02-26T10:40:46.637525+0000","last_undegraded":"2024-02-26T10:40:46.637525+0000","last_fullsized":"2024-02-26T10:40:46.637525+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.808899+0000","last_change":"2024-02-26T10:40:45.645804+0000","last_active":"2024-02-26T10:40:46.808899+0000","last_peered":"2024-02-26T10:40:46.808899+0000","last_clean":"2024-02-26T10:40:46.808899+0000","last_became_active":"2024-02-26T10:40:45.645530+0000","last_became_peered":"2024-02-26T10:40:45.645530+0000","last_unstale":"2024-02-26T10:40:46.808899+0000","last_undegraded":"2024-02-26T10:40:46.808899+0000","last_fullsized":"2024-02-26T10:40:46.808899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:50.576149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635288+0000","last_change":"2024-02-26T10:40:45.626504+0000","last_active":"2024-02-26T10:40:46.635288+0000","last_peered":"2024-02-26T10:40:46.635288+0000","last_clean":"2024-02-26T10:40:46.635288+0000","last_became_active":"2024-02-26T10:40:45.626190+0000","last_became_peered":"2024-02-26T10:40:45.626190+0000","last_unstale":"2024-02-26T10:40:46.635288+0000","last_undegraded":"2024-02-26T10:40:46.635288+0000","last_fullsized":"2024-02-26T10:40:46.635288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810937+0000","last_change":"2024-02-26T10:40:45.647799+0000","last_active":"2024-02-26T10:40:46.810937+0000","last_peered":"2024-02-26T10:40:46.810937+0000","last_clean":"2024-02-26T10:40:46.810937+0000","last_became_active":"2024-02-26T10:40:45.647134+0000","last_became_peered":"2024-02-26T10:40:45.647134+0000","last_unstale":"2024-02-26T10:40:46.810937+0000","last_undegraded":"2024-02-26T10:40:46.810937+0000","last_fullsized":"2024-02-26T10:40:46.810937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:26.570068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732445+0000","last_change":"2024-02-26T10:40:45.638686+0000","last_active":"2024-02-26T10:40:46.732445+0000","last_peered":"2024-02-26T10:40:46.732445+0000","last_clean":"2024-02-26T10:40:46.732445+0000","last_became_active":"2024-02-26T10:40:45.638372+0000","last_became_peered":"2024-02-26T10:40:45.638372+0000","last_unstale":"2024-02-26T10:40:46.732445+0000","last_undegraded":"2024-02-26T10:40:46.732445+0000","last_fullsized":"2024-02-26T10:40:46.732445+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:09:51.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.813105+0000","last_change":"2024-02-26T10:40:45.632997+0000","last_active":"2024-02-26T10:40:46.813105+0000","last_peered":"2024-02-26T10:40:46.813105+0000","last_clean":"2024-02-26T10:40:46.813105+0000","last_became_active":"2024-02-26T10:40:45.632665+0000","last_became_peered":"2024-02-26T10:40:45.632665+0000","last_unstale":"2024-02-26T10:40:46.813105+0000","last_undegraded":"2024-02-26T10:40:46.813105+0000","last_fullsized":"2024-02-26T10:40:46.813105+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:52.054393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.632821+0000","last_change":"2024-02-26T10:40:45.636607+0000","last_active":"2024-02-26T10:40:46.632821+0000","last_peered":"2024-02-26T10:40:46.632821+0000","last_clean":"2024-02-26T10:40:46.632821+0000","last_became_active":"2024-02-26T10:40:45.636328+0000","last_became_peered":"2024-02-26T10:40:45.636328+0000","last_unstale":"2024-02-26T10:40:46.632821+0000","last_undegraded":"2024-02-26T10:40:46.632821+0000","last_fullsized":"2024-02-26T10:40:46.632821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:26.570068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.633206+0000","last_change":"2024-02-26T10:40:45.628536+0000","last_active":"2024-02-26T10:40:46.633206+0000","last_peered":"2024-02-26T10:40:46.633206+0000","last_clean":"2024-02-26T10:40:46.633206+0000","last_became_active":"2024-02-26T10:40:45.628287+0000","last_became_peered":"2024-02-26T10:40:45.628287+0000","last_unstale":"2024-02-26T10:40:46.633206+0000","last_undegraded":"2024-02-26T10:40:46.633206+0000","last_fullsized":"2024-02-26T10:40:46.633206+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:35.142103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637647+0000","last_change":"2024-02-26T10:40:45.644900+0000","last_active":"2024-02-26T10:40:46.637647+0000","last_peered":"2024-02-26T10:40:46.637647+0000","last_clean":"2024-02-26T10:40:46.637647+0000","last_became_active":"2024-02-26T10:40:45.644626+0000","last_became_peered":"2024-02-26T10:40:45.644626+0000","last_unstale":"2024-02-26T10:40:46.637647+0000","last_undegraded":"2024-02-26T10:40:46.637647+0000","last_fullsized":"2024-02-26T10:40:46.637647+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811962+0000","last_change":"2024-02-26T10:40:45.641831+0000","last_active":"2024-02-26T10:40:46.811962+0000","last_peered":"2024-02-26T10:40:46.811962+0000","last_clean":"2024-02-26T10:40:46.811962+0000","last_became_active":"2024-02-26T10:40:45.641331+0000","last_became_peered":"2024-02-26T10:40:45.641331+0000","last_unstale":"2024-02-26T10:40:46.811962+0000","last_undegraded":"2024-02-26T10:40:46.811962+0000","last_fullsized":"2024-02-26T10:40:46.811962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:18:27.192800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810376+0000","last_change":"2024-02-26T10:40:45.645455+0000","last_active":"2024-02-26T10:40:46.810376+0000","last_peered":"2024-02-26T10:40:46.810376+0000","last_clean":"2024-02-26T10:40:46.810376+0000","last_became_active":"2024-02-26T10:40:45.645142+0000","last_became_peered":"2024-02-26T10:40:45.645142+0000","last_unstale":"2024-02-26T10:40:46.810376+0000","last_undegraded":"2024-02-26T10:40:46.810376+0000","last_fullsized":"2024-02-26T10:40:46.810376+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:03.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.812831+0000","last_change":"2024-02-26T10:40:45.648027+0000","last_active":"2024-02-26T10:40:46.812831+0000","last_peered":"2024-02-26T10:40:46.812831+0000","last_clean":"2024-02-26T10:40:46.812831+0000","last_became_active":"2024-02-26T10:40:45.647523+0000","last_became_peered":"2024-02-26T10:40:45.647523+0000","last_unstale":"2024-02-26T10:40:46.812831+0000","last_undegraded":"2024-02-26T10:40:46.812831+0000","last_fullsized":"2024-02-26T10:40:46.812831+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:33.097875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.634702+0000","last_change":"2024-02-26T10:40:45.639856+0000","last_active":"2024-02-26T10:40:46.634702+0000","last_peered":"2024-02-26T10:40:46.634702+0000","last_clean":"2024-02-26T10:40:46.634702+0000","last_became_active":"2024-02-26T10:40:45.639346+0000","last_became_peered":"2024-02-26T10:40:45.639346+0000","last_unstale":"2024-02-26T10:40:46.634702+0000","last_undegraded":"2024-02-26T10:40:46.634702+0000","last_fullsized":"2024-02-26T10:40:46.634702+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:59.659271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732735+0000","last_change":"2024-02-26T10:40:45.639209+0000","last_active":"2024-02-26T10:40:46.732735+0000","last_peered":"2024-02-26T10:40:46.732735+0000","last_clean":"2024-02-26T10:40:46.732735+0000","last_became_active":"2024-02-26T10:40:45.638129+0000","last_became_peered":"2024-02-26T10:40:45.638129+0000","last_unstale":"2024-02-26T10:40:46.732735+0000","last_undegraded":"2024-02-26T10:40:46.732735+0000","last_fullsized":"2024-02-26T10:40:46.732735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:08.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810804+0000","last_change":"2024-02-26T10:40:45.649827+0000","last_active":"2024-02-26T10:40:46.810804+0000","last_peered":"2024-02-26T10:40:46.810804+0000","last_clean":"2024-02-26T10:40:46.810804+0000","last_became_active":"2024-02-26T10:40:45.649273+0000","last_became_peered":"2024-02-26T10:40:45.649273+0000","last_unstale":"2024-02-26T10:40:46.810804+0000","last_undegraded":"2024-02-26T10:40:46.810804+0000","last_fullsized":"2024-02-26T10:40:46.810804+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:33.671100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809214+0000","last_change":"2024-02-26T10:40:45.626981+0000","last_active":"2024-02-26T10:40:46.809214+0000","last_peered":"2024-02-26T10:40:46.809214+0000","last_clean":"2024-02-26T10:40:46.809214+0000","last_became_active":"2024-02-26T10:40:45.626693+0000","last_became_peered":"2024-02-26T10:40:45.626693+0000","last_unstale":"2024-02-26T10:40:46.809214+0000","last_undegraded":"2024-02-26T10:40:46.809214+0000","last_fullsized":"2024-02-26T10:40:46.809214+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:37.225724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732843+0000","last_change":"2024-02-26T10:40:45.634499+0000","last_active":"2024-02-26T10:40:46.732843+0000","last_peered":"2024-02-26T10:40:46.732843+0000","last_clean":"2024-02-26T10:40:46.732843+0000","last_became_active":"2024-02-26T10:40:45.633868+0000","last_became_peered":"2024-02-26T10:40:45.633868+0000","last_unstale":"2024-02-26T10:40:46.732843+0000","last_undegraded":"2024-02-26T10:40:46.732843+0000","last_fullsized":"2024-02-26T10:40:46.732843+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:18:54.827479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809670+0000","last_change":"2024-02-26T10:40:45.650672+0000","last_active":"2024-02-26T10:40:46.809670+0000","last_peered":"2024-02-26T10:40:46.809670+0000","last_clean":"2024-02-26T10:40:46.809670+0000","last_became_active":"2024-02-26T10:40:45.649874+0000","last_became_peered":"2024-02-26T10:40:45.649874+0000","last_unstale":"2024-02-26T10:40:46.809670+0000","last_undegraded":"2024-02-26T10:40:46.809670+0000","last_fullsized":"2024-02-26T10:40:46.809670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732333+0000","last_change":"2024-02-26T10:40:45.636991+0000","last_active":"2024-02-26T10:40:46.732333+0000","last_peered":"2024-02-26T10:40:46.732333+0000","last_clean":"2024-02-26T10:40:46.732333+0000","last_became_active":"2024-02-26T10:40:45.634121+0000","last_became_peered":"2024-02-26T10:40:45.634121+0000","last_unstale":"2024-02-26T10:40:46.732333+0000","last_undegraded":"2024-02-26T10:40:46.732333+0000","last_fullsized":"2024-02-26T10:40:46.732333+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:23:35.604849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.808980+0000","last_change":"2024-02-26T10:40:45.642494+0000","last_active":"2024-02-26T10:40:46.808980+0000","last_peered":"2024-02-26T10:40:46.808980+0000","last_clean":"2024-02-26T10:40:46.808980+0000","last_became_active":"2024-02-26T10:40:45.641655+0000","last_became_peered":"2024-02-26T10:40:45.641655+0000","last_unstale":"2024-02-26T10:40:46.808980+0000","last_undegraded":"2024-02-26T10:40:46.808980+0000","last_fullsized":"2024-02-26T10:40:46.808980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:15.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811283+0000","last_change":"2024-02-26T10:40:45.646927+0000","last_active":"2024-02-26T10:40:46.811283+0000","last_peered":"2024-02-26T10:40:46.811283+0000","last_clean":"2024-02-26T10:40:46.811283+0000","last_became_active":"2024-02-26T10:40:45.646598+0000","last_became_peered":"2024-02-26T10:40:45.646598+0000","last_unstale":"2024-02-26T10:40:46.811283+0000","last_undegraded":"2024-02-26T10:40:46.811283+0000","last_fullsized":"2024-02-26T10:40:46.811283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:02.887116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809461+0000","last_change":"2024-02-26T10:40:45.637507+0000","last_active":"2024-02-26T10:40:46.809461+0000","last_peered":"2024-02-26T10:40:46.809461+0000","last_clean":"2024-02-26T10:40:46.809461+0000","last_became_active":"2024-02-26T10:40:45.631723+0000","last_became_peered":"2024-02-26T10:40:45.631723+0000","last_unstale":"2024-02-26T10:40:46.809461+0000","last_undegraded":"2024-02-26T10:40:46.809461+0000","last_fullsized":"2024-02-26T10:40:46.809461+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:07.605390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637594+0000","last_change":"2024-02-26T10:40:45.633049+0000","last_active":"2024-02-26T10:40:46.637594+0000","last_peered":"2024-02-26T10:40:46.637594+0000","last_clean":"2024-02-26T10:40:46.637594+0000","last_became_active":"2024-02-26T10:40:45.632003+0000","last_became_peered":"2024-02-26T10:40:45.632003+0000","last_unstale":"2024-02-26T10:40:46.637594+0000","last_undegraded":"2024-02-26T10:40:46.637594+0000","last_fullsized":"2024-02-26T10:40:46.637594+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635284+0000","last_change":"2024-02-26T10:40:45.639683+0000","last_active":"2024-02-26T10:40:46.635284+0000","last_peered":"2024-02-26T10:40:46.635284+0000","last_clean":"2024-02-26T10:40:46.635284+0000","last_became_active":"2024-02-26T10:40:45.639388+0000","last_became_peered":"2024-02-26T10:40:45.639388+0000","last_unstale":"2024-02-26T10:40:46.635284+0000","last_undegraded":"2024-02-26T10:40:46.635284+0000","last_fullsized":"2024-02-26T10:40:46.635284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809049+0000","last_change":"2024-02-26T10:40:45.632119+0000","last_active":"2024-02-26T10:40:46.809049+0000","last_peered":"2024-02-26T10:40:46.809049+0000","last_clean":"2024-02-26T10:40:46.809049+0000","last_became_active":"2024-02-26T10:40:45.631803+0000","last_became_peered":"2024-02-26T10:40:45.631803+0000","last_unstale":"2024-02-26T10:40:46.809049+0000","last_undegraded":"2024-02-26T10:40:46.809049+0000","last_fullsized":"2024-02-26T10:40:46.809049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:42.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732922+0000","last_change":"2024-02-26T10:40:45.636100+0000","last_active":"2024-02-26T10:40:46.732922+0000","last_peered":"2024-02-26T10:40:46.732922+0000","last_clean":"2024-02-26T10:40:46.732922+0000","last_became_active":"2024-02-26T10:40:45.635761+0000","last_became_peered":"2024-02-26T10:40:45.635761+0000","last_unstale":"2024-02-26T10:40:46.732922+0000","last_undegraded":"2024-02-26T10:40:46.732922+0000","last_fullsized":"2024-02-26T10:40:46.732922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:42.198286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809263+0000","last_change":"2024-02-26T10:40:45.649382+0000","last_active":"2024-02-26T10:40:46.809263+0000","last_peered":"2024-02-26T10:40:46.809263+0000","last_clean":"2024-02-26T10:40:46.809263+0000","last_became_active":"2024-02-26T10:40:45.649076+0000","last_became_peered":"2024-02-26T10:40:45.649076+0000","last_unstale":"2024-02-26T10:40:46.809263+0000","last_undegraded":"2024-02-26T10:40:46.809263+0000","last_fullsized":"2024-02-26T10:40:46.809263+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.808787+0000","last_change":"2024-02-26T10:40:45.625890+0000","last_active":"2024-02-26T10:40:46.808787+0000","last_peered":"2024-02-26T10:40:46.808787+0000","last_clean":"2024-02-26T10:40:46.808787+0000","last_became_active":"2024-02-26T10:40:45.625334+0000","last_became_peered":"2024-02-26T10:40:45.625334+0000","last_unstale":"2024-02-26T10:40:46.808787+0000","last_undegraded":"2024-02-26T10:40:46.808787+0000","last_fullsized":"2024-02-26T10:40:46.808787+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:38:29.332505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.634899+0000","last_change":"2024-02-26T10:40:45.626985+0000","last_active":"2024-02-26T10:40:46.634899+0000","last_peered":"2024-02-26T10:40:46.634899+0000","last_clean":"2024-02-26T10:40:46.634899+0000","last_became_active":"2024-02-26T10:40:45.626657+0000","last_became_peered":"2024-02-26T10:40:45.626657+0000","last_unstale":"2024-02-26T10:40:46.634899+0000","last_undegraded":"2024-02-26T10:40:46.634899+0000","last_fullsized":"2024-02-26T10:40:46.634899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:15.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636130+0000","last_change":"2024-02-26T10:40:45.643239+0000","last_active":"2024-02-26T10:40:46.636130+0000","last_peered":"2024-02-26T10:40:46.636130+0000","last_clean":"2024-02-26T10:40:46.636130+0000","last_became_active":"2024-02-26T10:40:45.642924+0000","last_became_peered":"2024-02-26T10:40:45.642924+0000","last_unstale":"2024-02-26T10:40:46.636130+0000","last_undegraded":"2024-02-26T10:40:46.636130+0000","last_fullsized":"2024-02-26T10:40:46.636130+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636418+0000","last_change":"2024-02-26T10:40:44.610146+0000","last_active":"2024-02-26T10:40:46.636418+0000","last_peered":"2024-02-26T10:40:46.636418+0000","last_clean":"2024-02-26T10:40:46.636418+0000","last_became_active":"2024-02-26T10:40:44.609886+0000","last_became_peered":"2024-02-26T10:40:44.609886+0000","last_unstale":"2024-02-26T10:40:46.636418+0000","last_undegraded":"2024-02-26T10:40:46.636418+0000","last_fullsized":"2024-02-26T10:40:46.636418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:55.475057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.634034+0000","last_change":"2024-02-26T10:40:45.645160+0000","last_active":"2024-02-26T10:40:46.634034+0000","last_peered":"2024-02-26T10:40:46.634034+0000","last_clean":"2024-02-26T10:40:46.634034+0000","last_became_active":"2024-02-26T10:40:45.644902+0000","last_became_peered":"2024-02-26T10:40:45.644902+0000","last_unstale":"2024-02-26T10:40:46.634034+0000","last_undegraded":"2024-02-26T10:40:46.634034+0000","last_fullsized":"2024-02-26T10:40:46.634034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:53:58.631698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732847+0000","last_change":"2024-02-26T10:40:44.617860+0000","last_active":"2024-02-26T10:40:46.732847+0000","last_peered":"2024-02-26T10:40:46.732847+0000","last_clean":"2024-02-26T10:40:46.732847+0000","last_became_active":"2024-02-26T10:40:44.617592+0000","last_became_peered":"2024-02-26T10:40:44.617592+0000","last_unstale":"2024-02-26T10:40:46.732847+0000","last_undegraded":"2024-02-26T10:40:46.732847+0000","last_fullsized":"2024-02-26T10:40:46.732847+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:16.970692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635508+0000","last_change":"2024-02-26T10:40:45.621813+0000","last_active":"2024-02-26T10:40:46.635508+0000","last_peered":"2024-02-26T10:40:46.635508+0000","last_clean":"2024-02-26T10:40:46.635508+0000","last_became_active":"2024-02-26T10:40:45.621532+0000","last_became_peered":"2024-02-26T10:40:45.621532+0000","last_unstale":"2024-02-26T10:40:46.635508+0000","last_undegraded":"2024-02-26T10:40:46.635508+0000","last_fullsized":"2024-02-26T10:40:46.635508+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635945+0000","last_change":"2024-02-26T10:40:44.594679+0000","last_active":"2024-02-26T10:40:46.635945+0000","last_peered":"2024-02-26T10:40:46.635945+0000","last_clean":"2024-02-26T10:40:46.635945+0000","last_became_active":"2024-02-26T10:40:44.593817+0000","last_became_peered":"2024-02-26T10:40:44.593817+0000","last_unstale":"2024-02-26T10:40:46.635945+0000","last_undegraded":"2024-02-26T10:40:46.635945+0000","last_fullsized":"2024-02-26T10:40:46.635945+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:47.713723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635458+0000","last_change":"2024-02-26T10:40:45.630558+0000","last_active":"2024-02-26T10:40:46.635458+0000","last_peered":"2024-02-26T10:40:46.635458+0000","last_clean":"2024-02-26T10:40:46.635458+0000","last_became_active":"2024-02-26T10:40:45.630319+0000","last_became_peered":"2024-02-26T10:40:45.630319+0000","last_unstale":"2024-02-26T10:40:46.635458+0000","last_undegraded":"2024-02-26T10:40:46.635458+0000","last_fullsized":"2024-02-26T10:40:46.635458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637815+0000","last_change":"2024-02-26T10:40:44.616740+0000","last_active":"2024-02-26T10:40:46.637815+0000","last_peered":"2024-02-26T10:40:46.637815+0000","last_clean":"2024-02-26T10:40:46.637815+0000","last_became_active":"2024-02-26T10:40:44.616416+0000","last_became_peered":"2024-02-26T10:40:44.616416+0000","last_unstale":"2024-02-26T10:40:46.637815+0000","last_undegraded":"2024-02-26T10:40:46.637815+0000","last_fullsized":"2024-02-26T10:40:46.637815+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:58.112139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732364+0000","last_change":"2024-02-26T10:40:45.631974+0000","last_active":"2024-02-26T10:40:46.732364+0000","last_peered":"2024-02-26T10:40:46.732364+0000","last_clean":"2024-02-26T10:40:46.732364+0000","last_became_active":"2024-02-26T10:40:45.631691+0000","last_became_peered":"2024-02-26T10:40:45.631691+0000","last_unstale":"2024-02-26T10:40:46.732364+0000","last_undegraded":"2024-02-26T10:40:46.732364+0000","last_fullsized":"2024-02-26T10:40:46.732364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:57.553070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811150+0000","last_change":"2024-02-26T10:40:44.614734+0000","last_active":"2024-02-26T10:40:46.811150+0000","last_peered":"2024-02-26T10:40:46.811150+0000","last_clean":"2024-02-26T10:40:46.811150+0000","last_became_active":"2024-02-26T10:40:44.612533+0000","last_became_peered":"2024-02-26T10:40:44.612533+0000","last_unstale":"2024-02-26T10:40:46.811150+0000","last_undegraded":"2024-02-26T10:40:46.811150+0000","last_fullsized":"2024-02-26T10:40:46.811150+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:00:39.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732480+0000","last_change":"2024-02-26T10:40:45.645152+0000","last_active":"2024-02-26T10:40:46.732480+0000","last_peered":"2024-02-26T10:40:46.732480+0000","last_clean":"2024-02-26T10:40:46.732480+0000","last_became_active":"2024-02-26T10:40:45.644904+0000","last_became_peered":"2024-02-26T10:40:45.644904+0000","last_unstale":"2024-02-26T10:40:46.732480+0000","last_undegraded":"2024-02-26T10:40:46.732480+0000","last_fullsized":"2024-02-26T10:40:46.732480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:43.820459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635510+0000","last_change":"2024-02-26T10:40:44.608899+0000","last_active":"2024-02-26T10:40:46.635510+0000","last_peered":"2024-02-26T10:40:46.635510+0000","last_clean":"2024-02-26T10:40:46.635510+0000","last_became_active":"2024-02-26T10:40:44.608621+0000","last_became_peered":"2024-02-26T10:40:44.608621+0000","last_unstale":"2024-02-26T10:40:46.635510+0000","last_undegraded":"2024-02-26T10:40:46.635510+0000","last_fullsized":"2024-02-26T10:40:46.635510+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:51.049847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810733+0000","last_change":"2024-02-26T10:40:45.643761+0000","last_active":"2024-02-26T10:40:46.810733+0000","last_peered":"2024-02-26T10:40:46.810733+0000","last_clean":"2024-02-26T10:40:46.810733+0000","last_became_active":"2024-02-26T10:40:45.643356+0000","last_became_peered":"2024-02-26T10:40:45.643356+0000","last_unstale":"2024-02-26T10:40:46.810733+0000","last_undegraded":"2024-02-26T10:40:46.810733+0000","last_fullsized":"2024-02-26T10:40:46.810733+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:30.965210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636824+0000","last_change":"2024-02-26T10:40:44.621252+0000","last_active":"2024-02-26T10:40:46.636824+0000","last_peered":"2024-02-26T10:40:46.636824+0000","last_clean":"2024-02-26T10:40:46.636824+0000","last_became_active":"2024-02-26T10:40:44.620990+0000","last_became_peered":"2024-02-26T10:40:44.620990+0000","last_unstale":"2024-02-26T10:40:46.636824+0000","last_undegraded":"2024-02-26T10:40:46.636824+0000","last_fullsized":"2024-02-26T10:40:46.636824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:32.271293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732521+0000","last_change":"2024-02-26T10:40:45.647365+0000","last_active":"2024-02-26T10:40:46.732521+0000","last_peered":"2024-02-26T10:40:46.732521+0000","last_clean":"2024-02-26T10:40:46.732521+0000","last_became_active":"2024-02-26T10:40:45.638659+0000","last_became_peered":"2024-02-26T10:40:45.638659+0000","last_unstale":"2024-02-26T10:40:46.732521+0000","last_undegraded":"2024-02-26T10:40:46.732521+0000","last_fullsized":"2024-02-26T10:40:46.732521+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:35:53.706078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809570+0000","last_change":"2024-02-26T10:40:44.615854+0000","last_active":"2024-02-26T10:40:46.809570+0000","last_peered":"2024-02-26T10:40:46.809570+0000","last_clean":"2024-02-26T10:40:46.809570+0000","last_became_active":"2024-02-26T10:40:44.615511+0000","last_became_peered":"2024-02-26T10:40:44.615511+0000","last_unstale":"2024-02-26T10:40:46.809570+0000","last_undegraded":"2024-02-26T10:40:46.809570+0000","last_fullsized":"2024-02-26T10:40:46.809570+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:04:43.874952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.812382+0000","last_change":"2024-02-26T10:40:45.645033+0000","last_active":"2024-02-26T10:40:46.812382+0000","last_peered":"2024-02-26T10:40:46.812382+0000","last_clean":"2024-02-26T10:40:46.812382+0000","last_became_active":"2024-02-26T10:40:45.644738+0000","last_became_peered":"2024-02-26T10:40:45.644738+0000","last_unstale":"2024-02-26T10:40:46.812382+0000","last_undegraded":"2024-02-26T10:40:46.812382+0000","last_fullsized":"2024-02-26T10:40:46.812382+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:53:58.631698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810165+0000","last_change":"2024-02-26T10:40:44.619564+0000","last_active":"2024-02-26T10:40:46.810165+0000","last_peered":"2024-02-26T10:40:46.810165+0000","last_clean":"2024-02-26T10:40:46.810165+0000","last_became_active":"2024-02-26T10:40:44.619054+0000","last_became_peered":"2024-02-26T10:40:44.619054+0000","last_unstale":"2024-02-26T10:40:46.810165+0000","last_undegraded":"2024-02-26T10:40:46.810165+0000","last_fullsized":"2024-02-26T10:40:46.810165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:55.475057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810130+0000","last_change":"2024-02-26T10:40:45.651473+0000","last_active":"2024-02-26T10:40:46.810130+0000","last_peered":"2024-02-26T10:40:46.810130+0000","last_clean":"2024-02-26T10:40:46.810130+0000","last_became_active":"2024-02-26T10:40:45.651136+0000","last_became_peered":"2024-02-26T10:40:45.651136+0000","last_unstale":"2024-02-26T10:40:46.810130+0000","last_undegraded":"2024-02-26T10:40:46.810130+0000","last_fullsized":"2024-02-26T10:40:46.810130+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732963+0000","last_change":"2024-02-26T10:40:44.613640+0000","last_active":"2024-02-26T10:40:46.732963+0000","last_peered":"2024-02-26T10:40:46.732963+0000","last_clean":"2024-02-26T10:40:46.732963+0000","last_became_active":"2024-02-26T10:40:44.613419+0000","last_became_peered":"2024-02-26T10:40:44.613419+0000","last_unstale":"2024-02-26T10:40:46.732963+0000","last_undegraded":"2024-02-26T10:40:46.732963+0000","last_fullsized":"2024-02-26T10:40:46.732963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:57.206752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.632553+0000","last_change":"2024-02-26T10:40:45.640605+0000","last_active":"2024-02-26T10:40:46.632553+0000","last_peered":"2024-02-26T10:40:46.632553+0000","last_clean":"2024-02-26T10:40:46.632553+0000","last_became_active":"2024-02-26T10:40:45.640078+0000","last_became_peered":"2024-02-26T10:40:45.640078+0000","last_unstale":"2024-02-26T10:40:46.632553+0000","last_undegraded":"2024-02-26T10:40:46.632553+0000","last_fullsized":"2024-02-26T10:40:46.632553+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:28.520753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810968+0000","last_change":"2024-02-26T10:40:44.618998+0000","last_active":"2024-02-26T10:40:46.810968+0000","last_peered":"2024-02-26T10:40:46.810968+0000","last_clean":"2024-02-26T10:40:46.810968+0000","last_became_active":"2024-02-26T10:40:44.618710+0000","last_became_peered":"2024-02-26T10:40:44.618710+0000","last_unstale":"2024-02-26T10:40:46.810968+0000","last_undegraded":"2024-02-26T10:40:46.810968+0000","last_fullsized":"2024-02-26T10:40:46.810968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:55.190540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810050+0000","last_change":"2024-02-26T10:40:45.652659+0000","last_active":"2024-02-26T10:40:46.810050+0000","last_peered":"2024-02-26T10:40:46.810050+0000","last_clean":"2024-02-26T10:40:46.810050+0000","last_became_active":"2024-02-26T10:40:45.652312+0000","last_became_peered":"2024-02-26T10:40:45.652312+0000","last_unstale":"2024-02-26T10:40:46.810050+0000","last_undegraded":"2024-02-26T10:40:46.810050+0000","last_fullsized":"2024-02-26T10:40:46.810050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":20,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.812008+0000","last_change":"2024-02-26T10:40:44.615861+0000","last_active":"2024-02-26T10:40:46.812008+0000","last_peered":"2024-02-26T10:40:46.812008+0000","last_clean":"2024-02-26T10:40:46.812008+0000","last_became_active":"2024-02-26T10:40:44.615538+0000","last_became_peered":"2024-02-26T10:40:44.615538+0000","last_unstale":"2024-02-26T10:40:46.812008+0000","last_undegraded":"2024-02-26T10:40:46.812008+0000","last_fullsized":"2024-02-26T10:40:46.812008+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:29.960665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637024+0000","last_change":"2024-02-26T10:40:45.628631+0000","last_active":"2024-02-26T10:40:46.637024+0000","last_peered":"2024-02-26T10:40:46.637024+0000","last_clean":"2024-02-26T10:40:46.637024+0000","last_became_active":"2024-02-26T10:40:45.628321+0000","last_became_peered":"2024-02-26T10:40:45.628321+0000","last_unstale":"2024-02-26T10:40:46.637024+0000","last_undegraded":"2024-02-26T10:40:46.637024+0000","last_fullsized":"2024-02-26T10:40:46.637024+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732587+0000","last_change":"2024-02-26T10:40:44.617278+0000","last_active":"2024-02-26T10:40:46.732587+0000","last_peered":"2024-02-26T10:40:46.732587+0000","last_clean":"2024-02-26T10:40:46.732587+0000","last_became_active":"2024-02-26T10:40:44.610922+0000","last_became_peered":"2024-02-26T10:40:44.610922+0000","last_unstale":"2024-02-26T10:40:46.732587+0000","last_undegraded":"2024-02-26T10:40:46.732587+0000","last_fullsized":"2024-02-26T10:40:46.732587+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:31.392364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809753+0000","last_change":"2024-02-26T10:40:45.644578+0000","last_active":"2024-02-26T10:40:46.809753+0000","last_peered":"2024-02-26T10:40:46.809753+0000","last_clean":"2024-02-26T10:40:46.809753+0000","last_became_active":"2024-02-26T10:40:45.644211+0000","last_became_peered":"2024-02-26T10:40:45.644211+0000","last_unstale":"2024-02-26T10:40:46.809753+0000","last_undegraded":"2024-02-26T10:40:46.809753+0000","last_fullsized":"2024-02-26T10:40:46.809753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811550+0000","last_change":"2024-02-26T10:40:44.618437+0000","last_active":"2024-02-26T10:40:46.811550+0000","last_peered":"2024-02-26T10:40:46.811550+0000","last_clean":"2024-02-26T10:40:46.811550+0000","last_became_active":"2024-02-26T10:40:44.617307+0000","last_became_peered":"2024-02-26T10:40:44.617307+0000","last_unstale":"2024-02-26T10:40:46.811550+0000","last_undegraded":"2024-02-26T10:40:46.811550+0000","last_fullsized":"2024-02-26T10:40:46.811550+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:15.126480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811582+0000","last_change":"2024-02-26T10:40:45.643600+0000","last_active":"2024-02-26T10:40:46.811582+0000","last_peered":"2024-02-26T10:40:46.811582+0000","last_clean":"2024-02-26T10:40:46.811582+0000","last_became_active":"2024-02-26T10:40:45.643190+0000","last_became_peered":"2024-02-26T10:40:45.643190+0000","last_unstale":"2024-02-26T10:40:46.811582+0000","last_undegraded":"2024-02-26T10:40:46.811582+0000","last_fullsized":"2024-02-26T10:40:46.811582+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:35.142103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636907+0000","last_change":"2024-02-26T10:40:44.615356+0000","last_active":"2024-02-26T10:40:46.636907+0000","last_peered":"2024-02-26T10:40:46.636907+0000","last_clean":"2024-02-26T10:40:46.636907+0000","last_became_active":"2024-02-26T10:40:44.615072+0000","last_became_peered":"2024-02-26T10:40:44.615072+0000","last_unstale":"2024-02-26T10:40:46.636907+0000","last_undegraded":"2024-02-26T10:40:46.636907+0000","last_fullsized":"2024-02-26T10:40:46.636907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:06.798865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809156+0000","last_change":"2024-02-26T10:40:45.653780+0000","last_active":"2024-02-26T10:40:46.809156+0000","last_peered":"2024-02-26T10:40:46.809156+0000","last_clean":"2024-02-26T10:40:46.809156+0000","last_became_active":"2024-02-26T10:40:45.653336+0000","last_became_peered":"2024-02-26T10:40:45.653336+0000","last_unstale":"2024-02-26T10:40:46.809156+0000","last_undegraded":"2024-02-26T10:40:46.809156+0000","last_fullsized":"2024-02-26T10:40:46.809156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:49:13.271700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":22,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809727+0000","last_change":"2024-02-26T10:40:44.621458+0000","last_active":"2024-02-26T10:40:46.809727+0000","last_peered":"2024-02-26T10:40:46.809727+0000","last_clean":"2024-02-26T10:40:46.809727+0000","last_became_active":"2024-02-26T10:40:44.620425+0000","last_became_peered":"2024-02-26T10:40:44.620425+0000","last_unstale":"2024-02-26T10:40:46.809727+0000","last_undegraded":"2024-02-26T10:40:46.809727+0000","last_fullsized":"2024-02-26T10:40:46.809727+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:47.713723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811180+0000","last_change":"2024-02-26T10:40:45.654175+0000","last_active":"2024-02-26T10:40:46.811180+0000","last_peered":"2024-02-26T10:40:46.811180+0000","last_clean":"2024-02-26T10:40:46.811180+0000","last_became_active":"2024-02-26T10:40:45.653092+0000","last_became_peered":"2024-02-26T10:40:45.653092+0000","last_unstale":"2024-02-26T10:40:46.811180+0000","last_undegraded":"2024-02-26T10:40:46.811180+0000","last_fullsized":"2024-02-26T10:40:46.811180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:48.803196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.733010+0000","last_change":"2024-02-26T10:40:44.614563+0000","last_active":"2024-02-26T10:40:46.733010+0000","last_peered":"2024-02-26T10:40:46.733010+0000","last_clean":"2024-02-26T10:40:46.733010+0000","last_became_active":"2024-02-26T10:40:44.614281+0000","last_became_peered":"2024-02-26T10:40:44.614281+0000","last_unstale":"2024-02-26T10:40:46.733010+0000","last_undegraded":"2024-02-26T10:40:46.733010+0000","last_fullsized":"2024-02-26T10:40:46.733010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:58.146019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.638160+0000","last_change":"2024-02-26T10:40:45.628322+0000","last_active":"2024-02-26T10:40:46.638160+0000","last_peered":"2024-02-26T10:40:46.638160+0000","last_clean":"2024-02-26T10:40:46.638160+0000","last_became_active":"2024-02-26T10:40:45.624341+0000","last_became_peered":"2024-02-26T10:40:45.624341+0000","last_unstale":"2024-02-26T10:40:46.638160+0000","last_undegraded":"2024-02-26T10:40:46.638160+0000","last_fullsized":"2024-02-26T10:40:46.638160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636036+0000","last_change":"2024-02-26T10:40:44.613428+0000","last_active":"2024-02-26T10:40:46.636036+0000","last_peered":"2024-02-26T10:40:46.636036+0000","last_clean":"2024-02-26T10:40:46.636036+0000","last_became_active":"2024-02-26T10:40:44.613145+0000","last_became_peered":"2024-02-26T10:40:44.613145+0000","last_unstale":"2024-02-26T10:40:46.636036+0000","last_undegraded":"2024-02-26T10:40:46.636036+0000","last_fullsized":"2024-02-26T10:40:46.636036+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:55.190540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732930+0000","last_change":"2024-02-26T10:40:45.637902+0000","last_active":"2024-02-26T10:40:46.732930+0000","last_peered":"2024-02-26T10:40:46.732930+0000","last_clean":"2024-02-26T10:40:46.732930+0000","last_became_active":"2024-02-26T10:40:45.636813+0000","last_became_peered":"2024-02-26T10:40:45.636813+0000","last_unstale":"2024-02-26T10:40:46.732930+0000","last_undegraded":"2024-02-26T10:40:46.732930+0000","last_fullsized":"2024-02-26T10:40:46.732930+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:06.754252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.638335+0000","last_change":"2024-02-26T10:40:44.616165+0000","last_active":"2024-02-26T10:40:46.638335+0000","last_peered":"2024-02-26T10:40:46.638335+0000","last_clean":"2024-02-26T10:40:46.638335+0000","last_became_active":"2024-02-26T10:40:44.614801+0000","last_became_peered":"2024-02-26T10:40:44.614801+0000","last_unstale":"2024-02-26T10:40:46.638335+0000","last_undegraded":"2024-02-26T10:40:46.638335+0000","last_fullsized":"2024-02-26T10:40:46.638335+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:17.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810507+0000","last_change":"2024-02-26T10:40:45.655962+0000","last_active":"2024-02-26T10:40:46.810507+0000","last_peered":"2024-02-26T10:40:46.810507+0000","last_clean":"2024-02-26T10:40:46.810507+0000","last_became_active":"2024-02-26T10:40:45.655701+0000","last_became_peered":"2024-02-26T10:40:45.655701+0000","last_unstale":"2024-02-26T10:40:46.810507+0000","last_undegraded":"2024-02-26T10:40:46.810507+0000","last_fullsized":"2024-02-26T10:40:46.810507+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635640+0000","last_change":"2024-02-26T10:40:44.609742+0000","last_active":"2024-02-26T10:40:46.635640+0000","last_peered":"2024-02-26T10:40:46.635640+0000","last_clean":"2024-02-26T10:40:46.635640+0000","last_became_active":"2024-02-26T10:40:44.609452+0000","last_became_peered":"2024-02-26T10:40:44.609452+0000","last_unstale":"2024-02-26T10:40:46.635640+0000","last_undegraded":"2024-02-26T10:40:46.635640+0000","last_fullsized":"2024-02-26T10:40:46.635640+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:55.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637375+0000","last_change":"2024-02-26T10:40:45.631443+0000","last_active":"2024-02-26T10:40:46.637375+0000","last_peered":"2024-02-26T10:40:46.637375+0000","last_clean":"2024-02-26T10:40:46.637375+0000","last_became_active":"2024-02-26T10:40:45.631119+0000","last_became_peered":"2024-02-26T10:40:45.631119+0000","last_unstale":"2024-02-26T10:40:46.637375+0000","last_undegraded":"2024-02-26T10:40:46.637375+0000","last_fullsized":"2024-02-26T10:40:46.637375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732700+0000","last_change":"2024-02-26T10:40:44.616723+0000","last_active":"2024-02-26T10:40:46.732700+0000","last_peered":"2024-02-26T10:40:46.732700+0000","last_clean":"2024-02-26T10:40:46.732700+0000","last_became_active":"2024-02-26T10:40:44.616480+0000","last_became_peered":"2024-02-26T10:40:44.616480+0000","last_unstale":"2024-02-26T10:40:46.732700+0000","last_undegraded":"2024-02-26T10:40:46.732700+0000","last_fullsized":"2024-02-26T10:40:46.732700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:42.095622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.635192+0000","last_change":"2024-02-26T10:40:45.639556+0000","last_active":"2024-02-26T10:40:46.635192+0000","last_peered":"2024-02-26T10:40:46.635192+0000","last_clean":"2024-02-26T10:40:46.635192+0000","last_became_active":"2024-02-26T10:40:45.639271+0000","last_became_peered":"2024-02-26T10:40:45.639271+0000","last_unstale":"2024-02-26T10:40:46.635192+0000","last_undegraded":"2024-02-26T10:40:46.635192+0000","last_fullsized":"2024-02-26T10:40:46.635192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.633017+0000","last_change":"2024-02-26T10:40:44.612287+0000","last_active":"2024-02-26T10:40:46.633017+0000","last_peered":"2024-02-26T10:40:46.633017+0000","last_clean":"2024-02-26T10:40:46.633017+0000","last_became_active":"2024-02-26T10:40:44.611972+0000","last_became_peered":"2024-02-26T10:40:44.611972+0000","last_unstale":"2024-02-26T10:40:46.633017+0000","last_undegraded":"2024-02-26T10:40:46.633017+0000","last_fullsized":"2024-02-26T10:40:46.633017+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:38.390162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.634914+0000","last_change":"2024-02-26T10:40:45.644041+0000","last_active":"2024-02-26T10:40:46.634914+0000","last_peered":"2024-02-26T10:40:46.634914+0000","last_clean":"2024-02-26T10:40:46.634914+0000","last_became_active":"2024-02-26T10:40:45.640298+0000","last_became_peered":"2024-02-26T10:40:45.640298+0000","last_unstale":"2024-02-26T10:40:46.634914+0000","last_undegraded":"2024-02-26T10:40:46.634914+0000","last_fullsized":"2024-02-26T10:40:46.634914+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:49:13.271700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636846+0000","last_change":"2024-02-26T10:40:44.611018+0000","last_active":"2024-02-26T10:40:46.636846+0000","last_peered":"2024-02-26T10:40:46.636846+0000","last_clean":"2024-02-26T10:40:46.636846+0000","last_became_active":"2024-02-26T10:40:44.610676+0000","last_became_peered":"2024-02-26T10:40:44.610676+0000","last_unstale":"2024-02-26T10:40:46.636846+0000","last_undegraded":"2024-02-26T10:40:46.636846+0000","last_fullsized":"2024-02-26T10:40:46.636846+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:46:52.342064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.638029+0000","last_change":"2024-02-26T10:40:45.641308+0000","last_active":"2024-02-26T10:40:46.638029+0000","last_peered":"2024-02-26T10:40:46.638029+0000","last_clean":"2024-02-26T10:40:46.638029+0000","last_became_active":"2024-02-26T10:40:45.641045+0000","last_became_peered":"2024-02-26T10:40:45.641045+0000","last_unstale":"2024-02-26T10:40:46.638029+0000","last_undegraded":"2024-02-26T10:40:46.638029+0000","last_fullsized":"2024-02-26T10:40:46.638029+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.634325+0000","last_change":"2024-02-26T10:40:44.609096+0000","last_active":"2024-02-26T10:40:46.634325+0000","last_peered":"2024-02-26T10:40:46.634325+0000","last_clean":"2024-02-26T10:40:46.634325+0000","last_became_active":"2024-02-26T10:40:44.608811+0000","last_became_peered":"2024-02-26T10:40:44.608811+0000","last_unstale":"2024-02-26T10:40:46.634325+0000","last_undegraded":"2024-02-26T10:40:46.634325+0000","last_fullsized":"2024-02-26T10:40:46.634325+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:00:39.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.638474+0000","last_change":"2024-02-26T10:40:45.642661+0000","last_active":"2024-02-26T10:40:46.638474+0000","last_peered":"2024-02-26T10:40:46.638474+0000","last_clean":"2024-02-26T10:40:46.638474+0000","last_became_active":"2024-02-26T10:40:45.640611+0000","last_became_peered":"2024-02-26T10:40:45.640611+0000","last_unstale":"2024-02-26T10:40:46.638474+0000","last_undegraded":"2024-02-26T10:40:46.638474+0000","last_fullsized":"2024-02-26T10:40:46.638474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:14.395806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:47.622443+0000","last_change":"2024-02-26T10:40:44.612941+0000","last_active":"2024-02-26T10:40:47.622443+0000","last_peered":"2024-02-26T10:40:47.622443+0000","last_clean":"2024-02-26T10:40:47.622443+0000","last_became_active":"2024-02-26T10:40:44.612610+0000","last_became_peered":"2024-02-26T10:40:44.612610+0000","last_unstale":"2024-02-26T10:40:47.622443+0000","last_undegraded":"2024-02-26T10:40:47.622443+0000","last_fullsized":"2024-02-26T10:40:47.622443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:06.688227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.632726+0000","last_change":"2024-02-26T10:40:45.640358+0000","last_active":"2024-02-26T10:40:46.632726+0000","last_peered":"2024-02-26T10:40:46.632726+0000","last_clean":"2024-02-26T10:40:46.632726+0000","last_became_active":"2024-02-26T10:40:45.639707+0000","last_became_peered":"2024-02-26T10:40:45.639707+0000","last_unstale":"2024-02-26T10:40:46.632726+0000","last_undegraded":"2024-02-26T10:40:46.632726+0000","last_fullsized":"2024-02-26T10:40:46.632726+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:03.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811010+0000","last_change":"2024-02-26T10:40:44.619084+0000","last_active":"2024-02-26T10:40:46.811010+0000","last_peered":"2024-02-26T10:40:46.811010+0000","last_clean":"2024-02-26T10:40:46.811010+0000","last_became_active":"2024-02-26T10:40:44.618715+0000","last_became_peered":"2024-02-26T10:40:44.618715+0000","last_unstale":"2024-02-26T10:40:46.811010+0000","last_undegraded":"2024-02-26T10:40:46.811010+0000","last_fullsized":"2024-02-26T10:40:46.811010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:58.496066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.636003+0000","last_change":"2024-02-26T10:40:45.630528+0000","last_active":"2024-02-26T10:40:46.636003+0000","last_peered":"2024-02-26T10:40:46.636003+0000","last_clean":"2024-02-26T10:40:46.636003+0000","last_became_active":"2024-02-26T10:40:45.630230+0000","last_became_peered":"2024-02-26T10:40:45.630230+0000","last_unstale":"2024-02-26T10:40:46.636003+0000","last_undegraded":"2024-02-26T10:40:46.636003+0000","last_fullsized":"2024-02-26T10:40:46.636003+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.637937+0000","last_change":"2024-02-26T10:40:44.619432+0000","last_active":"2024-02-26T10:40:46.637937+0000","last_peered":"2024-02-26T10:40:46.637937+0000","last_clean":"2024-02-26T10:40:46.637937+0000","last_became_active":"2024-02-26T10:40:44.619141+0000","last_became_peered":"2024-02-26T10:40:44.619141+0000","last_unstale":"2024-02-26T10:40:46.637937+0000","last_undegraded":"2024-02-26T10:40:46.637937+0000","last_fullsized":"2024-02-26T10:40:46.637937+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:05.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732511+0000","last_change":"2024-02-26T10:40:45.634981+0000","last_active":"2024-02-26T10:40:46.732511+0000","last_peered":"2024-02-26T10:40:46.732511+0000","last_clean":"2024-02-26T10:40:46.732511+0000","last_became_active":"2024-02-26T10:40:45.634638+0000","last_became_peered":"2024-02-26T10:40:45.634638+0000","last_unstale":"2024-02-26T10:40:46.732511+0000","last_undegraded":"2024-02-26T10:40:46.732511+0000","last_fullsized":"2024-02-26T10:40:46.732511+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:01:22.095627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.632254+0000","last_change":"2024-02-26T10:40:44.599450+0000","last_active":"2024-02-26T10:40:46.632254+0000","last_peered":"2024-02-26T10:40:46.632254+0000","last_clean":"2024-02-26T10:40:46.632254+0000","last_became_active":"2024-02-26T10:40:44.598802+0000","last_became_peered":"2024-02-26T10:40:44.598802+0000","last_unstale":"2024-02-26T10:40:46.632254+0000","last_undegraded":"2024-02-26T10:40:46.632254+0000","last_fullsized":"2024-02-26T10:40:46.632254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:58.496066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810183+0000","last_change":"2024-02-26T10:40:45.650467+0000","last_active":"2024-02-26T10:40:46.810183+0000","last_peered":"2024-02-26T10:40:46.810183+0000","last_clean":"2024-02-26T10:40:46.810183+0000","last_became_active":"2024-02-26T10:40:45.649189+0000","last_became_peered":"2024-02-26T10:40:45.649189+0000","last_unstale":"2024-02-26T10:40:46.810183+0000","last_undegraded":"2024-02-26T10:40:46.810183+0000","last_fullsized":"2024-02-26T10:40:46.810183+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.810675+0000","last_change":"2024-02-26T10:40:44.613063+0000","last_active":"2024-02-26T10:40:46.810675+0000","last_peered":"2024-02-26T10:40:46.810675+0000","last_clean":"2024-02-26T10:40:46.810675+0000","last_became_active":"2024-02-26T10:40:44.612792+0000","last_became_peered":"2024-02-26T10:40:44.612792+0000","last_unstale":"2024-02-26T10:40:46.810675+0000","last_undegraded":"2024-02-26T10:40:46.810675+0000","last_fullsized":"2024-02-26T10:40:46.810675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:38.390162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.631899+0000","last_change":"2024-02-26T10:40:45.636109+0000","last_active":"2024-02-26T10:40:46.631899+0000","last_peered":"2024-02-26T10:40:46.631899+0000","last_clean":"2024-02-26T10:40:46.631899+0000","last_became_active":"2024-02-26T10:40:45.635783+0000","last_became_peered":"2024-02-26T10:40:45.635783+0000","last_unstale":"2024-02-26T10:40:46.631899+0000","last_undegraded":"2024-02-26T10:40:46.631899+0000","last_fullsized":"2024-02-26T10:40:46.631899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:45:18.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.631934+0000","last_change":"2024-02-26T10:40:44.595757+0000","last_active":"2024-02-26T10:40:46.631934+0000","last_peered":"2024-02-26T10:40:46.631934+0000","last_clean":"2024-02-26T10:40:46.631934+0000","last_became_active":"2024-02-26T10:40:44.595491+0000","last_became_peered":"2024-02-26T10:40:44.595491+0000","last_unstale":"2024-02-26T10:40:46.631934+0000","last_undegraded":"2024-02-26T10:40:46.631934+0000","last_fullsized":"2024-02-26T10:40:46.631934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:15.126480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732743+0000","last_change":"2024-02-26T10:40:45.647115+0000","last_active":"2024-02-26T10:40:46.732743+0000","last_peered":"2024-02-26T10:40:46.732743+0000","last_clean":"2024-02-26T10:40:46.732743+0000","last_became_active":"2024-02-26T10:40:45.637035+0000","last_became_peered":"2024-02-26T10:40:45.637035+0000","last_unstale":"2024-02-26T10:40:46.732743+0000","last_undegraded":"2024-02-26T10:40:46.732743+0000","last_fullsized":"2024-02-26T10:40:46.732743+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:26.126929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732682+0000","last_change":"2024-02-26T10:40:44.620365+0000","last_active":"2024-02-26T10:40:46.732682+0000","last_peered":"2024-02-26T10:40:46.732682+0000","last_clean":"2024-02-26T10:40:46.732682+0000","last_became_active":"2024-02-26T10:40:44.619424+0000","last_became_peered":"2024-02-26T10:40:44.619424+0000","last_unstale":"2024-02-26T10:40:46.732682+0000","last_undegraded":"2024-02-26T10:40:46.732682+0000","last_fullsized":"2024-02-26T10:40:46.732682+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:15.076898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.732391+0000","last_change":"2024-02-26T10:40:45.633488+0000","last_active":"2024-02-26T10:40:46.732391+0000","last_peered":"2024-02-26T10:40:46.732391+0000","last_clean":"2024-02-26T10:40:46.732391+0000","last_became_active":"2024-02-26T10:40:45.633157+0000","last_became_peered":"2024-02-26T10:40:45.633157+0000","last_unstale":"2024-02-26T10:40:46.732391+0000","last_undegraded":"2024-02-26T10:40:46.732391+0000","last_fullsized":"2024-02-26T10:40:46.732391+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:33.441514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.811379+0000","last_change":"2024-02-26T10:40:44.614389+0000","last_active":"2024-02-26T10:40:46.811379+0000","last_peered":"2024-02-26T10:40:46.811379+0000","last_clean":"2024-02-26T10:40:46.811379+0000","last_became_active":"2024-02-26T10:40:44.613960+0000","last_became_peered":"2024-02-26T10:40:44.613960+0000","last_unstale":"2024-02-26T10:40:46.811379+0000","last_undegraded":"2024-02-26T10:40:46.811379+0000","last_fullsized":"2024-02-26T10:40:46.811379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:09:07.764711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-02-26T10:40:46.809863+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:40:46.809863+0000","last_peered":"2024-02-26T10:40:46.809863+0000","last_clean":"2024-02-26T10:40:46.809863+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:40:46.809863+0000","last_undegraded":"2024-02-26T10:40:46.809863+0000","last_fullsized":"2024-02-26T10:40:46.809863+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":872472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":281944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":663552,"data_stored":282596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":283556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":282000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":282000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":659456,"data_stored":282034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1265664,"data_stored":874506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:40:50.636 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:40:50.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-26T10:40:50.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.905+0000 7f8ff1e00640 1 Processor -- start 2024-02-26T10:40:50.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.906+0000 7f8ff1e00640 1 -- start start 2024-02-26T10:40:50.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.907+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 0x7f8fec0c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.907+0000 7f8ff1e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec1521c0 con 0x7f8fec155690 2024-02-26T10:40:50.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.907+0000 7f8ff1e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec1524a0 con 0x7f8fec073040 2024-02-26T10:40:50.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.907+0000 7f8ff1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec06cf00 con 0x7f8fec151c50 2024-02-26T10:40:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.908+0000 7f8feb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8fec155690 0x7f8fec155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46448/0 (socket says 172.21.15.67:46448) 2024-02-26T10:40:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.908+0000 7f8feb400640 1 -- 172.21.15.67:0/2332328439 learned_addr learned my addr 172.21.15.67:0/2332328439 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.908+0000 7f8fea000640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 0x7f8fec0c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.908+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181569133 0 0) 0x7f8fec1521c0 con 0x7f8fec155690 2024-02-26T10:40:50.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd0005730 con 0x7f8fec155690 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 129040326 0 0) 0x7f8fd0005730 con 0x7f8fec155690 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 msgr2=0x7f8fec0c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 0x7f8fec0c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 >> v1:172.21.15.82:6789/0 conn(0x7f8fec073040 legacy=0x7f8fec0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec152780 con 0x7f8fec155690 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8fe40019f0 con 0x7f8fec155690 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 >> v1:172.21.15.67:6789/0 conn(0x7f8fec155690 legacy=0x7f8fec155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0005bf0 0x7f8fd0005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 0x7f8fd0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd0006f50 0x7f8fd00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd000dc80 con 0x7f8fd0006f50 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd000de00 con 0x7f8fd0005bf0 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd000e000 con 0x7f8fec151c50 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.909+0000 7f8feb400640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd0006f50 0x7f8fd00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8fea000640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 0x7f8fd0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feaa00640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0005bf0 0x7f8fd0005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feb400640 1 -- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 msgr2=0x7f8fd0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feb400640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec151c50 0x7f8fd0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feb400640 1 -- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0005bf0 msgr2=0x7f8fd0005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feb400640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0005bf0 0x7f8fd0005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feb400640 1 -- 172.21.15.67:0/2332328439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0005730 con 0x7f8fd0006f50 2024-02-26T10:40:50.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.910+0000 7f8feaa00640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0005bf0 0x7f8fd0005fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.911+0000 7f8feb400640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd0006f50 0x7f8fd00073d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400a810 tx=0x7f8fd0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385f9e6ab9b5490e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.911+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400a430 con 0x7f8fd0006f50 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.911+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4034630 con 0x7f8fd0006f50 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.911+0000 7f8fe9600640 1 -- 172.21.15.67:0/2332328439 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4034a20 con 0x7f8fd0006f50 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2332328439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd0006f50 msgr2=0x7f8fd00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2332328439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fd0006f50 0x7f8fd00073d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400a810 tx=0x7f8fd0007a40 comp rx=0 tx=0).stop 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8feb400640 1 -- 172.21.15.67:0/2332328439 reap_dead start 2024-02-26T10:40:50.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2332328439 shutdown_connections 2024-02-26T10:40:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2332328439 >> 172.21.15.67:0/2332328439 conn(0x7f8fec0c7160 msgr2=0x7f8fec0a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2332328439 shutdown_connections 2024-02-26T10:40:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.912+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2332328439 wait complete. 2024-02-26T10:40:50.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.914+0000 7f8ff1e00640 1 Processor -- start 2024-02-26T10:40:50.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.914+0000 7f8ff1e00640 1 -- start start 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0006f50 0x7f8fec14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec1ea620 0x7f8fec0bf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8ff1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 0x7f8fec0a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8ff1e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c5c80 con 0x7f8fec0a7300 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8ff1e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c2a80 con 0x7f8fd0006f50 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8ff1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c5e00 con 0x7f8fec1ea620 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8feb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 0x7f8fec0a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8feb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 0x7f8fec0a7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40658/0 (socket says 172.21.15.67:40658) 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.915+0000 7f8feb400640 1 -- 172.21.15.67:0/2842038759 learned_addr learned my addr 172.21.15.67:0/2842038759 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8fea000640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec1ea620 0x7f8fec0bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feaa00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0006f50 0x7f8fec14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feb400640 1 -- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec1ea620 msgr2=0x7f8fec0bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feb400640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec1ea620 0x7f8fec0bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feb400640 1 -- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0006f50 msgr2=0x7f8fec14b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feb400640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0006f50 0x7f8fec14b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feb400640 1 -- 172.21.15.67:0/2842038759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0bb840 con 0x7f8fec0a7300 2024-02-26T10:40:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.916+0000 7f8feb400640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 0x7f8fec0a7790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400a930 tx=0x7f8fe4001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:50.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.917+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400a430 con 0x7f8fec0a7300 2024-02-26T10:40:50.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.917+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4044070 con 0x7f8fec0a7300 2024-02-26T10:40:50.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.917+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4034c10 con 0x7f8fec0a7300 2024-02-26T10:40:50.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.917+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec0bbb20 con 0x7f8fec0a7300 2024-02-26T10:40:50.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.917+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec14ad30 con 0x7f8fec0a7300 2024-02-26T10:40:50.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.918+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec074a30 con 0x7f8fec0a7300 2024-02-26T10:40:50.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.919+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fe4049460 con 0x7f8fec0a7300 2024-02-26T10:40:50.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.920+0000 7f8fe3400640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8fcc06b4d0 0x7f8fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:50.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.920+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe40ac610 con 0x7f8fec0a7300 2024-02-26T10:40:50.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.920+0000 7f8feaa00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8fcc06b4d0 0x7f8fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:50.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.922+0000 7f8feaa00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8fcc06b4d0 0x7f8fcc06d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:50.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:50.924+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fe407aed0 con 0x7f8fec0a7300 2024-02-26T10:40:51.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.195+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8fec0c2860 con 0x7f8fec0a7300 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.200+0000 7f8fe3400640 1 -- 172.21.15.67:0/2842038759 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8fe407ef70 con 0x7f8fec0a7300 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw== 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-02-26T10:40:51.200 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-02-26T10:40:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.203+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8fcc06b4d0 msgr2=0x7f8fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.203+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8fcc06b4d0 0x7f8fcc06d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-02-26T10:40:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.203+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 msgr2=0x7f8fec0a7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.203+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 0x7f8fec0a7790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400a930 tx=0x7f8fe4001d40 comp rx=0 tx=0).stop 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 shutdown_connections 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fec0a7300 0x7f8fec0a7790 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8fec1ea620 0x7f8fec0bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8fcc06b4d0 0x7f8fcc06d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 --2- 172.21.15.67:0/2842038759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0006f50 0x7f8fec14b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 >> 172.21.15.67:0/2842038759 conn(0x7f8fec0c7160 msgr2=0x7f8fec0c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 shutdown_connections 2024-02-26T10:40:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.204+0000 7f8ff1e00640 1 -- 172.21.15.67:0/2842038759 wait complete. 2024-02-26T10:40:51.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:40:51.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.445+0000 7fc999c00640 1 Processor -- start 2024-02-26T10:40:51.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.445+0000 7fc999c00640 1 -- start start 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc99406ac20 0x7fc9940a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc99406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99406b460 con 0x7fc994153610 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940a9a80 con 0x7fc99406ac20 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940a9d60 con 0x7fc99406b000 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc993e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc99406ac20 0x7fc9940a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc994153610 0x7fc9940abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46454/0 (socket says 172.21.15.67:46454) 2024-02-26T10:40:51.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.447+0000 7fc999200640 1 -- 172.21.15.67:0/209809599 learned_addr learned my addr 172.21.15.67:0/209809599 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:51.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.448+0000 7fc993400640 1 --2- 172.21.15.67:0/209809599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc99406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:51.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.448+0000 7fc993e00640 1 -- 172.21.15.67:0/209809599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 msgr2=0x7fc99406db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:51.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.448+0000 7fc993e00640 1 --2- 172.21.15.67:0/209809599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc99406db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.448+0000 7fc993e00640 1 -- 172.21.15.67:0/209809599 >> v1:172.21.15.67:6789/0 conn(0x7fc994153610 legacy=0x7fc9940abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.448+0000 7fc993e00640 1 -- 172.21.15.67:0/209809599 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940ab750 con 0x7fc99406ac20 2024-02-26T10:40:51.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.448+0000 7fc992a00640 1 -- 172.21.15.67:0/209809599 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318873387 0 0) 0x7fc99406b460 con 0x7fc994153610 2024-02-26T10:40:51.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.449+0000 7fc993e00640 1 --2- 172.21.15.67:0/209809599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc9940a94a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc9840097a0 tx=0x7fc98402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8c998ea739a5dc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:51.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.450+0000 7fc992a00640 1 -- 172.21.15.67:0/209809599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984035020 con 0x7fc99406ac20 2024-02-26T10:40:51.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.450+0000 7fc992a00640 1 -- 172.21.15.67:0/209809599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984031810 con 0x7fc99406ac20 2024-02-26T10:40:51.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.450+0000 7fc992a00640 1 -- 172.21.15.67:0/209809599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031be0 con 0x7fc99406ac20 2024-02-26T10:40:51.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.450+0000 7fc999c00640 1 -- 172.21.15.67:0/209809599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 msgr2=0x7fc9940a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.450+0000 7fc999c00640 1 --2- 172.21.15.67:0/209809599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc9940a94a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc9840097a0 tx=0x7fc98402b8b0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.451+0000 7fc999c00640 1 -- 172.21.15.67:0/209809599 shutdown_connections 2024-02-26T10:40:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.451+0000 7fc999c00640 1 --2- 172.21.15.67:0/209809599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc99406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.451+0000 7fc999c00640 1 --2- 172.21.15.67:0/209809599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc9940a94a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.451+0000 7fc999c00640 1 -- 172.21.15.67:0/209809599 >> 172.21.15.67:0/209809599 conn(0x7fc9940c63f0 msgr2=0x7fc9940c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.451+0000 7fc999c00640 1 -- 172.21.15.67:0/209809599 shutdown_connections 2024-02-26T10:40:51.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.451+0000 7fc999c00640 1 -- 172.21.15.67:0/209809599 wait complete. 2024-02-26T10:40:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.452+0000 7fc999c00640 1 Processor -- start 2024-02-26T10:40:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.452+0000 7fc999c00640 1 -- start start 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.453+0000 7fc999c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc994071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.453+0000 7fc999c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc994147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.453+0000 7fc999c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc994153610 0x7fc994148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.453+0000 7fc999c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994150410 con 0x7fc99406ac20 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.453+0000 7fc999c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994153aa0 con 0x7fc994153610 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.453+0000 7fc999c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc994150590 con 0x7fc99406b000 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc994071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc994071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40680/0 (socket says 172.21.15.67:40680) 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 -- 172.21.15.67:0/827188701 learned_addr learned my addr 172.21.15.67:0/827188701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993400640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc994147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc999200640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc994153610 0x7fc994148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 -- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 msgr2=0x7fc994147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc994147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 -- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc994153610 msgr2=0x7fc994148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc994153610 0x7fc994148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc993e00640 1 -- 172.21.15.67:0/827188701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9941eb0f0 con 0x7fc99406ac20 2024-02-26T10:40:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.454+0000 7fc999200640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc994153610 0x7fc994148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:51.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.455+0000 7fc993e00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc994071f10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc9840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.455+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98403cd00 con 0x7fc99406ac20 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.455+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984004030 con 0x7fc99406ac20 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.455+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9941eb3d0 con 0x7fc99406ac20 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.455+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984004420 con 0x7fc99406ac20 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.455+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9941eb990 con 0x7fc99406ac20 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.456+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99406bea0 con 0x7fc99406ac20 2024-02-26T10:40:51.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.457+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc984038020 con 0x7fc99406ac20 2024-02-26T10:40:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.458+0000 7fc98be00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc97406b4d0 0x7fc97406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.458+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9840ac6b0 con 0x7fc99406ac20 2024-02-26T10:40:51.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.458+0000 7fc993400640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc97406b4d0 0x7fc97406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:51.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.459+0000 7fc993400640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc97406b4d0 0x7fc97406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc994153070 tx=0x7fc980008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:51.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.462+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc99406bea0 con 0x7fc99406ac20 2024-02-26T10:40:51.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.691+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc994075020 con 0x7fc99406ac20 2024-02-26T10:40:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.692+0000 7fc98be00640 1 -- 172.21.15.67:0/827188701 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fc994075020 con 0x7fc99406ac20 2024-02-26T10:40:51.692 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:51.692 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-02-26T10:40:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 32 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc97406b4d0 msgr2=0x7fc97406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc97406b4d0 0x7fc97406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc994153070 tx=0x7fc980008040 comp rx=0 tx=0).stop 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 msgr2=0x7fc994071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc994071f10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc9840046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 shutdown_connections 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc97406b4d0 0x7fc97406d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc994153610 0x7fc994148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99406b000 0x7fc994147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 --2- 172.21.15.67:0/827188701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc99406ac20 0x7fc994071f10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:51.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.695+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 >> 172.21.15.67:0/827188701 conn(0x7fc9940c63f0 msgr2=0x7fc9940abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:51.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.696+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 shutdown_connections 2024-02-26T10:40:51.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:51.696+0000 7fc999c00640 1 -- 172.21.15.67:0/827188701 wait complete. 2024-02-26T10:40:52.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-26T10:40:45.064964+0000', 'modified': '2024-02-26T10:40:48.161474+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5727': {'gid': 5727, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.67:6834/1127454674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6832', 'nonce': 1127454674}, {'type': 'v1', 'addr': '172.21.15.67:6834', 'nonce': 1127454674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T10:40:52.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-26T10:40:52.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-26T10:40:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.345+0000 7f9e42000640 1 Processor -- start 2024-02-26T10:40:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.345+0000 7f9e42000640 1 -- start start 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.346+0000 7f9e42000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c152380 0x7f9e3c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.346+0000 7f9e42000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e3c0c7020 0x7f9e3c155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.346+0000 7f9e42000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e3c06d290 0x7f9e3c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.346+0000 7f9e42000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c152760 con 0x7f9e3c0c7020 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.346+0000 7f9e42000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c155cd0 con 0x7f9e3c06d290 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.346+0000 7f9e42000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c7550 con 0x7f9e3c152380 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e3c0c7020 0x7f9e3c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e3c0c7020 0x7f9e3c155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40688/0 (socket says 172.21.15.67:40688) 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 -- 172.21.15.67:0/2964251865 learned_addr learned my addr 172.21.15.67:0/2964251865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e40c00640 1 --2- 172.21.15.67:0/2964251865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c152380 0x7f9e3c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e41600640 1 --2- 172.21.15.67:0/2964251865 >> v2:172.21.15.82:3300/0 conn(0x7f9e3c06d290 0x7f9e3c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 -- 172.21.15.67:0/2964251865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c152380 msgr2=0x7f9e3c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 --2- 172.21.15.67:0/2964251865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c152380 0x7f9e3c0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 -- 172.21.15.67:0/2964251865 >> v2:172.21.15.82:3300/0 conn(0x7f9e3c06d290 msgr2=0x7f9e3c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 --2- 172.21.15.67:0/2964251865 >> v2:172.21.15.82:3300/0 conn(0x7f9e3c06d290 0x7f9e3c0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.347+0000 7f9e3be00640 1 -- 172.21.15.67:0/2964251865 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0c7830 con 0x7f9e3c0c7020 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.348+0000 7f9e3be00640 1 --2- 172.21.15.67:0/2964251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c0c7020 0x7f9e3c155790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c06dd70 tx=0x7f9e2802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de185a61294d78fd server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.349+0000 7f9e3b400640 1 -- 172.21.15.67:0/2964251865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2802c9e0 con 0x7f9e3c0c7020 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.349+0000 7f9e3b400640 1 -- 172.21.15.67:0/2964251865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2802cb80 con 0x7f9e3c0c7020 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.349+0000 7f9e3b400640 1 -- 172.21.15.67:0/2964251865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280343f0 con 0x7f9e3c0c7020 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 -- 172.21.15.67:0/2964251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c0c7020 msgr2=0x7f9e3c155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 --2- 172.21.15.67:0/2964251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c0c7020 0x7f9e3c155790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c06dd70 tx=0x7f9e2802be40 comp rx=0 tx=0).stop 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 -- 172.21.15.67:0/2964251865 shutdown_connections 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 --2- 172.21.15.67:0/2964251865 >> v2:172.21.15.82:3300/0 conn(0x7f9e3c06d290 0x7f9e3c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 --2- 172.21.15.67:0/2964251865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c0c7020 0x7f9e3c155790 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 --2- 172.21.15.67:0/2964251865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c152380 0x7f9e3c0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 -- 172.21.15.67:0/2964251865 >> 172.21.15.67:0/2964251865 conn(0x7f9e3c0ac1a0 msgr2=0x7f9e3c0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 -- 172.21.15.67:0/2964251865 shutdown_connections 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.350+0000 7f9e42000640 1 -- 172.21.15.67:0/2964251865 wait complete. 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.351+0000 7f9e42000640 1 Processor -- start 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.352+0000 7f9e42000640 1 -- start start 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e42000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e3c06d290 0x7f9e3c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e42000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c0c7020 0x7f9e3c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e42000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 0x7f9e3c0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e42000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c06d720 con 0x7f9e3c152380 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e42000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c06cd70 con 0x7f9e3c06d290 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e42000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c06d8a0 con 0x7f9e3c0c7020 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 0x7f9e3c0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 0x7f9e3c0c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40702/0 (socket says 172.21.15.67:40702) 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 -- 172.21.15.67:0/3686614253 learned_addr learned my addr 172.21.15.67:0/3686614253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e40c00640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e3c06d290 0x7f9e3c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e3be00640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c0c7020 0x7f9e3c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 -- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c0c7020 msgr2=0x7f9e3c0c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c0c7020 0x7f9e3c0c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 -- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e3c06d290 msgr2=0x7f9e3c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e3c06d290 0x7f9e3c0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.353+0000 7f9e41600640 1 -- 172.21.15.67:0/3686614253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0c18f0 con 0x7f9e3c152380 2024-02-26T10:40:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.354+0000 7f9e41600640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 0x7f9e3c0c43b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400d600 tx=0x7f9e3400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.354+0000 7f9e38c00640 1 -- 172.21.15.67:0/3686614253 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34018020 con 0x7f9e3c152380 2024-02-26T10:40:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.354+0000 7f9e38c00640 1 -- 172.21.15.67:0/3686614253 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e34014620 con 0x7f9e3c152380 2024-02-26T10:40:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.354+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c14ba10 con 0x7f9e3c152380 2024-02-26T10:40:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.354+0000 7f9e38c00640 1 -- 172.21.15.67:0/3686614253 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340149f0 con 0x7f9e3c152380 2024-02-26T10:40:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.354+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c0c09e0 con 0x7f9e3c152380 2024-02-26T10:40:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.357+0000 7f9e38c00640 1 -- 172.21.15.67:0/3686614253 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e3401b020 con 0x7f9e3c152380 2024-02-26T10:40:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.357+0000 7f9e38c00640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.358+0000 7f9e38c00640 1 -- 172.21.15.67:0/3686614253 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e3408a6e0 con 0x7f9e3c152380 2024-02-26T10:40:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.358+0000 7f9e465c8740 1 Processor -- start 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.358+0000 7f9e465c8740 1 -- start start 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.358+0000 7f9e40c00640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830a5410 0x55b0830df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e465c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0830085d0 con 0x55b0830e21e0 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b082fb1900 con 0x55b0830df070 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b083004670 con 0x55b0830a5410 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40712/0 (socket says 172.21.15.67:40712) 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e41600640 1 -- 172.21.15.67:0/3135674356 learned_addr learned my addr 172.21.15.67:0/3135674356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.359+0000 7f9e40c00640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006fd0 tx=0x7f9e2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e40c00640 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e3be00640 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830a5410 0x55b0830df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e41600640 1 -- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830a5410 msgr2=0x55b0830df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e41600640 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830a5410 0x55b0830df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e41600640 1 -- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 msgr2=0x55b0830e01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e41600640 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830e01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e41600640 1 -- 172.21.15.67:0/3135674356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b082d1d2d0 con 0x55b0830e21e0 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e41600640 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830a4190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9e34060fa0 tx=0x7f9e34058a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440e200de67d78ce server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e31200640 1 -- 172.21.15.67:0/3135674356 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340614d0 con 0x55b0830e21e0 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e31200640 1 -- 172.21.15.67:0/3135674356 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e340570c0 con 0x55b0830e21e0 2024-02-26T10:40:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.360+0000 7f9e31200640 1 -- 172.21.15.67:0/3135674356 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3405f070 con 0x55b0830e21e0 2024-02-26T10:40:52.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 -- 172.21.15.67:0/3135674356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 msgr2=0x55b0830a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830a4190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9e34060fa0 tx=0x7f9e34058a30 comp rx=0 tx=0).stop 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 -- 172.21.15.67:0/3135674356 shutdown_connections 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830a4190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830a5410 0x55b0830df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3135674356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830e01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 -- 172.21.15.67:0/3135674356 >> 172.21.15.67:0/3135674356 conn(0x55b0830a0a50 msgr2=0x55b0830c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 -- 172.21.15.67:0/3135674356 shutdown_connections 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.361+0000 7f9e465c8740 1 -- 172.21.15.67:0/3135674356 wait complete. 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.362+0000 7f9e465c8740 1 -- start start 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.362+0000 7f9e465c8740 1 Processor -- start 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830a3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b0830db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 0x55b0830f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0830e1d50 con 0x55b0830f3830 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b082fc4d60 con 0x55b0830df070 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0830e1810 con 0x55b0830e21e0 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e465c8740 10 client.0 _ll_register_callbacks cb 0x55b083000320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 0x55b0830f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e41600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 0x55b0830f3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40726/0 (socket says 172.21.15.67:40726) 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e41600640 1 -- 172.21.15.67:0/1567670555 learned_addr learned my addr 172.21.15.67:0/1567670555 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.363+0000 7f9e40c00640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e3be00640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b0830db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e41600640 1 -- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 msgr2=0x55b0830db6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e41600640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b0830db6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e41600640 1 -- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 msgr2=0x55b0830a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e41600640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e41600640 1 -- 172.21.15.67:0/1567670555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b082de76b0 con 0x55b0830f3830 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e3be00640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b0830db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:52.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e41600640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 0x55b0830f3cb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400d5d0 tx=0x7f9e3400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7399fb17b6019bc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e25600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:40:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e25600640 1 -- 172.21.15.67:0/1567670555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34055430 con 0x55b0830f3830 2024-02-26T10:40:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e25600640 1 -- 172.21.15.67:0/1567670555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e34014b90 con 0x55b0830f3830 2024-02-26T10:40:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.364+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e340570c0 con 0x55b0830f3830 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e25600640 1 -- 172.21.15.67:0/1567670555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e340555d0 con 0x55b0830f3830 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 10 client.5877 fetch_fsmap learned FSMap version 32 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55b0830dd330 con 0x55b0830f3830 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e25600640 1 -- 172.21.15.67:0/1567670555 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e340570c0 con 0x55b0830f3830 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e25600640 1 -- 172.21.15.67:0/1567670555 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e34013050 con 0x55b0830f3830 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 10 client.5877 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T10:40:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 10 client.5877 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 20 client.5877 populate_metadata read hostname 'smithi067' 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b0830d79c0 0x55b0830d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 4 client.5877 mds_command: new command op to 5727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 --> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b083015b10 con 0x55b0830d79c0 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.365+0000 7f9e40c00640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b0830d79c0 0x55b0830d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.366+0000 7f9e40c00640 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b0830d79c0 0x55b0830d7da0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85d975e8f86dd4d server_cookie=b48aca99b5ce24e6 in_seq=0 out_seq=0 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.366+0000 7f9e25600640 10 client.5877 ms_handle_connect on v2:172.21.15.67:6832/1127454674 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e25600640 1 -- 172.21.15.67:0/1567670555 <== mds.0 v2:172.21.15.67:6832/1127454674 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b083015b10 con 0x55b0830d79c0 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e25600640 10 client.5877 handle_command_reply: tid=0 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 client.5877 shutdown 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 msgr2=0x55b0830f3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 0x55b0830f3cb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400d5d0 tx=0x7f9e3400bfa0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 shutdown_connections 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830f3830 0x55b0830f3cb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b0830d79c0 0x55b0830d7da0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b0830db6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 --2- 172.21.15.67:0/1567670555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 >> 172.21.15.67:0/1567670555 conn(0x55b0830a5410 msgr2=0x55b08309d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e25600640 0 client.5877 ms_handle_reset on v2:172.21.15.67:6832/1127454674 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 shutdown_connections 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.367+0000 7f9e465c8740 1 -- 172.21.15.67:0/1567670555 wait complete. 2024-02-26T10:40:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.368+0000 7f9e465c8740 20 client.5877 trim_cache size 0 max 16384 2024-02-26T10:40:52.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.385+0000 7f9e465c8740 1 Processor -- start 2024-02-26T10:40:52.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 -- start start 2024-02-26T10:40:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 0x55b0830f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b08309f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830f3830 0x55b08309fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1f6520 con 0x55b0830df070 2024-02-26T10:40:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1ee2e0 con 0x55b0830f3830 2024-02-26T10:40:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.386+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1ee8e0 con 0x55b0830e21e0 2024-02-26T10:40:52.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 0x55b0830f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 0x55b0830f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40738/0 (socket says 172.21.15.67:40738) 2024-02-26T10:40:52.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 -- 172.21.15.67:0/634003567 learned_addr learned my addr 172.21.15.67:0/634003567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e3be00640 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b08309f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 -- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 msgr2=0x55b08309f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b08309f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 -- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830f3830 msgr2=0x55b08309fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830f3830 0x55b08309fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.387+0000 7f9e40c00640 1 -- 172.21.15.67:0/634003567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08309a190 con 0x55b0830df070 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.388+0000 7f9e40c00640 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 0x55b0830f0430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55b0830dd540 tx=0x7f9e2c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4258c4749df9c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.388+0000 7f9e27e00640 1 -- 172.21.15.67:0/634003567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c01b040 con 0x55b0830df070 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.388+0000 7f9e27e00640 1 -- 172.21.15.67:0/634003567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c02b070 con 0x55b0830df070 2024-02-26T10:40:52.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.388+0000 7f9e27e00640 1 -- 172.21.15.67:0/634003567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c030670 con 0x55b0830df070 2024-02-26T10:40:52.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.388+0000 7f9e465c8740 1 -- 172.21.15.67:0/634003567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 msgr2=0x55b0830f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.388+0000 7f9e465c8740 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 0x55b0830f0430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55b0830dd540 tx=0x7f9e2c014000 comp rx=0 tx=0).stop 2024-02-26T10:40:52.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 -- 172.21.15.67:0/634003567 shutdown_connections 2024-02-26T10:40:52.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830f3830 0x55b08309fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830e21e0 0x55b08309f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 --2- 172.21.15.67:0/634003567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830df070 0x55b0830f0430 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 -- 172.21.15.67:0/634003567 >> 172.21.15.67:0/634003567 conn(0x55b0830a5410 msgr2=0x55b0830db4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 -- 172.21.15.67:0/634003567 shutdown_connections 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 -- 172.21.15.67:0/634003567 wait complete. 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.389+0000 7f9e465c8740 1 -- start start 2024-02-26T10:40:52.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.390+0000 7f9e465c8740 1 Processor -- start 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.390+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.390+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.390+0000 7f9e465c8740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830f3830 0x55b083148510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.390+0000 7f9e465c8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0830e1d50 con 0x55b0830e21e0 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b082fc4d60 con 0x55b0830df070 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e465c8740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0830dd610 con 0x55b0830f3830 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e465c8740 10 client.0 _ll_register_callbacks cb 0x7f9e340550e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40754/0 (socket says 172.21.15.67:40754) 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 -- 172.21.15.67:0/3683766469 learned_addr learned my addr 172.21.15.67:0/3683766469 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e41600640 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830f3830 0x55b083148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 -- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830f3830 msgr2=0x55b083148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830f3830 0x55b083148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 -- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 msgr2=0x55b0830df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.391+0000 7f9e3be00640 1 -- 172.21.15.67:0/3683766469 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0830decb0 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e3be00640 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830c5350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9e2802be80 tx=0x7f9e28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50a219524204eac server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e25600640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e25600640 1 -- 172.21.15.67:0/3683766469 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28009510 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e25600640 1 -- 172.21.15.67:0/3683766469 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e2803eac0 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e2c02b070 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e25600640 1 -- 172.21.15.67:0/3683766469 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e2803ecc0 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e465c8740 10 client.5883 fetch_fsmap learned FSMap version 32 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.392+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55b08314bd80 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e25600640 1 -- 172.21.15.67:0/3683766469 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c02b070 con 0x55b0830e21e0 2024-02-26T10:40:52.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e25600640 1 -- 172.21.15.67:0/3683766469 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e280483f0 con 0x55b0830e21e0 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e465c8740 10 client.5883 fetch_fsmap finished waiting for FSMap version 32 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e465c8740 10 client.5883 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e465c8740 20 client.5883 populate_metadata read hostname 'smithi067' 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b08314e6d0 0x55b08314eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e465c8740 4 client.5883 mds_command: new command op to 5727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 --> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b08300fe60 con 0x55b08314e6d0 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e40c00640 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b08314e6d0 0x55b08314eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e40c00640 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b08314e6d0 0x55b08314eab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92a1c67b2b88b40 server_cookie=2925121e5cfd85d2 in_seq=0 out_seq=0 2024-02-26T10:40:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.393+0000 7f9e25600640 10 client.5883 ms_handle_connect on v2:172.21.15.67:6832/1127454674 2024-02-26T10:40:52.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e25600640 1 -- 172.21.15.67:0/3683766469 <== mds.0 v2:172.21.15.67:6832/1127454674 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b08300fe60 con 0x55b08314e6d0 2024-02-26T10:40:52.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e25600640 10 client.5883 handle_command_reply: tid=0 2024-02-26T10:40:52.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 client.5883 shutdown 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 msgr2=0x55b0830c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830c5350 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9e2802be80 tx=0x7f9e28004ab0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 shutdown_connections 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:6832/1127454674,v1:172.21.15.67:6834/1127454674] conn(0x55b08314e6d0 0x55b08314eab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b0830f3830 0x55b083148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b0830e21e0 0x55b0830c5350 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 --2- 172.21.15.67:0/3683766469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0830df070 0x55b0830df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 >> 172.21.15.67:0/3683766469 conn(0x55b0830a5410 msgr2=0x55b0830f0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e25600640 0 client.5883 ms_handle_reset on v2:172.21.15.67:6832/1127454674 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 shutdown_connections 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.394+0000 7f9e465c8740 1 -- 172.21.15.67:0/3683766469 wait complete. 2024-02-26T10:40:52.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e465c8740 20 client.5883 trim_cache size 0 max 16384 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "f4a3a702-2ce8-46d3-b32d-e4f6872b8b58", 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 5727, 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 6.7923466650000002, 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 32, 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 55, 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 55, 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 13.364272395 2024-02-26T10:40:52.398 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:40:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9e1c06b4d0 msgr2=0x7f9e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006fd0 tx=0x7f9e2c008040 comp rx=0 tx=0).stop 2024-02-26T10:40:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 msgr2=0x7f9e3c0c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 0x7f9e3c0c43b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400d600 tx=0x7f9e3400dac0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 shutdown_connections 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e3c152380 0x7f9e3c0c43b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e3c0c7020 0x7f9e3c0c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 --2- 172.21.15.67:0/3686614253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e3c06d290 0x7f9e3c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 >> 172.21.15.67:0/3686614253 conn(0x7f9e3c0ac1a0 msgr2=0x7f9e3c14cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 shutdown_connections 2024-02-26T10:40:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.395+0000 7f9e42000640 1 -- 172.21.15.67:0/3686614253 wait complete. 2024-02-26T10:40:52.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:40:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.621+0000 7f858b200640 1 Processor -- start 2024-02-26T10:40:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.621+0000 7f858b200640 1 -- start start 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f858b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 0x7f858c153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f858b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f858c06b000 0x7f858c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f858b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c155f10 con 0x7f858c06ac20 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f858b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c06b470 con 0x7f858c06b000 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f858b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0c28d0 con 0x7f858c0c2310 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f8589e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f858c06ac20 0x7f858c152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46464/0 (socket says 172.21.15.67:46464) 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.623+0000 7f8589e00640 1 -- 172.21.15.67:0/3253464735 learned_addr learned my addr 172.21.15.67:0/3253464735 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8589400640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 0x7f858c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f858a800640 1 --2- 172.21.15.67:0/3253464735 >> v2:172.21.15.82:3300/0 conn(0x7f858c06b000 0x7f858c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199612183 0 0) 0x7f858c155f10 con 0x7f858c06ac20 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8570003520 con 0x7f858c06ac20 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1576444812 0 0) 0x7f8570003520 con 0x7f858c06ac20 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 msgr2=0x7f858c153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 0x7f858c153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 >> v2:172.21.15.82:3300/0 conn(0x7f858c06b000 msgr2=0x7f858c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 --2- 172.21.15.67:0/3253464735 >> v2:172.21.15.82:3300/0 conn(0x7f858c06b000 0x7f858c1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c155150 con 0x7f858c06ac20 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f857c001670 con 0x7f858c06ac20 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 >> v1:172.21.15.67:6789/0 conn(0x7f858c06ac20 legacy=0x7f858c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 0x7f8570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8570004d40 0x7f85700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c06b470 con 0x7f85700039b0 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0c28d0 con 0x7f8570004d40 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.624+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857000ba40 con 0x7f858c0c2310 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589e00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589400640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 0x7f8570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589e00640 1 -- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 msgr2=0x7f8570004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589e00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c0c2310 0x7f8570004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589e00640 1 -- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8570004d40 msgr2=0x7f85700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589e00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8570004d40 0x7f85700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.625+0000 7f8589e00640 1 -- 172.21.15.67:0/3253464735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8570003520 con 0x7f85700039b0 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.626+0000 7f8589e00640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c290 tx=0x7f8570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41011d5b5c595694 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c007d50 con 0x7f85700039b0 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c030070 con 0x7f85700039b0 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f8588a00640 1 -- 172.21.15.67:0/3253464735 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c02cd80 con 0x7f85700039b0 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858b200640 1 -- 172.21.15.67:0/3253464735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 msgr2=0x7f8570003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858b200640 1 --2- 172.21.15.67:0/3253464735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c290 tx=0x7f8570005830 comp rx=0 tx=0).stop 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858a800640 1 -- 172.21.15.67:0/3253464735 reap_dead start 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858b200640 1 -- 172.21.15.67:0/3253464735 shutdown_connections 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858b200640 1 -- 172.21.15.67:0/3253464735 >> 172.21.15.67:0/3253464735 conn(0x7f858c0c63f0 msgr2=0x7f858c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858b200640 1 -- 172.21.15.67:0/3253464735 shutdown_connections 2024-02-26T10:40:52.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.627+0000 7f858b200640 1 -- 172.21.15.67:0/3253464735 wait complete. 2024-02-26T10:40:52.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.629+0000 7f858b200640 1 Processor -- start 2024-02-26T10:40:52.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.629+0000 7f858b200640 1 -- start start 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f858b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f858c070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f858b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c1516f0 0x7f858c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f858b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c1e6e20 0x7f858c141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f858b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c1559f0 con 0x7f85700039b0 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f858b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c155710 con 0x7f858c1516f0 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f858b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c148850 con 0x7f858c1e6e20 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f8589e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f858c070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f8589e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f858c070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40772/0 (socket says 172.21.15.67:40772) 2024-02-26T10:40:52.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.630+0000 7f8589e00640 1 -- 172.21.15.67:0/2208031635 learned_addr learned my addr 172.21.15.67:0/2208031635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:52.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8589e00640 1 -- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c1e6e20 msgr2=0x7f858c141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8589e00640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c1e6e20 0x7f858c141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8589e00640 1 -- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c1516f0 msgr2=0x7f858c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8589e00640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c1516f0 0x7f858c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8589e00640 1 -- 172.21.15.67:0/2208031635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c1508a0 con 0x7f85700039b0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8589e00640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f858c070c30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c290 tx=0x7f858c14bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.631+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c001c80 con 0x7f85700039b0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.632+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c004bf0 con 0x7f85700039b0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.632+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c150b80 con 0x7f85700039b0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.632+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c007070 con 0x7f85700039b0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.632+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c1e65d0 con 0x7f85700039b0 2024-02-26T10:40:52.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.633+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c074a30 con 0x7f85700039b0 2024-02-26T10:40:52.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.634+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f857c047060 con 0x7f85700039b0 2024-02-26T10:40:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.635+0000 7f8582000640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f856c06b410 0x7f856c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:52.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.635+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f857c0ab010 con 0x7f85700039b0 2024-02-26T10:40:52.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.635+0000 7f8589400640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f856c06b410 0x7f856c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:52.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.636+0000 7f8589400640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f856c06b410 0x7f856c06d8d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f858c14f160 tx=0x7f8578008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:52.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.640+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f857c079f20 con 0x7f85700039b0 2024-02-26T10:40:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.878+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f858c0c20f0 con 0x7f85700039b0 2024-02-26T10:40:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.879+0000 7f8582000640 1 -- 172.21.15.67:0/2208031635 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f857c079820 con 0x7f85700039b0 2024-02-26T10:40:52.878 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:40:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f856c06b410 msgr2=0x7f856c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f856c06b410 0x7f856c06d8d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f858c14f160 tx=0x7f8578008040 comp rx=0 tx=0).stop 2024-02-26T10:40:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 msgr2=0x7f858c070c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f858c070c30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c290 tx=0x7f858c14bff0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 shutdown_connections 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f858c1e6e20 0x7f858c141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f856c06b410 0x7f856c06d8d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c1516f0 0x7f858c14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 --2- 172.21.15.67:0/2208031635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85700039b0 0x7f858c070c30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.881+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 >> 172.21.15.67:0/2208031635 conn(0x7f858c0c63f0 msgr2=0x7f858c146740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.882+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 shutdown_connections 2024-02-26T10:40:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:52.882+0000 7f858b200640 1 -- 172.21.15.67:0/2208031635 wait complete. 2024-02-26T10:40:52.900 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:40:52.900 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:40:52.900 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:40:52.900 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T10:40:52.900 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:52.900 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:40:52.901 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:40:52.901 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:40:52.901 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:40:52.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:52.901 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:40:52.957 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::288e:93ff:fe9f:f95d/64 scope link 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:40:52.958 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:40:52.959 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::3cfc:46ff:fe71:84e3/64 scope link 2024-02-26T10:40:52.959 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:52.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:52.960 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:40:53.055 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:40:53.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.060 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:40:53.116 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:40:53.116 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:40:53.116 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:40:53.116 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:40:53.116 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:40:53.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.117 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:40:53.117 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:40:53.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:40:53.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:40:53.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:40:53.118 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:40:53.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.312 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:40:53.364 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:40:53.365 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:40:53.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:40:53.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:53.464 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:53.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:53.521 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:53.575 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-26T10:40:53.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:40:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:40:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:40:53.752 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T10:40:53.752 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:40:53.753 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:40:53.753 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:40:53.753 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:40:53.757 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:40:53.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:40:53.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:53.840 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:40:53.912 INFO:teuthology.orchestra.run.smithi067.stdout:client4988 2024-02-26T10:40:53.912 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:40:53.912 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:53.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:40:54.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.193+0000 7ff919c00640 1 Processor -- start 2024-02-26T10:40:54.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.193+0000 7ff919c00640 1 -- start start 2024-02-26T10:40:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406ac20 0x7ff9140a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff91406db00 0x7ff9140abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91406b460 con 0x7ff91406b000 2024-02-26T10:40:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91406dfb0 con 0x7ff91406db00 2024-02-26T10:40:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140a9a80 con 0x7ff91406ac20 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff91406db00 0x7ff9140abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff91406db00 0x7ff9140abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40774/0 (socket says 172.21.15.67:40774) 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.195+0000 7ff919200640 1 -- 172.21.15.67:0/1929421803 learned_addr learned my addr 172.21.15.67:0/1929421803 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.196+0000 7ff913e00640 1 --2- 172.21.15.67:0/1929421803 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406ac20 0x7ff9140a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.196+0000 7ff919200640 1 -- 172.21.15.67:0/1929421803 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406ac20 msgr2=0x7ff9140a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.196+0000 7ff919200640 1 --2- 172.21.15.67:0/1929421803 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406ac20 0x7ff9140a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.196+0000 7ff919200640 1 -- 172.21.15.67:0/1929421803 >> v1:172.21.15.82:6789/0 conn(0x7ff91406b000 legacy=0x7ff914153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.196+0000 7ff919200640 1 -- 172.21.15.67:0/1929421803 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140a9d60 con 0x7ff91406db00 2024-02-26T10:40:54.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.197+0000 7ff919200640 1 --2- 172.21.15.67:0/1929421803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff9140abf50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009870 tx=0x7ff90c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43edfe346c1eeef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:54.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.197+0000 7ff912a00640 1 -- 172.21.15.67:0/1929421803 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c034070 con 0x7ff91406db00 2024-02-26T10:40:54.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.197+0000 7ff912a00640 1 -- 172.21.15.67:0/1929421803 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c030840 con 0x7ff91406db00 2024-02-26T10:40:54.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.197+0000 7ff912a00640 1 -- 172.21.15.67:0/1929421803 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c030c10 con 0x7ff91406db00 2024-02-26T10:40:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 -- 172.21.15.67:0/1929421803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 msgr2=0x7ff9140abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 --2- 172.21.15.67:0/1929421803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff9140abf50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009870 tx=0x7ff90c02b980 comp rx=0 tx=0).stop 2024-02-26T10:40:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 -- 172.21.15.67:0/1929421803 shutdown_connections 2024-02-26T10:40:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 --2- 172.21.15.67:0/1929421803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff9140abf50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 --2- 172.21.15.67:0/1929421803 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406ac20 0x7ff9140a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 -- 172.21.15.67:0/1929421803 >> 172.21.15.67:0/1929421803 conn(0x7ff9140c63f0 msgr2=0x7ff9140c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 -- 172.21.15.67:0/1929421803 shutdown_connections 2024-02-26T10:40:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.198+0000 7ff919c00640 1 -- 172.21.15.67:0/1929421803 wait complete. 2024-02-26T10:40:54.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.201+0000 7ff919c00640 1 Processor -- start 2024-02-26T10:40:54.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.201+0000 7ff919c00640 1 -- start start 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff91406ac20 0x7ff914072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406b000 0x7ff914156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff914070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140ac490 con 0x7ff91406db00 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140aba30 con 0x7ff91406ac20 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff914150410 con 0x7ff91406b000 2024-02-26T10:40:54.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff914070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.202+0000 7ff919200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff914070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40776/0 (socket says 172.21.15.67:40776) 2024-02-26T10:40:54.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff919200640 1 -- 172.21.15.67:0/3805034145 learned_addr learned my addr 172.21.15.67:0/3805034145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:54.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff913e00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff91406ac20 0x7ff914072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff913400640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406b000 0x7ff914156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff919200640 1 -- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406b000 msgr2=0x7ff914156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff919200640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406b000 0x7ff914156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff919200640 1 -- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff91406ac20 msgr2=0x7ff914072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff919200640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff91406ac20 0x7ff914072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.203+0000 7ff919200640 1 -- 172.21.15.67:0/3805034145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9141eb190 con 0x7ff91406db00 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.204+0000 7ff919200640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff914070c50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff90c02b9c0 tx=0x7ff90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:54.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.204+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c034040 con 0x7ff91406db00 2024-02-26T10:40:54.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.204+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c039d90 con 0x7ff91406db00 2024-02-26T10:40:54.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.204+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c043630 con 0x7ff91406db00 2024-02-26T10:40:54.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.204+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9141eb430 con 0x7ff91406db00 2024-02-26T10:40:54.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.204+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff914148350 con 0x7ff91406db00 2024-02-26T10:40:54.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.205+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff914074a30 con 0x7ff91406db00 2024-02-26T10:40:54.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.206+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff90c04a060 con 0x7ff91406db00 2024-02-26T10:40:54.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.207+0000 7ff90be00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff8f006b4d0 0x7ff8f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:54.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.207+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff90c0ac0a0 con 0x7ff91406db00 2024-02-26T10:40:54.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.208+0000 7ff913e00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff8f006b4d0 0x7ff8f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:54.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.209+0000 7ff913e00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff8f006b4d0 0x7ff8f006d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff904003b80 tx=0x7ff9040098d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:54.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.212+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff90c03e070 con 0x7ff91406db00 2024-02-26T10:40:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.451+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff91406b9b0 con 0x7ff91406db00 2024-02-26T10:40:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.452+0000 7ff90be00640 1 -- 172.21.15.67:0/3805034145 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff90c07a960 con 0x7ff91406db00 2024-02-26T10:40:54.452 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff8f006b4d0 msgr2=0x7ff8f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff8f006b4d0 0x7ff8f006d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff904003b80 tx=0x7ff9040098d0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 msgr2=0x7ff914070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff914070c50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff90c02b9c0 tx=0x7ff90c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 shutdown_connections 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff91406db00 0x7ff914070c50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff91406b000 0x7ff914156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff8f006b4d0 0x7ff8f006d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 --2- 172.21.15.67:0/3805034145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff91406ac20 0x7ff914072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 >> 172.21.15.67:0/3805034145 conn(0x7ff9140c63f0 msgr2=0x7ff91414ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 shutdown_connections 2024-02-26T10:40:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:54.455+0000 7ff919c00640 1 -- 172.21.15.67:0/3805034145 wait complete. 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:40:54.472 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:40:54.473 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:40:54.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:54.473 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:54.533 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42224sec preferred_lft 42224sec 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:54.534 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:40:54.535 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:40:54.535 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:40:54.535 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:40:54.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:54.536 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:40:54.629 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:40:54.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:54.633 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:40:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:40:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:40:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:40:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:40:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:40:54.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:54.685 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:40:54.685 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:40:54.686 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:40:54.686 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:40:54.686 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:40:54.686 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:40:54.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:54.858 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:40:54.907 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:40:54.908 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:40:54.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:54.908 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:40:55.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:55.007 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:55.057 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:40:55.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:40:55.058 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:55.106 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-26T10:40:55.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:55.106 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:55.239 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:55.239 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:40:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:40:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:40:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:40:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:40:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:40:55.280 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:40:55.280 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:40:55.280 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:40:55.280 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:40:55.285 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:40:55.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:55.286 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:40:55.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:40:55.363 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:40:55.420 INFO:teuthology.orchestra.run.smithi082.stdout:client5898 2024-02-26T10:40:55.420 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:40:55.420 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:40:55.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-02-26T10:40:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.701+0000 7f4bd8200640 1 Processor -- start 2024-02-26T10:40:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.702+0000 7f4bd8200640 1 -- start start 2024-02-26T10:40:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.703+0000 7f4bd8200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4bd00c1090 0x7f4bd01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.703+0000 7f4bd8200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd00c15e0 con 0x7f4bd0152420 2024-02-26T10:40:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.703+0000 7f4bd8200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd006e660 con 0x7f4bd00bfa30 2024-02-26T10:40:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.703+0000 7f4bd8200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd006e920 con 0x7f4bd00c1090 2024-02-26T10:40:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.704+0000 7f4bd5800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4bd0152420 0x7f4bd00c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46466/0 (socket says 172.21.15.67:46466) 2024-02-26T10:40:55.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.704+0000 7f4bd5800640 1 -- 172.21.15.67:0/139796269 learned_addr learned my addr 172.21.15.67:0/139796269 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.704+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144021271 0 0) 0x7f4bd00c15e0 con 0x7f4bd0152420 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.704+0000 7f4bd6200640 1 --2- 172.21.15.67:0/139796269 >> v2:172.21.15.82:3300/0 conn(0x7f4bd00c1090 0x7f4bd01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.704+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb4003520 con 0x7f4bd0152420 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544120037 0 0) 0x7f4bb4003520 con 0x7f4bd0152420 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 >> v2:172.21.15.82:3300/0 conn(0x7f4bd00c1090 msgr2=0x7f4bd01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 --2- 172.21.15.67:0/139796269 >> v2:172.21.15.82:3300/0 conn(0x7f4bd00c1090 0x7f4bd01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 >> v1:172.21.15.82:6789/0 conn(0x7f4bd00bfa30 legacy=0x7f4bd0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd00c1880 con 0x7f4bd0152420 2024-02-26T10:40:55.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f4bc0001670 con 0x7f4bd0152420 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 >> v1:172.21.15.67:6789/0 conn(0x7f4bd0152420 legacy=0x7f4bd00c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bb4003a00 0x7f4bb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bb4004800 0x7f4bb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4005210 0x7f4bb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd006e920 con 0x7f4bb4003a00 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb400c160 con 0x7f4bb4004800 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb400c360 con 0x7f4bb4005210 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bd5800640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bb4003a00 0x7f4bb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.705+0000 7f4bd4e00640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bb4004800 0x7f4bb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd5800640 1 -- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4005210 msgr2=0x7f4bb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd5800640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4005210 0x7f4bb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd5800640 1 -- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bb4004800 msgr2=0x7f4bb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd5800640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bb4004800 0x7f4bb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd5800640 1 -- 172.21.15.67:0/139796269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4003520 con 0x7f4bb4003a00 2024-02-26T10:40:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd6200640 1 -- 172.21.15.67:0/139796269 reap_dead start 2024-02-26T10:40:55.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.706+0000 7f4bd4e00640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bb4004800 0x7f4bb4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.707+0000 7f4bd5800640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bb4003a00 0x7f4bb4003e20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c0e0 tx=0x7f4bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271d137a2f5fbaaf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.707+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007c40 con 0x7f4bb4003a00 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.707+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc0030070 con 0x7f4bb4003a00 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.707+0000 7f4bcfe00640 1 -- 172.21.15.67:0/139796269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0005d70 con 0x7f4bb4003a00 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 -- 172.21.15.67:0/139796269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bb4003a00 msgr2=0x7f4bb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bb4003a00 0x7f4bb4003e20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c0e0 tx=0x7f4bb4005830 comp rx=0 tx=0).stop 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 -- 172.21.15.67:0/139796269 shutdown_connections 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 --2- 172.21.15.67:0/139796269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bb4003a00 0x7f4bb4003e20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 -- 172.21.15.67:0/139796269 >> 172.21.15.67:0/139796269 conn(0x7f4bd00a84a0 msgr2=0x7f4bd00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 -- 172.21.15.67:0/139796269 shutdown_connections 2024-02-26T10:40:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.708+0000 7f4bd8200640 1 -- 172.21.15.67:0/139796269 wait complete. 2024-02-26T10:40:55.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.709+0000 7f4bd8200640 1 Processor -- start 2024-02-26T10:40:55.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.710+0000 7f4bd8200640 1 -- start start 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.710+0000 7f4bd8200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4003a00 0x7f4bd01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd8200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 0x7f4bd01432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd8200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bd00ab070 0x7f4bd0143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd8200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0152800 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd8200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0158400 con 0x7f4bd00ab070 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd8200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0154e10 con 0x7f4bb4003a00 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 0x7f4bd01432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 0x7f4bd01432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40790/0 (socket says 172.21.15.67:40790) 2024-02-26T10:40:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 -- 172.21.15.67:0/2317050328 learned_addr learned my addr 172.21.15.67:0/2317050328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd6200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bd00ab070 0x7f4bd0143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd5800640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4003a00 0x7f4bd01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 -- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4003a00 msgr2=0x7f4bd01574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4003a00 0x7f4bd01574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 -- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bd00ab070 msgr2=0x7f4bd0143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bd00ab070 0x7f4bd0143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.711+0000 7f4bd4e00640 1 -- 172.21.15.67:0/2317050328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0143d60 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.712+0000 7f4bd4e00640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 0x7f4bd01432e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d4b0 tx=0x7f4bbc00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.712+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc005480 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.712+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc017070 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.712+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc003850 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.712+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd00aac50 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.712+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0144390 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.714+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd0074a30 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.714+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bbc0039f0 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.715+0000 7f4bcd600640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4bb006b500 0x7f4bb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.715+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bbc089f20 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.716+0000 7f4bd5800640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4bb006b500 0x7f4bb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:55.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.717+0000 7f4bd5800640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4bb006b500 0x7f4bb006d9c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0034000 tx=0x7f4bc0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:55.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.721+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bbc00f2f0 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.986+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7f4bd006e440 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.987+0000 7f4bcd600640 1 -- 172.21.15.67:0/2317050328 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v13) v1 ==== 84+0+0 (secure 0 0 0) 0x7f4bbc058d90 con 0x7f4bd01ee5e0 2024-02-26T10:40:55.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4bb006b500 msgr2=0x7f4bb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4bb006b500 0x7f4bb006d9c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0034000 tx=0x7f4bc0009030 comp rx=0 tx=0).stop 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 msgr2=0x7f4bd01432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 0x7f4bd01432e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d4b0 tx=0x7f4bbc00d970 comp rx=0 tx=0).stop 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 shutdown_connections 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4bb006b500 0x7f4bb006d9c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bd00ab070 0x7f4bd0143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4bd01ee5e0 0x7f4bd01432e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 --2- 172.21.15.67:0/2317050328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bb4003a00 0x7f4bd01574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 >> 172.21.15.67:0/2317050328 conn(0x7f4bd00a84a0 msgr2=0x7f4bd0158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 shutdown_connections 2024-02-26T10:40:55.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:55.989+0000 7f4bd8200640 1 -- 172.21.15.67:0/2317050328 wait complete. 2024-02-26T10:40:56.006 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:40:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.217+0000 7f7463200640 1 Processor -- start 2024-02-26T10:40:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.217+0000 7f7463200640 1 -- start start 2024-02-26T10:40:56.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7463200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74640c0850 0x7f746406ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7463200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f746406e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7463200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746406dca0 con 0x7f74640c1c10 2024-02-26T10:40:56.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7463200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640c2a70 con 0x7f74640c0850 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7463200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74640c21d0 con 0x7f746406b160 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7461400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74640c0850 0x7f746406ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7461400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74640c0850 0x7f746406ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40798/0 (socket says 172.21.15.67:40798) 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7461400640 1 -- 172.21.15.67:0/661542528 learned_addr learned my addr 172.21.15.67:0/661542528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.219+0000 7f7461e00640 1 --1- 172.21.15.67:0/661542528 >> v1:172.21.15.67:6789/0 conn(0x7f74640c1c10 0x7f74641557b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46482/0 (socket says 172.21.15.67:46482) 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.220+0000 7f7462800640 1 --2- 172.21.15.67:0/661542528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f746406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.220+0000 7f7461400640 1 -- 172.21.15.67:0/661542528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 msgr2=0x7f746406e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.220+0000 7f7461400640 1 --2- 172.21.15.67:0/661542528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f746406e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.220+0000 7f7461400640 1 -- 172.21.15.67:0/661542528 >> v1:172.21.15.67:6789/0 conn(0x7f74640c1c10 legacy=0x7f74641557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.220+0000 7f7461400640 1 -- 172.21.15.67:0/661542528 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640c24b0 con 0x7f74640c0850 2024-02-26T10:40:56.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.221+0000 7f7461400640 1 --2- 172.21.15.67:0/661542528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c0850 0x7f746406ac20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7450009870 tx=0x7f745002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b75402f8fbe92b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:56.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.221+0000 7f7460a00640 1 -- 172.21.15.67:0/661542528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74500317c0 con 0x7f74640c0850 2024-02-26T10:40:56.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.221+0000 7f7460a00640 1 -- 172.21.15.67:0/661542528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450037070 con 0x7f74640c0850 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.221+0000 7f7460a00640 1 -- 172.21.15.67:0/661542528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450031b80 con 0x7f74640c0850 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 -- 172.21.15.67:0/661542528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c0850 msgr2=0x7f746406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 --2- 172.21.15.67:0/661542528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c0850 0x7f746406ac20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7450009870 tx=0x7f745002b860 comp rx=0 tx=0).stop 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 -- 172.21.15.67:0/661542528 shutdown_connections 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 --2- 172.21.15.67:0/661542528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f746406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 --2- 172.21.15.67:0/661542528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c0850 0x7f746406ac20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 -- 172.21.15.67:0/661542528 >> 172.21.15.67:0/661542528 conn(0x7f74640c63f0 msgr2=0x7f74640c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 -- 172.21.15.67:0/661542528 shutdown_connections 2024-02-26T10:40:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.222+0000 7f7463200640 1 -- 172.21.15.67:0/661542528 wait complete. 2024-02-26T10:40:56.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.224+0000 7f7463200640 1 Processor -- start 2024-02-26T10:40:56.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.224+0000 7f7463200640 1 -- start start 2024-02-26T10:40:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7463200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f74641df860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7463200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74640c0850 0x7f7464146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7463200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 0x7f74641ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7463200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640c0f70 con 0x7f74640c1c10 2024-02-26T10:40:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7463200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640c0df0 con 0x7f74640c0850 2024-02-26T10:40:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7463200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746406e6c0 con 0x7f746406b160 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7462800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 0x7f74641ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7462800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 0x7f74641ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40808/0 (socket says 172.21.15.67:40808) 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7462800640 1 -- 172.21.15.67:0/2885900081 learned_addr learned my addr 172.21.15.67:0/2885900081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.225+0000 7f7461e00640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f74641df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7461400640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74640c0850 0x7f7464146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7462800640 1 -- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 msgr2=0x7f74641df860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7462800640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f74641df860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7462800640 1 -- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74640c0850 msgr2=0x7f7464146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7462800640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74640c0850 0x7f7464146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7462800640 1 -- 172.21.15.67:0/2885900081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74641e0640 con 0x7f74640c1c10 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7461400640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74640c0850 0x7f7464146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7462800640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 0x7f74641ee010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f745c007e40 tx=0x7f745c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c014620 con 0x7f74640c1c10 2024-02-26T10:40:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c019070 con 0x7f74640c1c10 2024-02-26T10:40:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c014a20 con 0x7f74640c1c10 2024-02-26T10:40:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.226+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74641df0a0 con 0x7f74640c1c10 2024-02-26T10:40:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.227+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74641eebc0 con 0x7f74640c1c10 2024-02-26T10:40:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.228+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74641552f0 con 0x7f74640c1c10 2024-02-26T10:40:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.228+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f745c014bc0 con 0x7f74640c1c10 2024-02-26T10:40:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.230+0000 7f745a000640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f744406b4d0 0x7f744406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.230+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f745c08ab60 con 0x7f74640c1c10 2024-02-26T10:40:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.230+0000 7f7461e00640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f744406b4d0 0x7f744406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.232+0000 7f7461e00640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f744406b4d0 0x7f744406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7454007fd0 tx=0x7f7454007c40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:56.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.235+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74641552f0 con 0x7f74640c1c10 2024-02-26T10:40:56.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.435+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7464074a30 con 0x7f74640c1c10 2024-02-26T10:40:56.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.437+0000 7f745a000640 1 -- 172.21.15.67:0/2885900081 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f74641e0640 con 0x7f74640c1c10 2024-02-26T10:40:56.437 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:56.437 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:40:46.616937+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:40:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.439+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f744406b4d0 msgr2=0x7f744406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.439+0000 7f7463200640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f744406b4d0 0x7f744406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7454007fd0 tx=0x7f7454007c40 comp rx=0 tx=0).stop 2024-02-26T10:40:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.439+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 msgr2=0x7f74641ee010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.439+0000 7f7463200640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 0x7f74641ee010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f745c007e40 tx=0x7f745c00e4b0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 shutdown_connections 2024-02-26T10:40:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74640c1c10 0x7f74641ee010 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f744406b4d0 0x7f744406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74640c0850 0x7f7464146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 --2- 172.21.15.67:0/2885900081 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f746406b160 0x7f74641df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 >> 172.21.15.67:0/2885900081 conn(0x7f74640c63f0 msgr2=0x7f74640c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 shutdown_connections 2024-02-26T10:40:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.440+0000 7f7463200640 1 -- 172.21.15.67:0/2885900081 wait complete. 2024-02-26T10:40:56.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:40:43.472687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:40:44.096497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:40:56.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:40:56.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.684+0000 7fe652000640 1 Processor -- start 2024-02-26T10:40:56.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.684+0000 7fe652000640 1 -- start start 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe652000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c1523b0 0x7fe64c0c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe652000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe64c071fd0 0x7fe64c072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe652000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64c0c3840 con 0x7fe64c154cf0 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe652000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64c152790 con 0x7fe64c071fd0 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe652000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c06da30 con 0x7fe64c1523b0 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe64be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe64c154cf0 0x7fe64c0c3130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46496/0 (socket says 172.21.15.67:46496) 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.686+0000 7fe64be00640 1 -- 172.21.15.67:0/2863655744 learned_addr learned my addr 172.21.15.67:0/2863655744 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:56.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe651600640 1 --2- 172.21.15.67:0/2863655744 >> v2:172.21.15.67:3300/0 conn(0x7fe64c071fd0 0x7fe64c072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe650c00640 1 --2- 172.21.15.67:0/2863655744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c1523b0 0x7fe64c0c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe64b400640 1 -- 172.21.15.67:0/2863655744 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349932400 0 0) 0x7fe64c0c3840 con 0x7fe64c154cf0 2024-02-26T10:40:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe64b400640 1 -- 172.21.15.67:0/2863655744 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe634003520 con 0x7fe64c154cf0 2024-02-26T10:40:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe651600640 1 -- 172.21.15.67:0/2863655744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c1523b0 msgr2=0x7fe64c0c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe651600640 1 --2- 172.21.15.67:0/2863655744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c1523b0 0x7fe64c0c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe651600640 1 -- 172.21.15.67:0/2863655744 >> v1:172.21.15.67:6789/0 conn(0x7fe64c154cf0 legacy=0x7fe64c0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe64b400640 1 -- 172.21.15.67:0/2863655744 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 603191971 0 0) 0x7fe634003520 con 0x7fe64c154cf0 2024-02-26T10:40:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.687+0000 7fe651600640 1 -- 172.21.15.67:0/2863655744 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c06dc80 con 0x7fe64c071fd0 2024-02-26T10:40:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.689+0000 7fe651600640 1 --2- 172.21.15.67:0/2863655744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c071fd0 0x7fe64c072440 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe644009d90 tx=0x7fe64402bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc671ecbba94196a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.689+0000 7fe64b400640 1 -- 172.21.15.67:0/2863655744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644039070 con 0x7fe64c071fd0 2024-02-26T10:40:56.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.689+0000 7fe64b400640 1 -- 172.21.15.67:0/2863655744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64402c810 con 0x7fe64c071fd0 2024-02-26T10:40:56.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.689+0000 7fe64b400640 1 -- 172.21.15.67:0/2863655744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638001670 con 0x7fe64c071fd0 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.689+0000 7fe652000640 1 -- 172.21.15.67:0/2863655744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c071fd0 msgr2=0x7fe64c072440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.689+0000 7fe652000640 1 --2- 172.21.15.67:0/2863655744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c071fd0 0x7fe64c072440 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe644009d90 tx=0x7fe64402bce0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.690+0000 7fe652000640 1 -- 172.21.15.67:0/2863655744 shutdown_connections 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.690+0000 7fe652000640 1 --2- 172.21.15.67:0/2863655744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c071fd0 0x7fe64c072440 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.690+0000 7fe652000640 1 --2- 172.21.15.67:0/2863655744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c1523b0 0x7fe64c0c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.690+0000 7fe652000640 1 -- 172.21.15.67:0/2863655744 >> 172.21.15.67:0/2863655744 conn(0x7fe64c0c6380 msgr2=0x7fe64c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.690+0000 7fe652000640 1 -- 172.21.15.67:0/2863655744 shutdown_connections 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.690+0000 7fe652000640 1 -- 172.21.15.67:0/2863655744 wait complete. 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.691+0000 7fe652000640 1 Processor -- start 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.691+0000 7fe652000640 1 -- start start 2024-02-26T10:40:56.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.692+0000 7fe652000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe64c071fd0 0x7fe64c0c1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.692+0000 7fe652000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 0x7fe64c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.692+0000 7fe652000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c154cf0 0x7fe64c14b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.692+0000 7fe652000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c06e520 con 0x7fe64c1523b0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.692+0000 7fe652000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c06e240 con 0x7fe64c071fd0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.692+0000 7fe652000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c06e800 con 0x7fe64c154cf0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 0x7fe64c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 0x7fe64c1499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40832/0 (socket says 172.21.15.67:40832) 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 -- 172.21.15.67:0/3938402604 learned_addr learned my addr 172.21.15.67:0/3938402604 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 -- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c154cf0 msgr2=0x7fe64c14b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c154cf0 0x7fe64c14b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 -- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe64c071fd0 msgr2=0x7fe64c0c1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe64c071fd0 0x7fe64c0c1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 -- 172.21.15.67:0/3938402604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c1ee6c0 con 0x7fe64c1523b0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.693+0000 7fe64be00640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 0x7fe64c1499d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c700 tx=0x7fe63800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.694+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638017070 con 0x7fe64c1523b0 2024-02-26T10:40:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.694+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe638005ad0 con 0x7fe64c1523b0 2024-02-26T10:40:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.694+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c14ce20 con 0x7fe64c1523b0 2024-02-26T10:40:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.694+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638007070 con 0x7fe64c1523b0 2024-02-26T10:40:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.694+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c0c19d0 con 0x7fe64c1523b0 2024-02-26T10:40:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.695+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c0678e0 con 0x7fe64c1523b0 2024-02-26T10:40:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.696+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe63801a070 con 0x7fe64c1523b0 2024-02-26T10:40:56.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.697+0000 7fe648c00640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe62c06b4d0 0x7fe62c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:56.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.697+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe638089650 con 0x7fe64c1523b0 2024-02-26T10:40:56.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.698+0000 7fe650c00640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe62c06b4d0 0x7fe62c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:56.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.699+0000 7fe650c00640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe62c06b4d0 0x7fe62c06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe640006a90 tx=0x7fe640006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:56.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.703+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe638058540 con 0x7fe64c1523b0 2024-02-26T10:40:56.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.903+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe64c154270 con 0x7fe64c1523b0 2024-02-26T10:40:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.904+0000 7fe648c00640 1 -- 172.21.15.67:0/3938402604 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe638057e40 con 0x7fe64c1523b0 2024-02-26T10:40:56.904 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe62c06b4d0 msgr2=0x7fe62c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe62c06b4d0 0x7fe62c06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe640006a90 tx=0x7fe640006a20 comp rx=0 tx=0).stop 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 msgr2=0x7fe64c1499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 0x7fe64c1499d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c700 tx=0x7fe63800cbc0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 shutdown_connections 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe64c154cf0 0x7fe64c14b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe64c1523b0 0x7fe64c1499d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe62c06b4d0 0x7fe62c06d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 --2- 172.21.15.67:0/3938402604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe64c071fd0 0x7fe64c0c1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 >> 172.21.15.67:0/3938402604 conn(0x7fe64c0c6380 msgr2=0x7fe64c06ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 shutdown_connections 2024-02-26T10:40:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:56.907+0000 7fe652000640 1 -- 172.21.15.67:0/3938402604 wait complete. 2024-02-26T10:40:56.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:40:57.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.145+0000 7f3337600640 1 Processor -- start 2024-02-26T10:40:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.145+0000 7f3337600640 1 -- start start 2024-02-26T10:40:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3337600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33301591b0 0x7f3330158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3337600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f3330070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3337600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3330072ce0 con 0x7f3330071410 2024-02-26T10:40:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3337600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3330073010 con 0x7f33301591b0 2024-02-26T10:40:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3337600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333014f490 con 0x7f33300c0ef0 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3335600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3330071410 0x7f333006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46500/0 (socket says 172.21.15.67:46500) 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f3335600640 1 -- 172.21.15.67:0/2217622304 learned_addr learned my addr 172.21.15.67:0/2217622304 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.147+0000 7f332fe00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f3330070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644015996 0 0) 0x7f3330072ce0 con 0x7f3330071410 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f3334c00640 1 --2- 172.21.15.67:0/2217622304 >> v2:172.21.15.82:3300/0 conn(0x7f33301591b0 0x7f3330158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3318003520 con 0x7f3330071410 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 501497997 0 0) 0x7f3318003520 con 0x7f3330071410 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 msgr2=0x7f3330070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f3330070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 >> v2:172.21.15.82:3300/0 conn(0x7f33301591b0 msgr2=0x7f3330158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 --2- 172.21.15.67:0/2217622304 >> v2:172.21.15.82:3300/0 conn(0x7f33301591b0 0x7f3330158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330155bf0 con 0x7f3330071410 2024-02-26T10:40:57.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f33280019f0 con 0x7f3330071410 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332fe00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f3330070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 >> v1:172.21.15.67:6789/0 conn(0x7f3330071410 legacy=0x7f333006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f3318003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3318004800 0x7f3318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f33180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3330073010 con 0x7f33180039b0 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333014f490 con 0x7f3318004800 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.148+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331800baa0 con 0x7f33300c0ef0 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3334c00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f3318003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3335600640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f33180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f332fe00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3318004800 0x7f3318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3334c00640 1 -- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 msgr2=0x7f33180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3334c00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f33180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3334c00640 1 -- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3318004800 msgr2=0x7f3318004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3334c00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3318004800 0x7f3318004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3334c00640 1 -- 172.21.15.67:0/2217622304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318003520 con 0x7f33180039b0 2024-02-26T10:40:57.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f3335600640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33300c0ef0 0x7f33180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:40:57.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.149+0000 7f332fe00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3318004800 0x7f3318004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f3334c00640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f3318003df0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3324009a90 tx=0x7f3318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5c104cbaf68723 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324039070 con 0x7f33180039b0 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332402c6b0 con 0x7f33180039b0 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f332f400640 1 -- 172.21.15.67:0/2217622304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324009690 con 0x7f33180039b0 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f3337600640 1 -- 172.21.15.67:0/2217622304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 msgr2=0x7f3318003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f3337600640 1 --2- 172.21.15.67:0/2217622304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f3318003df0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3324009a90 tx=0x7f3318005830 comp rx=0 tx=0).stop 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.151+0000 7f3335600640 1 -- 172.21.15.67:0/2217622304 reap_dead start 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.152+0000 7f3337600640 1 -- 172.21.15.67:0/2217622304 shutdown_connections 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.152+0000 7f3337600640 1 -- 172.21.15.67:0/2217622304 >> 172.21.15.67:0/2217622304 conn(0x7f33300c6da0 msgr2=0x7f33300c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.152+0000 7f3337600640 1 -- 172.21.15.67:0/2217622304 shutdown_connections 2024-02-26T10:40:57.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.152+0000 7f3337600640 1 -- 172.21.15.67:0/2217622304 wait complete. 2024-02-26T10:40:57.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.153+0000 7f3337600640 1 Processor -- start 2024-02-26T10:40:57.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.154+0000 7f3337600640 1 -- start start 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.154+0000 7f3337600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f33300ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.154+0000 7f3337600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33301e2210 0x7f33301edd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.154+0000 7f3337600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33301e0440 0x7f33301ee290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.154+0000 7f3337600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333006e6e0 con 0x7f33180039b0 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.154+0000 7f3337600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333006e560 con 0x7f33301e0440 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3337600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300a8ba0 con 0x7f33301e2210 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f33300ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f33300ab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40850/0 (socket says 172.21.15.67:40850) 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 -- 172.21.15.67:0/2644786465 learned_addr learned my addr 172.21.15.67:0/2644786465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 -- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33301e2210 msgr2=0x7f33301edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33301e2210 0x7f33301edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 -- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33301e0440 msgr2=0x7f33301ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33301e0440 0x7f33301ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 -- 172.21.15.67:0/2644786465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33301df8b0 con 0x7f33180039b0 2024-02-26T10:40:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.155+0000 7f3334c00640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f33300ab800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3324009bb0 tx=0x7f3324004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:57.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.156+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324039050 con 0x7f33180039b0 2024-02-26T10:40:57.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.156+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3324045070 con 0x7f33180039b0 2024-02-26T10:40:57.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.156+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324003850 con 0x7f33180039b0 2024-02-26T10:40:57.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.156+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33301e0be0 con 0x7f33180039b0 2024-02-26T10:40:57.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.156+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333014f9d0 con 0x7f33180039b0 2024-02-26T10:40:57.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.157+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3330074a30 con 0x7f33180039b0 2024-02-26T10:40:57.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.158+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33240039f0 con 0x7f33180039b0 2024-02-26T10:40:57.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.159+0000 7f332cc00640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f331006b500 0x7f331006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.159+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33240abbd0 con 0x7f33180039b0 2024-02-26T10:40:57.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.160+0000 7f332fe00640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f331006b500 0x7f331006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.161+0000 7f332fe00640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f331006b500 0x7f331006d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f33300c1850 tx=0x7f331c006df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.164+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f332404e020 con 0x7f33180039b0 2024-02-26T10:40:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.361+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f333006eae0 con 0x7f33180039b0 2024-02-26T10:40:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.362+0000 7f332cc00640 1 -- 172.21.15.67:0/2644786465 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f332407aa40 con 0x7f33180039b0 2024-02-26T10:40:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:40:57.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.364+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f331006b500 msgr2=0x7f331006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.364+0000 7f3337600640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f331006b500 0x7f331006d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f33300c1850 tx=0x7f331c006df0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 msgr2=0x7f33300ab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f33300ab800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3324009bb0 tx=0x7f3324004730 comp rx=0 tx=0).stop 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 shutdown_connections 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f331006b500 0x7f331006d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33301e0440 0x7f33301ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33301e2210 0x7f33301edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 --2- 172.21.15.67:0/2644786465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33180039b0 0x7f33300ab800 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 >> 172.21.15.67:0/2644786465 conn(0x7f33300c6da0 msgr2=0x7f33300c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 shutdown_connections 2024-02-26T10:40:57.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.365+0000 7f3337600640 1 -- 172.21.15.67:0/2644786465 wait complete. 2024-02-26T10:40:57.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:40:57.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.600+0000 7f73a1e00640 1 Processor -- start 2024-02-26T10:40:57.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.600+0000 7f73a1e00640 1 -- start start 2024-02-26T10:40:57.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.601+0000 7f73a1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f739c0c25a0 0x7f739c155880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.601+0000 7f73a1e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c151d70 con 0x7f739c0a6150 2024-02-26T10:40:57.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.601+0000 7f73a1e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c152030 con 0x7f739c06e860 2024-02-26T10:40:57.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.601+0000 7f73a1e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739c152b10 con 0x7f739c0c25a0 2024-02-26T10:40:57.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.602+0000 7f739aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f739c0a6150 0x7f739c155170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46502/0 (socket says 172.21.15.67:46502) 2024-02-26T10:40:57.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.602+0000 7f739aa00640 1 -- 172.21.15.67:0/1548204123 learned_addr learned my addr 172.21.15.67:0/1548204123 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:57.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.602+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423558252 0 0) 0x7f739c151d70 con 0x7f739c0a6150 2024-02-26T10:40:57.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.602+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7380003520 con 0x7f739c0a6150 2024-02-26T10:40:57.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739b400640 1 --2- 172.21.15.67:0/1548204123 >> v2:172.21.15.82:3300/0 conn(0x7f739c0c25a0 0x7f739c155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682408200 0 0) 0x7f7380003520 con 0x7f739c0a6150 2024-02-26T10:40:57.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 >> v2:172.21.15.82:3300/0 conn(0x7f739c0c25a0 msgr2=0x7f739c155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 --2- 172.21.15.67:0/1548204123 >> v2:172.21.15.82:3300/0 conn(0x7f739c0c25a0 0x7f739c155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 >> v1:172.21.15.82:6789/0 conn(0x7f739c06e860 legacy=0x7f739c15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c152310 con 0x7f739c0a6150 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f738c0019f0 con 0x7f739c0a6150 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 >> v1:172.21.15.67:6789/0 conn(0x7f739c0a6150 legacy=0x7f739c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7380003a00 0x7f7380003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7380004800 0x7f7380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f738000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c152b10 con 0x7f73800052e0 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738000c230 con 0x7f7380003a00 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738000c430 con 0x7f7380004800 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.603+0000 7f739b400640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f738000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f739aa00640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7380003a00 0x7f7380003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f7392000640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7380004800 0x7f7380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f739b400640 1 -- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7380004800 msgr2=0x7f7380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f739b400640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7380004800 0x7f7380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f739b400640 1 -- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7380003a00 msgr2=0x7f7380003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:57.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f739b400640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7380003a00 0x7f7380003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.604+0000 7f739b400640 1 -- 172.21.15.67:0/1548204123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7380003520 con 0x7f73800052e0 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.605+0000 7f739b400640 1 -- 172.21.15.67:0/1548204123 reap_dead start 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.605+0000 7f739b400640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f738000b9c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f739400a970 tx=0x7f7380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfba0f8521bc9ef2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.605+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394034070 con 0x7f73800052e0 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.605+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394030620 con 0x7f73800052e0 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.605+0000 7f739a000640 1 -- 172.21.15.67:0/1548204123 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739400a570 con 0x7f73800052e0 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 -- 172.21.15.67:0/1548204123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 msgr2=0x7f738000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f738000b9c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f739400a970 tx=0x7f7380005830 comp rx=0 tx=0).stop 2024-02-26T10:40:57.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 -- 172.21.15.67:0/1548204123 shutdown_connections 2024-02-26T10:40:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1548204123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f738000b9c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 -- 172.21.15.67:0/1548204123 >> 172.21.15.67:0/1548204123 conn(0x7f739c0c6da0 msgr2=0x7f739c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 -- 172.21.15.67:0/1548204123 shutdown_connections 2024-02-26T10:40:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.606+0000 7f73a1e00640 1 -- 172.21.15.67:0/1548204123 wait complete. 2024-02-26T10:40:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.608+0000 7f73a1e00640 1 Processor -- start 2024-02-26T10:40:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.608+0000 7f73a1e00640 1 -- start start 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f73a1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f739c1edfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f73a1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c1ee4f0 0x7f739c0c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f73a1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f739c0c4620 0x7f739c13b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f73a1e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c14f3a0 con 0x7f73800052e0 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f73a1e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c152680 con 0x7f739c1ee4f0 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f73a1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c14f520 con 0x7f739c0c4620 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f739aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f739c1edfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f739aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f739c1edfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40860/0 (socket says 172.21.15.67:40860) 2024-02-26T10:40:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f739aa00640 1 -- 172.21.15.67:0/1785567319 learned_addr learned my addr 172.21.15.67:0/1785567319 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.609+0000 7f7392000640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c1ee4f0 0x7f739c0c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739b400640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f739c0c4620 0x7f739c13b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739aa00640 1 -- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f739c0c4620 msgr2=0x7f739c13b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739aa00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f739c0c4620 0x7f739c13b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739aa00640 1 -- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c1ee4f0 msgr2=0x7f739c0c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739aa00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c1ee4f0 0x7f739c0c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739aa00640 1 -- 172.21.15.67:0/1785567319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c13fdd0 con 0x7f73800052e0 2024-02-26T10:40:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f739aa00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f739c1edfb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f738c007a10 tx=0x7f738c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:57.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00edf0 con 0x7f73800052e0 2024-02-26T10:40:57.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c018070 con 0x7f73800052e0 2024-02-26T10:40:57.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c013660 con 0x7f73800052e0 2024-02-26T10:40:57.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.610+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c1400b0 con 0x7f73800052e0 2024-02-26T10:40:57.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.611+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c13d190 con 0x7f73800052e0 2024-02-26T10:40:57.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.612+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c1405d0 con 0x7f73800052e0 2024-02-26T10:40:57.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.612+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f738c013800 con 0x7f73800052e0 2024-02-26T10:40:57.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.614+0000 7f7393e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f737806b4d0 0x7f737806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:57.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.614+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f738c08a880 con 0x7f73800052e0 2024-02-26T10:40:57.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.614+0000 7f7392000640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f737806b4d0 0x7f737806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:57.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.616+0000 7f7392000640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f737806b4d0 0x7f737806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f739c14edb0 tx=0x7f7384009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:57.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.619+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f738c059660 con 0x7f73800052e0 2024-02-26T10:40:57.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.821+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f739c074a30 con 0x7f73800052e0 2024-02-26T10:40:57.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.822+0000 7f7393e00640 1 -- 172.21.15.67:0/1785567319 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f738c058f60 con 0x7f73800052e0 2024-02-26T10:40:57.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f737806b4d0 msgr2=0x7f737806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f737806b4d0 0x7f737806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f739c14edb0 tx=0x7f7384009040 comp rx=0 tx=0).stop 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 msgr2=0x7f739c1edfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f739c1edfb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f738c007a10 tx=0x7f738c00e530 comp rx=0 tx=0).stop 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 shutdown_connections 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f739c0c4620 0x7f739c13b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f737806b4d0 0x7f737806d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f739c1ee4f0 0x7f739c0c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 --2- 172.21.15.67:0/1785567319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73800052e0 0x7f739c1edfb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:57.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.825+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 >> 172.21.15.67:0/1785567319 conn(0x7f739c0c6da0 msgr2=0x7f739c0bf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.826+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 shutdown_connections 2024-02-26T10:40:57.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:57.826+0000 7f73a1e00640 1 -- 172.21.15.67:0/1785567319 wait complete. 2024-02-26T10:40:57.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:40:58.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.064+0000 7fd29fc00640 1 Processor -- start 2024-02-26T10:40:58.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.065+0000 7fd29fc00640 1 -- start start 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.066+0000 7fd29fc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2980c7d70 0x7fd2980c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.066+0000 7fd29fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2980c85b0 0x7fd29814c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.066+0000 7fd29fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2980c56a0 con 0x7fd2980c3e70 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.066+0000 7fd29fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd298152ec0 con 0x7fd2980c85b0 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.066+0000 7fd29fc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2980c5980 con 0x7fd2980c7d70 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd297e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd2980c3e70 0x7fd298153350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46516/0 (socket says 172.21.15.67:46516) 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd297e00640 1 -- 172.21.15.67:0/86591485 learned_addr learned my addr 172.21.15.67:0/86591485 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:58.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd29dc00640 1 --2- 172.21.15.67:0/86591485 >> v2:172.21.15.67:3300/0 conn(0x7fd2980c85b0 0x7fd29814c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd29d200640 1 --2- 172.21.15.67:0/86591485 >> v2:172.21.15.82:3300/0 conn(0x7fd2980c7d70 0x7fd2980c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd29dc00640 1 -- 172.21.15.67:0/86591485 >> v2:172.21.15.82:3300/0 conn(0x7fd2980c7d70 msgr2=0x7fd2980c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:58.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd29dc00640 1 --2- 172.21.15.67:0/86591485 >> v2:172.21.15.82:3300/0 conn(0x7fd2980c7d70 0x7fd2980c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd29dc00640 1 -- 172.21.15.67:0/86591485 >> v1:172.21.15.67:6789/0 conn(0x7fd2980c3e70 legacy=0x7fd298153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd297400640 1 -- 172.21.15.67:0/86591485 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221843555 0 0) 0x7fd2980c56a0 con 0x7fd2980c3e70 2024-02-26T10:40:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.067+0000 7fd29dc00640 1 -- 172.21.15.67:0/86591485 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980c5c60 con 0x7fd2980c85b0 2024-02-26T10:40:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.069+0000 7fd29dc00640 1 --2- 172.21.15.67:0/86591485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c85b0 0x7fd29814c430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd29806f8b0 tx=0x7fd29002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea35cc253b2b594 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.069+0000 7fd297400640 1 -- 172.21.15.67:0/86591485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290039070 con 0x7fd2980c85b0 2024-02-26T10:40:58.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.069+0000 7fd297400640 1 -- 172.21.15.67:0/86591485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29002cc90 con 0x7fd2980c85b0 2024-02-26T10:40:58.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.069+0000 7fd297400640 1 -- 172.21.15.67:0/86591485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290034660 con 0x7fd2980c85b0 2024-02-26T10:40:58.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 -- 172.21.15.67:0/86591485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c85b0 msgr2=0x7fd29814c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 --2- 172.21.15.67:0/86591485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c85b0 0x7fd29814c430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd29806f8b0 tx=0x7fd29002bee0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 -- 172.21.15.67:0/86591485 shutdown_connections 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 --2- 172.21.15.67:0/86591485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c85b0 0x7fd29814c430 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 --2- 172.21.15.67:0/86591485 >> v2:172.21.15.82:3300/0 conn(0x7fd2980c7d70 0x7fd2980c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 -- 172.21.15.67:0/86591485 >> 172.21.15.67:0/86591485 conn(0x7fd29806b040 msgr2=0x7fd29806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 -- 172.21.15.67:0/86591485 shutdown_connections 2024-02-26T10:40:58.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.070+0000 7fd29fc00640 1 -- 172.21.15.67:0/86591485 wait complete. 2024-02-26T10:40:58.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.072+0000 7fd29fc00640 1 Processor -- start 2024-02-26T10:40:58.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.072+0000 7fd29fc00640 1 -- start start 2024-02-26T10:40:58.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 0x7fd29814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29fc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2980c7d70 0x7fd298151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2980c85b0 0x7fd2981ea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980c4ed0 con 0x7fd2980c3e70 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29fc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980c4d50 con 0x7fd2980c7d70 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2980c8150 con 0x7fd2980c85b0 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 0x7fd29814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 0x7fd29814bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40882/0 (socket says 172.21.15.67:40882) 2024-02-26T10:40:58.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.073+0000 7fd29d200640 1 -- 172.21.15.67:0/883819965 learned_addr learned my addr 172.21.15.67:0/883819965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:58.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29dc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2980c85b0 0x7fd2981ea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd297e00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2980c7d70 0x7fd298151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29d200640 1 -- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2980c85b0 msgr2=0x7fd2981ea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29d200640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2980c85b0 0x7fd2981ea6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29d200640 1 -- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2980c7d70 msgr2=0x7fd298151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29d200640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2980c7d70 0x7fd298151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29d200640 1 -- 172.21.15.67:0/883819965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29814c140 con 0x7fd2980c3e70 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd297e00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2980c7d70 0x7fd298151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29d200640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 0x7fd29814bc00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd28800d800 tx=0x7fd28800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800b6c0 con 0x7fd2980c3e70 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288019070 con 0x7fd2980c3e70 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2980a9270 con 0x7fd2980c3e70 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.074+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800ba80 con 0x7fd2980c3e70 2024-02-26T10:40:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.075+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2981ee090 con 0x7fd2980c3e70 2024-02-26T10:40:58.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.076+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298074a30 con 0x7fd2980c3e70 2024-02-26T10:40:58.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.077+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28800bc20 con 0x7fd2980c3e70 2024-02-26T10:40:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.077+0000 7fd294c00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd27806b4d0 0x7fd27806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.078+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd28808a6c0 con 0x7fd2980c3e70 2024-02-26T10:40:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.078+0000 7fd297e00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd27806b4d0 0x7fd27806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.079+0000 7fd297e00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd27806b4d0 0x7fd27806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd2980a7fc0 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:58.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.082+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd288059530 con 0x7fd2980c3e70 2024-02-26T10:40:58.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.287+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd2980adba0 con 0x7fd2980c3e70 2024-02-26T10:40:58.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.288+0000 7fd294c00640 1 -- 172.21.15.67:0/883819965 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+207 (secure 0 0 0) 0x7fd288058e30 con 0x7fd2980c3e70 2024-02-26T10:40:58.287 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:40:58.287 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:40:58.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 8, 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "cephfs_metadata" 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 9, 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "cephfs_data" 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:40:58.288 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd27806b4d0 msgr2=0x7fd27806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd27806b4d0 0x7fd27806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd2980a7fc0 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 msgr2=0x7fd29814bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 0x7fd29814bc00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd28800d800 tx=0x7fd28800dcc0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 shutdown_connections 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2980c85b0 0x7fd2981ea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd27806b4d0 0x7fd27806d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2980c7d70 0x7fd298151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.290+0000 7fd29fc00640 1 --2- 172.21.15.67:0/883819965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2980c3e70 0x7fd29814bc00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.291+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 >> 172.21.15.67:0/883819965 conn(0x7fd29806b040 msgr2=0x7fd298153350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.291+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 shutdown_connections 2024-02-26T10:40:58.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.291+0000 7fd29fc00640 1 -- 172.21.15.67:0/883819965 wait complete. 2024-02-26T10:40:58.310 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-02-26T10:40:58.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-02-26T10:40:58.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.540+0000 7fb834a00640 1 Processor -- start 2024-02-26T10:40:58.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.541+0000 7fb834a00640 1 -- start start 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb834a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8301523e0 0x7fb8300c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb834a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83006d600 con 0x7fb8300c7600 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb834a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8300c51b0 con 0x7fb830155940 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb834a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8301584b0 con 0x7fb8301523e0 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb82fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb8300c7600 0x7fb83015f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46522/0 (socket says 172.21.15.67:46522) 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb82fe00640 1 -- 172.21.15.67:0/3176681336 learned_addr learned my addr 172.21.15.67:0/3176681336 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.542+0000 7fb82f400640 1 --2- 172.21.15.67:0/3176681336 >> v2:172.21.15.67:3300/0 conn(0x7fb8301523e0 0x7fb8300c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.543+0000 7fb82f400640 1 -- 172.21.15.67:0/3176681336 >> v1:172.21.15.82:6789/0 conn(0x7fb830155940 legacy=0x7fb830158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:40:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.543+0000 7fb82f400640 1 -- 172.21.15.67:0/3176681336 >> v1:172.21.15.67:6789/0 conn(0x7fb8300c7600 legacy=0x7fb83015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.543+0000 7fb82f400640 1 -- 172.21.15.67:0/3176681336 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300c5490 con 0x7fb8301523e0 2024-02-26T10:40:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.543+0000 7fb82e000640 1 -- 172.21.15.67:0/3176681336 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749875610 0 0) 0x7fb83006d600 con 0x7fb8300c7600 2024-02-26T10:40:58.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.544+0000 7fb82f400640 1 --2- 172.21.15.67:0/3176681336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8301523e0 0x7fb8300c8570 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb8200097a0 tx=0x7fb82002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a458db2e79dadcb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.544+0000 7fb82e000640 1 -- 172.21.15.67:0/3176681336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200315a0 con 0x7fb8301523e0 2024-02-26T10:40:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb82e000640 1 -- 172.21.15.67:0/3176681336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820037070 con 0x7fb8301523e0 2024-02-26T10:40:58.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb82e000640 1 -- 172.21.15.67:0/3176681336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820031960 con 0x7fb8301523e0 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 -- 172.21.15.67:0/3176681336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8301523e0 msgr2=0x7fb8300c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 --2- 172.21.15.67:0/3176681336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8301523e0 0x7fb8300c8570 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb8200097a0 tx=0x7fb82002b840 comp rx=0 tx=0).stop 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 -- 172.21.15.67:0/3176681336 shutdown_connections 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 --2- 172.21.15.67:0/3176681336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8301523e0 0x7fb8300c8570 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 -- 172.21.15.67:0/3176681336 >> 172.21.15.67:0/3176681336 conn(0x7fb830072770 msgr2=0x7fb830072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 -- 172.21.15.67:0/3176681336 shutdown_connections 2024-02-26T10:40:58.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.545+0000 7fb834a00640 1 -- 172.21.15.67:0/3176681336 wait complete. 2024-02-26T10:40:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.547+0000 7fb834a00640 1 Processor -- start 2024-02-26T10:40:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.547+0000 7fb834a00640 1 -- start start 2024-02-26T10:40:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb834a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8300c7600 0x7fb8301ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb834a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8301523e0 0x7fb83014b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb834a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 0x7fb83014b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb834a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830159330 con 0x7fb830155940 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb834a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830158700 con 0x7fb8300c7600 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb834a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb830155d80 con 0x7fb8301523e0 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb82fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 0x7fb83014b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb82fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 0x7fb83014b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40912/0 (socket says 172.21.15.67:40912) 2024-02-26T10:40:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.548+0000 7fb82fe00640 1 -- 172.21.15.67:0/1907004050 learned_addr learned my addr 172.21.15.67:0/1907004050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:58.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82ea00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8301523e0 0x7fb83014b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82f400640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8300c7600 0x7fb8301ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82fe00640 1 -- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8301523e0 msgr2=0x7fb83014b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82fe00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8301523e0 0x7fb83014b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82fe00640 1 -- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8300c7600 msgr2=0x7fb8301ede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82fe00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8300c7600 0x7fb8301ede30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82fe00640 1 -- 172.21.15.67:0/1907004050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300c4810 con 0x7fb830155940 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb82fe00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 0x7fb83014b980 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c700 tx=0x7fb82800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828018070 con 0x7fb830155940 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82800ce10 con 0x7fb830155940 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800a860 con 0x7fb830155940 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.549+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8301ee900 con 0x7fb830155940 2024-02-26T10:40:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.550+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8300c0460 con 0x7fb830155940 2024-02-26T10:40:58.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.551+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830074a30 con 0x7fb830155940 2024-02-26T10:40:58.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.552+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82800aa00 con 0x7fb830155940 2024-02-26T10:40:58.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.553+0000 7fb827400640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb81006b4d0 0x7fb81006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:58.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.553+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8280897f0 con 0x7fb830155940 2024-02-26T10:40:58.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.554+0000 7fb82f400640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb81006b4d0 0x7fb81006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:58.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.555+0000 7fb82f400640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb81006b4d0 0x7fb81006d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb820036000 tx=0x7fb820008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:58.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.558+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb828058660 con 0x7fb830155940 2024-02-26T10:40:58.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:58.756+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb83006d860 con 0x7fb830155940 2024-02-26T10:40:59.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.734+0000 7fb827400640 1 -- 172.21.15.67:0/1907004050 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v56) v1 ==== 140+0+0 (secure 0 0 0) 0x7fb828057f60 con 0x7fb830155940 2024-02-26T10:40:59.735 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_metadata' created 2024-02-26T10:40:59.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.738+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb81006b4d0 msgr2=0x7fb81006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.738+0000 7fb834a00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb81006b4d0 0x7fb81006d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb820036000 tx=0x7fb820008030 comp rx=0 tx=0).stop 2024-02-26T10:40:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.738+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 msgr2=0x7fb83014b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.738+0000 7fb834a00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 0x7fb83014b980 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c700 tx=0x7fb82800cbc0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 shutdown_connections 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb830155940 0x7fb83014b980 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8301523e0 0x7fb83014b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb81006b4d0 0x7fb81006d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 --2- 172.21.15.67:0/1907004050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8300c7600 0x7fb8301ede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 >> 172.21.15.67:0/1907004050 conn(0x7fb830072770 msgr2=0x7fb830072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 shutdown_connections 2024-02-26T10:40:59.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.739+0000 7fb834a00640 1 -- 172.21.15.67:0/1907004050 wait complete. 2024-02-26T10:40:59.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.974+0000 7fdc34600640 1 Processor -- start 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.974+0000 7fdc34600640 1 -- start start 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.975+0000 7fdc34600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc2c14cf30 0x7fdc2c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.976+0000 7fdc34600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c1542d0 con 0x7fdc2c06ea10 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.976+0000 7fdc34600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c154570 con 0x7fdc2c155050 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.976+0000 7fdc34600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c154850 con 0x7fdc2c14cf30 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.976+0000 7fdc31200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdc2c06ea10 0x7fdc2c06ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37386/0 (socket says 172.21.15.67:37386) 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.976+0000 7fdc31200640 1 -- 172.21.15.67:0/679071896 learned_addr learned my addr 172.21.15.67:0/679071896 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:40:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.976+0000 7fdc31c00640 1 --2- 172.21.15.67:0/679071896 >> v2:172.21.15.82:3300/0 conn(0x7fdc2c14cf30 0x7fdc2c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.977+0000 7fdc2be00640 1 -- 172.21.15.67:0/679071896 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91304243 0 0) 0x7fdc2c1542d0 con 0x7fdc2c06ea10 2024-02-26T10:40:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.977+0000 7fdc2be00640 1 -- 172.21.15.67:0/679071896 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc14003520 con 0x7fdc2c06ea10 2024-02-26T10:40:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.977+0000 7fdc31c00640 1 -- 172.21.15.67:0/679071896 >> v1:172.21.15.82:6789/0 conn(0x7fdc2c155050 legacy=0x7fdc2c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.977+0000 7fdc31c00640 1 -- 172.21.15.67:0/679071896 >> v1:172.21.15.67:6789/0 conn(0x7fdc2c06ea10 legacy=0x7fdc2c06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.977+0000 7fdc31c00640 1 -- 172.21.15.67:0/679071896 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c154b30 con 0x7fdc2c14cf30 2024-02-26T10:40:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.978+0000 7fdc31c00640 1 --2- 172.21.15.67:0/679071896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1538e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc200097a0 tx=0x7fdc2002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c564b9fb9a6d619 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.979+0000 7fdc2be00640 1 -- 172.21.15.67:0/679071896 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20035020 con 0x7fdc2c14cf30 2024-02-26T10:40:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.979+0000 7fdc2be00640 1 -- 172.21.15.67:0/679071896 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc20031830 con 0x7fdc2c14cf30 2024-02-26T10:40:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.979+0000 7fdc2be00640 1 -- 172.21.15.67:0/679071896 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20031c00 con 0x7fdc2c14cf30 2024-02-26T10:40:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 -- 172.21.15.67:0/679071896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c14cf30 msgr2=0x7fdc2c1538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 --2- 172.21.15.67:0/679071896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1538e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc200097a0 tx=0x7fdc2002b8d0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 -- 172.21.15.67:0/679071896 shutdown_connections 2024-02-26T10:40:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 --2- 172.21.15.67:0/679071896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1538e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 -- 172.21.15.67:0/679071896 >> 172.21.15.67:0/679071896 conn(0x7fdc2c0c7160 msgr2=0x7fdc2c0a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:40:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 -- 172.21.15.67:0/679071896 shutdown_connections 2024-02-26T10:40:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.980+0000 7fdc34600640 1 -- 172.21.15.67:0/679071896 wait complete. 2024-02-26T10:40:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.982+0000 7fdc34600640 1 Processor -- start 2024-02-26T10:40:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.982+0000 7fdc34600640 1 -- start start 2024-02-26T10:40:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc34600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc2c06ea10 0x7fdc2c1de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc34600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc34600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c155050 0x7fdc2c148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc34600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c06b330 con 0x7fdc2c14cf30 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc34600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c155c10 con 0x7fdc2c155050 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc34600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c06b4b0 con 0x7fdc2c06ea10 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc31200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc31200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54734/0 (socket says 172.21.15.67:54734) 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc31200640 1 -- 172.21.15.67:0/3085058406 learned_addr learned my addr 172.21.15.67:0/3085058406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.984+0000 7fdc31c00640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc2c06ea10 0x7fdc2c1de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc32600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c155050 0x7fdc2c148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc31200640 1 -- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc2c06ea10 msgr2=0x7fdc2c1de6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc31200640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc2c06ea10 0x7fdc2c1de6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc31200640 1 -- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c155050 msgr2=0x7fdc2c148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc31200640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c155050 0x7fdc2c148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc31200640 1 -- 172.21.15.67:0/3085058406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c148d70 con 0x7fdc2c14cf30 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc32600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c155050 0x7fdc2c148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.985+0000 7fdc31200640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1482f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c530 tx=0x7fdc1800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.986+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc2c14cf30 2024-02-26T10:40:59.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.986+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18005ce0 con 0x7fdc2c14cf30 2024-02-26T10:40:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.986+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c1dda00 con 0x7fdc2c14cf30 2024-02-26T10:40:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.986+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180072b0 con 0x7fdc2c14cf30 2024-02-26T10:40:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.986+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c1dec00 con 0x7fdc2c14cf30 2024-02-26T10:40:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.987+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c074a30 con 0x7fdc2c14cf30 2024-02-26T10:40:59.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.989+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc1801a070 con 0x7fdc2c14cf30 2024-02-26T10:40:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.990+0000 7fdc29600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:40:59.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.990+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6080+0+0 (secure 0 0 0) 0x7fdc18089720 con 0x7fdc2c14cf30 2024-02-26T10:40:59.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.991+0000 7fdc31c00640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:40:59.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.992+0000 7fdc31c00640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc20036000 tx=0x7fdc20008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:40:59.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:40:59.995+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc18058580 con 0x7fdc2c14cf30 2024-02-26T10:41:00.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.193+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdc2c0abaf0 con 0x7fdc2c14cf30 2024-02-26T10:41:00.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:41:00.440 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:41:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.749+0000 7fdc29600640 1 -- 172.21.15.67:0/3085058406 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v57) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdc18057e80 con 0x7fdc2c14cf30 2024-02-26T10:41:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_data' created 2024-02-26T10:41:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.753+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc0c06b4d0 msgr2=0x7fdc0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.753+0000 7fdc34600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc20036000 tx=0x7fdc20008030 comp rx=0 tx=0).stop 2024-02-26T10:41:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.753+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 msgr2=0x7fdc2c1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.753+0000 7fdc34600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1482f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c530 tx=0x7fdc1800c9f0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 shutdown_connections 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc20036000 tx=0x7fdc20008030 comp rx=0 tx=0).stop 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2c155050 0x7fdc2c148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2c14cf30 0x7fdc2c1482f0 secure :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c530 tx=0x7fdc1800c9f0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 --2- 172.21.15.67:0/3085058406 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc2c06ea10 0x7fdc2c1de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 >> 172.21.15.67:0/3085058406 conn(0x7fdc2c0c7160 msgr2=0x7fdc2c0a82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 shutdown_connections 2024-02-26T10:41:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.754+0000 7fdc34600640 1 -- 172.21.15.67:0/3085058406 wait complete. 2024-02-26T10:41:00.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-02-26T10:41:00.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.976+0000 7f5dec200640 1 Processor -- start 2024-02-26T10:41:00.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.976+0000 7f5dec200640 1 -- start start 2024-02-26T10:41:00.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.978+0000 7f5dec200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5de40c79c0 0x7f5de4158570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:00.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.978+0000 7f5dec200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5de40c6590 0x7f5de4158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:00.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.978+0000 7f5dec200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de40c70a0 con 0x7f5de40c5e80 2024-02-26T10:41:00.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.978+0000 7f5dec200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de40ca670 con 0x7f5de40c6590 2024-02-26T10:41:00.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.978+0000 7f5dec200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de406c330 con 0x7f5de40c79c0 2024-02-26T10:41:00.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.978+0000 7f5de8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5de40c6590 0x7f5de4158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:00.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5de40c6590 0x7f5de4158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54736/0 (socket says 172.21.15.67:54736) 2024-02-26T10:41:00.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de8e00640 1 -- 172.21.15.67:0/2010093533 learned_addr learned my addr 172.21.15.67:0/2010093533 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:00.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de9800640 1 --2- 172.21.15.67:0/2010093533 >> v2:172.21.15.82:3300/0 conn(0x7f5de40c79c0 0x7f5de4158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:00.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de8e00640 1 -- 172.21.15.67:0/2010093533 >> v2:172.21.15.82:3300/0 conn(0x7f5de40c79c0 msgr2=0x7f5de4158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:00.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de8e00640 1 --2- 172.21.15.67:0/2010093533 >> v2:172.21.15.82:3300/0 conn(0x7f5de40c79c0 0x7f5de4158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de8e00640 1 -- 172.21.15.67:0/2010093533 >> v1:172.21.15.82:6789/0 conn(0x7f5de40c5e80 legacy=0x7f5de4152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:00.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.979+0000 7f5de8e00640 1 -- 172.21.15.67:0/2010093533 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de406c580 con 0x7f5de40c6590 2024-02-26T10:41:00.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.980+0000 7f5de8e00640 1 --2- 172.21.15.67:0/2010093533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c6590 0x7f5de4158ab0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5de40c7720 tx=0x7f5dd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31332ac4e7ade1b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:00.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5de3e00640 1 -- 172.21.15.67:0/2010093533 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd002e070 con 0x7f5de40c6590 2024-02-26T10:41:00.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5de3e00640 1 -- 172.21.15.67:0/2010093533 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0037070 con 0x7f5de40c6590 2024-02-26T10:41:00.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5de3e00640 1 -- 172.21.15.67:0/2010093533 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0031a00 con 0x7f5de40c6590 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5dec200640 1 -- 172.21.15.67:0/2010093533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c6590 msgr2=0x7f5de4158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5dec200640 1 --2- 172.21.15.67:0/2010093533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c6590 0x7f5de4158ab0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5de40c7720 tx=0x7f5dd002b880 comp rx=0 tx=0).stop 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5dec200640 1 -- 172.21.15.67:0/2010093533 shutdown_connections 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5dec200640 1 --2- 172.21.15.67:0/2010093533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c6590 0x7f5de4158ab0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5dec200640 1 --2- 172.21.15.67:0/2010093533 >> v2:172.21.15.82:3300/0 conn(0x7f5de40c79c0 0x7f5de4158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.981+0000 7f5dec200640 1 -- 172.21.15.67:0/2010093533 >> 172.21.15.67:0/2010093533 conn(0x7f5de40ac750 msgr2=0x7f5de40acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.982+0000 7f5dec200640 1 -- 172.21.15.67:0/2010093533 shutdown_connections 2024-02-26T10:41:00.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.982+0000 7f5dec200640 1 -- 172.21.15.67:0/2010093533 wait complete. 2024-02-26T10:41:00.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.983+0000 7f5dec200640 1 Processor -- start 2024-02-26T10:41:00.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.983+0000 7f5dec200640 1 -- start start 2024-02-26T10:41:00.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.984+0000 7f5dec200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 0x7f5de41eb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:00.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.984+0000 7f5dec200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de40c6590 0x7f5de4155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:00.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.984+0000 7f5dec200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5de40c79c0 0x7f5de414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.984+0000 7f5dec200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40c5650 con 0x7f5de40c5e80 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.984+0000 7f5dec200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40c6b70 con 0x7f5de40c6590 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.984+0000 7f5dec200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40c5ae0 con 0x7f5de40c79c0 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 0x7f5de41eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 0x7f5de41eb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54748/0 (socket says 172.21.15.67:54748) 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 -- 172.21.15.67:0/142041326 learned_addr learned my addr 172.21.15.67:0/142041326 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 -- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5de40c79c0 msgr2=0x7f5de414abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5de40c79c0 0x7f5de414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 -- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de40c6590 msgr2=0x7f5de4155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de40c6590 0x7f5de4155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 -- 172.21.15.67:0/142041326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40c1560 con 0x7f5de40c5e80 2024-02-26T10:41:00.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.985+0000 7f5de9800640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 0x7f5de41eb270 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800d800 tx=0x7f5dd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:00.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.986+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800b6c0 con 0x7f5de40c5e80 2024-02-26T10:41:00.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.986+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8019070 con 0x7f5de40c5e80 2024-02-26T10:41:00.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.986+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800ba60 con 0x7f5de40c5e80 2024-02-26T10:41:00.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.986+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de41eb7b0 con 0x7f5de40c5e80 2024-02-26T10:41:00.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.986+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de40c4400 con 0x7f5de40c5e80 2024-02-26T10:41:00.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.987+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de4074a30 con 0x7f5de40c5e80 2024-02-26T10:41:00.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.988+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dd800bc00 con 0x7f5de40c5e80 2024-02-26T10:41:00.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.989+0000 7f5de1600640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5dc406b4d0 0x7f5dc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:00.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.989+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f5dd808aa70 con 0x7f5de40c5e80 2024-02-26T10:41:00.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.990+0000 7f5de8e00640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5dc406b4d0 0x7f5dc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:00.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.991+0000 7f5de8e00640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5dc406b4d0 0x7f5dc406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0036000 tx=0x7f5dd0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:00.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:00.994+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5de41eb7b0 con 0x7f5de40c5e80 2024-02-26T10:41:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.226+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f5de4151a80 con 0x7f5de40c5e80 2024-02-26T10:41:01.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.753+0000 7f5de1600640 1 -- 172.21.15.67:0/142041326 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 10 and data pool 11 v33) v1 ==== 177+0+0 (secure 0 0 0) 0x7f5dd8011340 con 0x7f5de40c5e80 2024-02-26T10:41:01.753 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 10 and data pool 11 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5dc406b4d0 msgr2=0x7f5dc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5dc406b4d0 0x7f5dc406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0036000 tx=0x7f5dd0030000 comp rx=0 tx=0).stop 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 msgr2=0x7f5de41eb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 0x7f5de41eb270 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800d800 tx=0x7f5dd800dcc0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 shutdown_connections 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5de40c79c0 0x7f5de414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5dc406b4d0 0x7f5dc406d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de40c6590 0x7f5de4155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 --2- 172.21.15.67:0/142041326 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5de40c5e80 0x7f5de41eb270 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.757+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 >> 172.21.15.67:0/142041326 conn(0x7f5de40ac750 msgr2=0x7f5de406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.758+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 shutdown_connections 2024-02-26T10:41:01.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.758+0000 7f5dec200640 1 -- 172.21.15.67:0/142041326 wait complete. 2024-02-26T10:41:01.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.976+0000 7f015de00640 1 Processor -- start 2024-02-26T10:41:01.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.976+0000 7f015de00640 1 -- start start 2024-02-26T10:41:01.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.977+0000 7f015de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 0x7f01580c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.977+0000 7f015de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01580bfa30 0x7f0158158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.977+0000 7f015de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01580c15e0 con 0x7f01580c35d0 2024-02-26T10:41:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.977+0000 7f015de00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015806e660 con 0x7f01580bfa30 2024-02-26T10:41:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.977+0000 7f015de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015806e920 con 0x7f0158152420 2024-02-26T10:41:01.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.978+0000 7f0156a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 0x7f01580c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.978+0000 7f0156a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 0x7f01580c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:51170/0 (socket says 172.21.15.67:51170) 2024-02-26T10:41:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.978+0000 7f0156a00640 1 -- 172.21.15.67:0/3129167653 learned_addr learned my addr 172.21.15.67:0/3129167653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.978+0000 7f0156000640 1 --2- 172.21.15.67:0/3129167653 >> v2:172.21.15.82:3300/0 conn(0x7f01580bfa30 0x7f0158158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.979+0000 7f0156a00640 1 -- 172.21.15.67:0/3129167653 >> v2:172.21.15.82:3300/0 conn(0x7f01580bfa30 msgr2=0x7f0158158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.979+0000 7f0156a00640 1 --2- 172.21.15.67:0/3129167653 >> v2:172.21.15.82:3300/0 conn(0x7f01580bfa30 0x7f0158158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.979+0000 7f0156a00640 1 -- 172.21.15.67:0/3129167653 >> v1:172.21.15.82:6789/0 conn(0x7f01580c35d0 legacy=0x7f015806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:01.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.979+0000 7f0156a00640 1 -- 172.21.15.67:0/3129167653 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580c1880 con 0x7f0158152420 2024-02-26T10:41:01.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.980+0000 7f0156a00640 1 --2- 172.21.15.67:0/3129167653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 0x7f01580c3090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0148009e50 tx=0x7f014802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478c6b62898bc1f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f0155600640 1 -- 172.21.15.67:0/3129167653 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014802ca70 con 0x7f0158152420 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f0155600640 1 -- 172.21.15.67:0/3129167653 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014802cc10 con 0x7f0158152420 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f0155600640 1 -- 172.21.15.67:0/3129167653 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480343f0 con 0x7f0158152420 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 -- 172.21.15.67:0/3129167653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 msgr2=0x7f01580c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 --2- 172.21.15.67:0/3129167653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 0x7f01580c3090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0148009e50 tx=0x7f014802bed0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 -- 172.21.15.67:0/3129167653 shutdown_connections 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 --2- 172.21.15.67:0/3129167653 >> v2:172.21.15.82:3300/0 conn(0x7f01580bfa30 0x7f0158158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 --2- 172.21.15.67:0/3129167653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0158152420 0x7f01580c3090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 -- 172.21.15.67:0/3129167653 >> 172.21.15.67:0/3129167653 conn(0x7f01580a84a0 msgr2=0x7f01580a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 -- 172.21.15.67:0/3129167653 shutdown_connections 2024-02-26T10:41:01.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.981+0000 7f015de00640 1 -- 172.21.15.67:0/3129167653 wait complete. 2024-02-26T10:41:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.983+0000 7f015de00640 1 Processor -- start 2024-02-26T10:41:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.983+0000 7f015de00640 1 -- start start 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f015de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01580bfa30 0x7f01581ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f015de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 0x7f0158143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f015de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158152420 0x7f0158144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f015de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158152800 con 0x7f01580c35d0 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f015de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158158400 con 0x7f0158152420 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f015de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0158154e10 con 0x7f01580bfa30 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f0156000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 0x7f0158143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f0156000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 0x7f0158143e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54750/0 (socket says 172.21.15.67:54750) 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f0156000640 1 -- 172.21.15.67:0/1245706019 learned_addr learned my addr 172.21.15.67:0/1245706019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f0156a00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01580bfa30 0x7f01581ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.984+0000 7f0157400640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158152420 0x7f0158144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0156000640 1 -- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01580bfa30 msgr2=0x7f01581ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0156000640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01580bfa30 0x7f01581ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0156000640 1 -- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158152420 msgr2=0x7f0158144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0156000640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158152420 0x7f0158144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0156000640 1 -- 172.21.15.67:0/1245706019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580c4390 con 0x7f01580c35d0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0157400640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158152420 0x7f0158144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f0156000640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 0x7f0158143e10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f014000e840 tx=0x7f014000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140017070 con 0x7f01580c35d0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014000c940 con 0x7f01580c35d0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014000cd10 con 0x7f01580c35d0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01581ee890 con 0x7f01580c35d0 2024-02-26T10:41:01.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.985+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0158144890 con 0x7f01580c35d0 2024-02-26T10:41:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.986+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0158154220 con 0x7f01580c35d0 2024-02-26T10:41:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.987+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f014001a070 con 0x7f01580c35d0 2024-02-26T10:41:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.988+0000 7f014f400640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f013406b4d0 0x7f013406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.988+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f014008a770 con 0x7f01580c35d0 2024-02-26T10:41:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.989+0000 7f0156a00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f013406b4d0 0x7f013406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:01.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.990+0000 7f0156a00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f013406b4d0 0x7f013406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0148035000 tx=0x7f0148030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:01.993+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0158154220 con 0x7f01580c35d0 2024-02-26T10:41:02.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.181+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0158074a30 con 0x7f01580c35d0 2024-02-26T10:41:02.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.185+0000 7f014f400640 1 -- 172.21.15.67:0/1245706019 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+20387 (secure 0 0 0) 0x7f0140059320 con 0x7f01580c35d0 2024-02-26T10:41:02.185 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:02.185 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":58,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:01.228640+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:02.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f013406b4d0 msgr2=0x7f013406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f013406b4d0 0x7f013406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0148035000 tx=0x7f0148030000 comp rx=0 tx=0).stop 2024-02-26T10:41:02.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 msgr2=0x7f0158143e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 0x7f0158143e10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f014000e840 tx=0x7f014000ed00 comp rx=0 tx=0).stop 2024-02-26T10:41:02.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 shutdown_connections 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f013406b4d0 0x7f013406d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0158152420 0x7f0158144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01580c35d0 0x7f0158143e10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 --2- 172.21.15.67:0/1245706019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01580bfa30 0x7f01581ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.187+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 >> 172.21.15.67:0/1245706019 conn(0x7f01580a84a0 msgr2=0x7f0158158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.188+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 shutdown_connections 2024-02-26T10:41:02.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.188+0000 7f015de00640 1 -- 172.21.15.67:0/1245706019 wait complete. 2024-02-26T10:41:02.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:02.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.421+0000 7fea17e00640 1 Processor -- start 2024-02-26T10:41:02.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.421+0000 7fea17e00640 1 -- start start 2024-02-26T10:41:02.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.422+0000 7fea17e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea180a94a0 0x7fea1806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.422+0000 7fea17e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea1806db00 0x7fea180abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.422+0000 7fea17e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1806df70 con 0x7fea1806ac20 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.422+0000 7fea17e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea180a9a80 con 0x7fea180a94a0 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.422+0000 7fea17e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea180a9d60 con 0x7fea1806db00 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea180a94a0 0x7fea1806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea180a94a0 0x7fea1806b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54752/0 (socket says 172.21.15.67:54752) 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 -- 172.21.15.67:0/60685817 learned_addr learned my addr 172.21.15.67:0/60685817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 -- 172.21.15.67:0/60685817 >> v2:172.21.15.82:3300/0 conn(0x7fea1806db00 msgr2=0x7fea180abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 --2- 172.21.15.67:0/60685817 >> v2:172.21.15.82:3300/0 conn(0x7fea1806db00 0x7fea180abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 -- 172.21.15.67:0/60685817 >> v1:172.21.15.82:6789/0 conn(0x7fea1806ac20 legacy=0x7fea18153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:02.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.423+0000 7fea0fe00640 1 -- 172.21.15.67:0/60685817 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180ab750 con 0x7fea180a94a0 2024-02-26T10:41:02.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.424+0000 7fea0fe00640 1 --2- 172.21.15.67:0/60685817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea180a94a0 0x7fea1806b000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fea18157220 tx=0x7fea0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc3214c4079c4d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:02.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.425+0000 7fea16000640 1 -- 172.21.15.67:0/60685817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00035020 con 0x7fea180a94a0 2024-02-26T10:41:02.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.425+0000 7fea16000640 1 -- 172.21.15.67:0/60685817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00031870 con 0x7fea180a94a0 2024-02-26T10:41:02.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.425+0000 7fea16000640 1 -- 172.21.15.67:0/60685817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00031c40 con 0x7fea180a94a0 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.425+0000 7fea17e00640 1 -- 172.21.15.67:0/60685817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea180a94a0 msgr2=0x7fea1806b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.425+0000 7fea17e00640 1 --2- 172.21.15.67:0/60685817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea180a94a0 0x7fea1806b000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fea18157220 tx=0x7fea0002b910 comp rx=0 tx=0).stop 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.426+0000 7fea17e00640 1 -- 172.21.15.67:0/60685817 shutdown_connections 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.426+0000 7fea17e00640 1 --2- 172.21.15.67:0/60685817 >> v2:172.21.15.82:3300/0 conn(0x7fea1806db00 0x7fea180abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.426+0000 7fea17e00640 1 --2- 172.21.15.67:0/60685817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea180a94a0 0x7fea1806b000 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.426+0000 7fea17e00640 1 -- 172.21.15.67:0/60685817 >> 172.21.15.67:0/60685817 conn(0x7fea180c63f0 msgr2=0x7fea180c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.426+0000 7fea17e00640 1 -- 172.21.15.67:0/60685817 shutdown_connections 2024-02-26T10:41:02.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.426+0000 7fea17e00640 1 -- 172.21.15.67:0/60685817 wait complete. 2024-02-26T10:41:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.428+0000 7fea17e00640 1 Processor -- start 2024-02-26T10:41:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.428+0000 7fea17e00640 1 -- start start 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.429+0000 7fea17e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1806ac20 0x7fea181ee190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.429+0000 7fea17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 0x7fea18071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.429+0000 7fea17e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fea180a94a0 0x7fea18146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.429+0000 7fea17e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea18150410 con 0x7fea1806db00 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.429+0000 7fea17e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea180ac490 con 0x7fea1806ac20 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.429+0000 7fea17e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea18150590 con 0x7fea180a94a0 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 0x7fea18071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 0x7fea18071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54756/0 (socket says 172.21.15.67:54756) 2024-02-26T10:41:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 -- 172.21.15.67:0/3221185479 learned_addr learned my addr 172.21.15.67:0/3221185479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea16a00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1806ac20 0x7fea181ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea17400640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fea180a94a0 0x7fea18146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 -- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fea180a94a0 msgr2=0x7fea18146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fea180a94a0 0x7fea18146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 -- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1806ac20 msgr2=0x7fea181ee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1806ac20 0x7fea181ee190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 -- 172.21.15.67:0/3221185479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea181eea60 con 0x7fea1806db00 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea16a00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1806ac20 0x7fea181ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea17400640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fea180a94a0 0x7fea18146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.430+0000 7fea0fe00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 0x7fea18071290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.431+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0003ad60 con 0x7fea1806db00 2024-02-26T10:41:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.431+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea000443f0 con 0x7fea1806db00 2024-02-26T10:41:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.431+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea180721a0 con 0x7fea1806db00 2024-02-26T10:41:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.431+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea000447e0 con 0x7fea1806db00 2024-02-26T10:41:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.431+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea181487e0 con 0x7fea1806db00 2024-02-26T10:41:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.432+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea18074a30 con 0x7fea1806db00 2024-02-26T10:41:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.433+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea0004b060 con 0x7fea1806db00 2024-02-26T10:41:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.434+0000 7fea0ea00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe9f406b500 0x7fe9f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.434+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fea000ac710 con 0x7fea1806db00 2024-02-26T10:41:02.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.435+0000 7fea16a00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe9f406b500 0x7fe9f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.436+0000 7fea16a00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe9f406b500 0x7fe9f406d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea08007fd0 tx=0x7fea08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:02.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.438+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea0003f070 con 0x7fea1806db00 2024-02-26T10:41:02.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.638+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea180c0df0 con 0x7fea1806db00 2024-02-26T10:41:02.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.642+0000 7fea0ea00640 1 -- 172.21.15.67:0/3221185479 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+20387 (secure 0 0 0) 0x7fea0002f320 con 0x7fea1806db00 2024-02-26T10:41:02.642 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:02.642 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":58,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:01.228640+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:02.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe9f406b500 msgr2=0x7fe9f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe9f406b500 0x7fe9f406d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea08007fd0 tx=0x7fea08009040 comp rx=0 tx=0).stop 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 msgr2=0x7fea18071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 0x7fea18071290 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea000046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 shutdown_connections 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fea180a94a0 0x7fea18146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea1806db00 0x7fea18071290 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe9f406b500 0x7fe9f406d9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 --2- 172.21.15.67:0/3221185479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1806ac20 0x7fea181ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 >> 172.21.15.67:0/3221185479 conn(0x7fea180c63f0 msgr2=0x7fea1814f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 shutdown_connections 2024-02-26T10:41:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.645+0000 7fea17e00640 1 -- 172.21.15.67:0/3221185479 wait complete. 2024-02-26T10:41:02.665 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-02-26T10:41:02.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.900+0000 7fb7e7e00640 1 Processor -- start 2024-02-26T10:41:02.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.901+0000 7fb7e7e00640 1 -- start start 2024-02-26T10:41:02.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.902+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 0x7fb7e8153150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.902+0000 7fb7e7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e8070ae0 con 0x7fb7e80c4890 2024-02-26T10:41:02.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.902+0000 7fb7e7e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e8070dc0 con 0x7fb7e81558c0 2024-02-26T10:41:02.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.902+0000 7fb7e7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80c9380 con 0x7fb7e80c2d60 2024-02-26T10:41:02.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.902+0000 7fb7e6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb7e80c4890 0x7fb7e8152a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37400/0 (socket says 172.21.15.67:37400) 2024-02-26T10:41:02.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.902+0000 7fb7e6000640 1 -- 172.21.15.67:0/602599570 learned_addr learned my addr 172.21.15.67:0/602599570 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:02.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e7400640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 0x7fb7e8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948241820 0 0) 0x7fb7e8070ae0 con 0x7fb7e80c4890 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7cc003520 con 0x7fb7e80c4890 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3158319613 0 0) 0x7fb7cc003520 con 0x7fb7e80c4890 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 msgr2=0x7fb7e8153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 0x7fb7e8153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 >> v1:172.21.15.82:6789/0 conn(0x7fb7e81558c0 legacy=0x7fb7e80c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e81537a0 con 0x7fb7e80c4890 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fb7d0001670 con 0x7fb7e80c4890 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 >> v1:172.21.15.67:6789/0 conn(0x7fb7e80c4890 legacy=0x7fb7e8152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7cc0039e0 0x7fb7cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 0x7fb7cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7cc004d40 0x7fb7cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.903+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc00baf0 con 0x7fb7cc0039e0 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc00bca0 con 0x7fb7cc004d40 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc00bea0 con 0x7fb7e80c2d60 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7cc0039e0 0x7fb7cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6000640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 0x7fb7cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e7400640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7cc004d40 0x7fb7cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6a00640 1 -- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 msgr2=0x7fb7cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e80c2d60 0x7fb7cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6a00640 1 -- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7cc004d40 msgr2=0x7fb7cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7cc004d40 0x7fb7cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.904+0000 7fb7e6a00640 1 -- 172.21.15.67:0/602599570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc003520 con 0x7fb7cc0039e0 2024-02-26T10:41:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.905+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7cc0039e0 0x7fb7cc003e00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8001e50 tx=0x7fb7cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cdcd874baa22f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8001a70 con 0x7fb7cc0039e0 2024-02-26T10:41:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d8003b70 con 0x7fb7cc0039e0 2024-02-26T10:41:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e5600640 1 -- 172.21.15.67:0/602599570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80050d0 con 0x7fb7cc0039e0 2024-02-26T10:41:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7e00640 1 -- 172.21.15.67:0/602599570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7cc0039e0 msgr2=0x7fb7cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/602599570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7cc0039e0 0x7fb7cc003e00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8001e50 tx=0x7fb7cc005830 comp rx=0 tx=0).stop 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7400640 1 -- 172.21.15.67:0/602599570 reap_dead start 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7e00640 1 -- 172.21.15.67:0/602599570 shutdown_connections 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7e00640 1 -- 172.21.15.67:0/602599570 >> 172.21.15.67:0/602599570 conn(0x7fb7e80c83e0 msgr2=0x7fb7e80c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7e00640 1 -- 172.21.15.67:0/602599570 shutdown_connections 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.906+0000 7fb7e7e00640 1 -- 172.21.15.67:0/602599570 wait complete. 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.908+0000 7fb7e7e00640 1 Processor -- start 2024-02-26T10:41:02.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.908+0000 7fb7e7e00640 1 -- start start 2024-02-26T10:41:02.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7cc0039e0 0x7fb7e80c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 0x7fb7e81ea700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e81eac40 0x7fb7e814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8155e20 con 0x7fb7e814eac0 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8155ca0 con 0x7fb7e81eac40 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8153150 con 0x7fb7cc0039e0 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 0x7fb7e81ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 0x7fb7e81ea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54778/0 (socket says 172.21.15.67:54778) 2024-02-26T10:41:02.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.909+0000 7fb7e6000640 1 -- 172.21.15.67:0/905313167 learned_addr learned my addr 172.21.15.67:0/905313167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:02.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7cc0039e0 0x7fb7e80c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e7400640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e81eac40 0x7fb7e814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6000640 1 -- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7cc0039e0 msgr2=0x7fb7e80c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6000640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7cc0039e0 0x7fb7e80c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6000640 1 -- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e81eac40 msgr2=0x7fb7e814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6000640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e81eac40 0x7fb7e814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6000640 1 -- 172.21.15.67:0/905313167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e81dc6c0 con 0x7fb7e814eac0 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e7400640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e81eac40 0x7fb7e814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7cc0039e0 0x7fb7e80c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e6000640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 0x7fb7e81ea700 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000e780 tx=0x7fb7d000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000c870 con 0x7fb7e814eac0 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d0018070 con 0x7fb7e814eac0 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000cc70 con 0x7fb7e814eac0 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.910+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e814e7e0 con 0x7fb7e814eac0 2024-02-26T10:41:02.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.911+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e8150b90 con 0x7fb7e814eac0 2024-02-26T10:41:02.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.912+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e8074a30 con 0x7fb7e814eac0 2024-02-26T10:41:02.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.913+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7d001d460 con 0x7fb7e814eac0 2024-02-26T10:41:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.914+0000 7fb7dea00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b4d0 0x7fb7c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.914+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb7d008a470 con 0x7fb7e814eac0 2024-02-26T10:41:02.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.914+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b4d0 0x7fb7c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:02.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.916+0000 7fb7e6a00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b4d0 0x7fb7c406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8034000 tx=0x7fb7d8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:02.918+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7d0058f80 con 0x7fb7e814eac0 2024-02-26T10:41:03.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.163+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7e81eb050 con 0x7fb7e814eac0 2024-02-26T10:41:03.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.796+0000 7fb7dea00640 1 -- 172.21.15.67:0/905313167 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 119+0+0 (secure 0 0 0) 0x7fb7d0058880 con 0x7fb7e814eac0 2024-02-26T10:41:03.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b4d0 msgr2=0x7fb7c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:03.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b4d0 0x7fb7c406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8034000 tx=0x7fb7d8009030 comp rx=0 tx=0).stop 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 msgr2=0x7fb7e81ea700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 0x7fb7e81ea700 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000e780 tx=0x7fb7d000ec40 comp rx=0 tx=0).stop 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 shutdown_connections 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7c406b4d0 0x7fb7c406d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e81eac40 0x7fb7e814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e814eac0 0x7fb7e81ea700 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 --2- 172.21.15.67:0/905313167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7cc0039e0 0x7fb7e80c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 >> 172.21.15.67:0/905313167 conn(0x7fb7e80c83e0 msgr2=0x7fb7e80ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 shutdown_connections 2024-02-26T10:41:03.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:03.800+0000 7fb7e7e00640 1 -- 172.21.15.67:0/905313167 wait complete. 2024-02-26T10:41:03.822 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:41:03.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-02-26T10:41:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.041+0000 7fd51ec00640 1 Processor -- start 2024-02-26T10:41:04.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.041+0000 7fd51ec00640 1 -- start start 2024-02-26T10:41:04.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.043+0000 7fd51ec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd5181523b0 0x7fd5180c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:04.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.043+0000 7fd51ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:04.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.043+0000 7fd51ec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180c3840 con 0x7fd518154cf0 2024-02-26T10:41:04.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.043+0000 7fd51ec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd518152790 con 0x7fd5181523b0 2024-02-26T10:41:04.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.043+0000 7fd51ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51806da30 con 0x7fd518071fd0 2024-02-26T10:41:04.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.044+0000 7fd51e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:04.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.044+0000 7fd51d800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd5181523b0 0x7fd5180c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.044+0000 7fd51ce00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd518154cf0 0x7fd5180c3130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:45892/0 (socket says 172.21.15.67:45892) 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.044+0000 7fd51ce00640 1 -- 172.21.15.67:0/1437821215 learned_addr learned my addr 172.21.15.67:0/1437821215 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.045+0000 7fd51d800640 1 -- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 msgr2=0x7fd518072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.045+0000 7fd51d800640 1 --2- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.045+0000 7fd51d800640 1 -- 172.21.15.67:0/1437821215 >> v1:172.21.15.82:6789/0 conn(0x7fd518154cf0 legacy=0x7fd5180c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.045+0000 7fd51d800640 1 -- 172.21.15.67:0/1437821215 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51806dc80 con 0x7fd5181523b0 2024-02-26T10:41:04.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.045+0000 7fd51e200640 1 --2- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:41:04.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.046+0000 7fd51d800640 1 --2- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd5180c2bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0097a0 tx=0x7fd50c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9deee31d65482c92 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:04.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd517e00640 1 -- 172.21.15.67:0/1437821215 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c039070 con 0x7fd5181523b0 2024-02-26T10:41:04.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd517e00640 1 -- 172.21.15.67:0/1437821215 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c0316f0 con 0x7fd5181523b0 2024-02-26T10:41:04.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd517e00640 1 -- 172.21.15.67:0/1437821215 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c031ac0 con 0x7fd5181523b0 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd51ec00640 1 -- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 msgr2=0x7fd5180c2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd5180c2bf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0097a0 tx=0x7fd50c02b790 comp rx=0 tx=0).stop 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd51ec00640 1 -- 172.21.15.67:0/1437821215 shutdown_connections 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1437821215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd5180c2bf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.047+0000 7fd51ec00640 1 -- 172.21.15.67:0/1437821215 >> 172.21.15.67:0/1437821215 conn(0x7fd5180c6380 msgr2=0x7fd5180c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:04.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.048+0000 7fd51ec00640 1 -- 172.21.15.67:0/1437821215 shutdown_connections 2024-02-26T10:41:04.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.048+0000 7fd51ec00640 1 -- 172.21.15.67:0/1437821215 wait complete. 2024-02-26T10:41:04.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.049+0000 7fd51ec00640 1 Processor -- start 2024-02-26T10:41:04.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.049+0000 7fd51ec00640 1 -- start start 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.050+0000 7fd51ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.050+0000 7fd51ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd51814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.050+0000 7fd51ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 0x7fd51814a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.050+0000 7fd51ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51806e520 con 0x7fd518154cf0 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.050+0000 7fd51ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51806e240 con 0x7fd5181523b0 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.050+0000 7fd51ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51806e800 con 0x7fd518071fd0 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 0x7fd51814a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 0x7fd51814a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54786/0 (socket says 172.21.15.67:54786) 2024-02-26T10:41:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 -- 172.21.15.67:0/1804862583 learned_addr learned my addr 172.21.15.67:0/1804862583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51d800640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51ce00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd51814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 -- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 msgr2=0x7fd518158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 -- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 msgr2=0x7fd51814a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd51814a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.051+0000 7fd51e200640 1 -- 172.21.15.67:0/1804862583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5181eb5b0 con 0x7fd518154cf0 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.052+0000 7fd51e200640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 0x7fd51814a840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd51000efc0 tx=0x7fd51000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.052+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510018030 con 0x7fd518154cf0 2024-02-26T10:41:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.052+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51000cdf0 con 0x7fd518154cf0 2024-02-26T10:41:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.052+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510003bb0 con 0x7fd518154cf0 2024-02-26T10:41:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.052+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5181eb830 con 0x7fd518154cf0 2024-02-26T10:41:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.052+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518158960 con 0x7fd518154cf0 2024-02-26T10:41:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.053+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd518074a30 con 0x7fd518154cf0 2024-02-26T10:41:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.054+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd510005d10 con 0x7fd518154cf0 2024-02-26T10:41:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.055+0000 7fd515600640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd4f806b4d0 0x7fd4f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.056+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd51008aef0 con 0x7fd518154cf0 2024-02-26T10:41:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.056+0000 7fd51d800640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd4f806b4d0 0x7fd4f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:04.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.057+0000 7fd51d800640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd4f806b4d0 0x7fd4f806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd50c036000 tx=0x7fd50c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:04.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.060+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd510022070 con 0x7fd518154cf0 2024-02-26T10:41:04.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.304+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd5180bfc30 con 0x7fd518154cf0 2024-02-26T10:41:04.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.808+0000 7fd515600640 1 -- 172.21.15.67:0/1804862583 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd5181eb5b0 con 0x7fd518154cf0 2024-02-26T10:41:04.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd4f806b4d0 msgr2=0x7fd4f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd4f806b4d0 0x7fd4f806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd50c036000 tx=0x7fd50c008030 comp rx=0 tx=0).stop 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 msgr2=0x7fd51814a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 0x7fd51814a840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd51000efc0 tx=0x7fd51000c490 comp rx=0 tx=0).stop 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 shutdown_connections 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd518154cf0 0x7fd51814a840 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd4f806b4d0 0x7fd4f806d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5181523b0 0x7fd51814a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 --2- 172.21.15.67:0/1804862583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd518071fd0 0x7fd518158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 >> 172.21.15.67:0/1804862583 conn(0x7fd5180c6380 msgr2=0x7fd518155210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 shutdown_connections 2024-02-26T10:41:04.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:04.812+0000 7fd51ec00640 1 -- 172.21.15.67:0/1804862583 wait complete. 2024-02-26T10:41:04.834 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-02-26T10:41:04.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:04.922 INFO:tasks.ceph.mds.d.smithi082.stderr:2024-02-26T10:41:04.924+0000 7fc2b8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:41:05.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.058+0000 7f33dca00640 1 Processor -- start 2024-02-26T10:41:05.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.059+0000 7f33dca00640 1 -- start start 2024-02-26T10:41:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33dca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33d806ac20 0x7f33d806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33dca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33dca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d8155f10 con 0x7f33d80c2310 2024-02-26T10:41:05.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33dca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d8153aa0 con 0x7f33d806ac20 2024-02-26T10:41:05.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33dca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c28d0 con 0x7f33d8153630 2024-02-26T10:41:05.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33d7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33d806ac20 0x7f33d806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33d7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33d806ac20 0x7f33d806b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54794/0 (socket says 172.21.15.67:54794) 2024-02-26T10:41:05.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.060+0000 7f33d7400640 1 -- 172.21.15.67:0/2908961920 learned_addr learned my addr 172.21.15.67:0/2908961920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d7e00640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d7400640 1 -- 172.21.15.67:0/2908961920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 msgr2=0x7f33d81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d7400640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d7400640 1 -- 172.21.15.67:0/2908961920 >> v1:172.21.15.67:6789/0 conn(0x7f33d80c2310 legacy=0x7f33d8152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d7400640 1 -- 172.21.15.67:0/2908961920 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8155150 con 0x7f33d806ac20 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d6000640 1 -- 172.21.15.67:0/2908961920 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675863806 0 0) 0x7f33d8155f10 con 0x7f33d80c2310 2024-02-26T10:41:05.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.061+0000 7f33d7e00640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.062+0000 7f33d7400640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d806b000 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e6862ea953596 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.062+0000 7f33d6000640 1 -- 172.21.15.67:0/2908961920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c80317c0 con 0x7f33d806ac20 2024-02-26T10:41:05.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.062+0000 7f33d6000640 1 -- 172.21.15.67:0/2908961920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8037070 con 0x7f33d806ac20 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.062+0000 7f33d6000640 1 -- 172.21.15.67:0/2908961920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8031b80 con 0x7f33d806ac20 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 -- 172.21.15.67:0/2908961920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 msgr2=0x7f33d806b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d806b000 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c802b860 comp rx=0 tx=0).stop 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 -- 172.21.15.67:0/2908961920 shutdown_connections 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 --2- 172.21.15.67:0/2908961920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d806b000 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 -- 172.21.15.67:0/2908961920 >> 172.21.15.67:0/2908961920 conn(0x7f33d80c63f0 msgr2=0x7f33d80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 -- 172.21.15.67:0/2908961920 shutdown_connections 2024-02-26T10:41:05.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.063+0000 7f33dca00640 1 -- 172.21.15.67:0/2908961920 wait complete. 2024-02-26T10:41:05.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.065+0000 7f33dca00640 1 Processor -- start 2024-02-26T10:41:05.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.065+0000 7f33dca00640 1 -- start start 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33dca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d81df710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33dca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d80c2310 0x7f33d81456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33dca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d8145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33dca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d81559f0 con 0x7f33d806ac20 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33dca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8155710 con 0x7f33d80c2310 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33dca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8148850 con 0x7f33d8153630 2024-02-26T10:41:05.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d81df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d81df710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54810/0 (socket says 172.21.15.67:54810) 2024-02-26T10:41:05.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33d7400640 1 -- 172.21.15.67:0/3324018022 learned_addr learned my addr 172.21.15.67:0/3324018022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.066+0000 7f33d6a00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d80c2310 0x7f33d81456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7e00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d8145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7400640 1 -- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 msgr2=0x7f33d8145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7400640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d8145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7400640 1 -- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d80c2310 msgr2=0x7f33d81456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7400640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d80c2310 0x7f33d81456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7400640 1 -- 172.21.15.67:0/3324018022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d81eb210 con 0x7f33d806ac20 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33d7400640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d81df710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f33c802b8a0 tx=0x7f33c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8032860 con 0x7f33d806ac20 2024-02-26T10:41:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8032a00 con 0x7f33d806ac20 2024-02-26T10:41:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8032df0 con 0x7f33d806ac20 2024-02-26T10:41:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.067+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d8070cd0 con 0x7f33d806ac20 2024-02-26T10:41:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.068+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d8071290 con 0x7f33d806ac20 2024-02-26T10:41:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.069+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d8074a30 con 0x7f33d806ac20 2024-02-26T10:41:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.070+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33c80453f0 con 0x7f33d806ac20 2024-02-26T10:41:05.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.070+0000 7f33cf400640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f33b406b500 0x7f33b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.071+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f33c80ac8d0 con 0x7f33d806ac20 2024-02-26T10:41:05.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.071+0000 7f33d6a00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f33b406b500 0x7f33b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.072+0000 7f33d6a00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f33b406b500 0x7f33b406d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f33d806eb80 tx=0x7f33c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:05.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.075+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33c8040070 con 0x7f33d806ac20 2024-02-26T10:41:05.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.312+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33d80c20f0 con 0x7f33d806ac20 2024-02-26T10:41:05.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.314+0000 7f33cf400640 1 -- 172.21.15.67:0/3324018022 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f33c80aae20 con 0x7f33d806ac20 2024-02-26T10:41:05.313 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:05.313 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:05.314 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f33b406b500 msgr2=0x7f33b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f33b406b500 0x7f33b406d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f33d806eb80 tx=0x7f33c0009040 comp rx=0 tx=0).stop 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 msgr2=0x7f33d81df710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d81df710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f33c802b8a0 tx=0x7f33c80046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 shutdown_connections 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33d8153630 0x7f33d8145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f33b406b500 0x7f33b406d9c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d80c2310 0x7f33d81456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 --2- 172.21.15.67:0/3324018022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33d806ac20 0x7f33d81df710 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.317+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 >> 172.21.15.67:0/3324018022 conn(0x7f33d80c63f0 msgr2=0x7f33d8154140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.318+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 shutdown_connections 2024-02-26T10:41:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.318+0000 7f33dca00640 1 -- 172.21.15.67:0/3324018022 wait complete. 2024-02-26T10:41:05.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:05.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.547+0000 7f60c6a00640 1 Processor -- start 2024-02-26T10:41:05.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.547+0000 7f60c6a00640 1 -- start start 2024-02-26T10:41:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.548+0000 7f60c6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f60c00a94a0 0x7f60c006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c6a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c006df70 con 0x7f60c006db00 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c6a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c00a9a80 con 0x7f60c006ac20 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c6a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c00a9d60 con 0x7f60c00a94a0 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f60c00a94a0 0x7f60c006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f60c00a94a0 0x7f60c006b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54822/0 (socket says 172.21.15.67:54822) 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f60c006db00 0x7f60c00abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37426/0 (socket says 172.21.15.67:37426) 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c4c00640 1 -- 172.21.15.67:0/2379269201 learned_addr learned my addr 172.21.15.67:0/2379269201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c4c00640 1 -- 172.21.15.67:0/2379269201 >> v1:172.21.15.82:6789/0 conn(0x7f60c006ac20 legacy=0x7f60c0153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c4c00640 1 -- 172.21.15.67:0/2379269201 >> v1:172.21.15.67:6789/0 conn(0x7f60c006db00 legacy=0x7f60c00abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.549+0000 7f60c4c00640 1 -- 172.21.15.67:0/2379269201 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00ab750 con 0x7f60c00a94a0 2024-02-26T10:41:05.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.550+0000 7f60c4c00640 1 --2- 172.21.15.67:0/2379269201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c00a94a0 0x7f60c006b000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009870 tx=0x7f60ac02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ff73f94232a34f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:05.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60bfe00640 1 -- 172.21.15.67:0/2379269201 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac034070 con 0x7f60c00a94a0 2024-02-26T10:41:05.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60bfe00640 1 -- 172.21.15.67:0/2379269201 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac030850 con 0x7f60c00a94a0 2024-02-26T10:41:05.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60bfe00640 1 -- 172.21.15.67:0/2379269201 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac030c20 con 0x7f60c00a94a0 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60c6a00640 1 -- 172.21.15.67:0/2379269201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c00a94a0 msgr2=0x7f60c006b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60c6a00640 1 --2- 172.21.15.67:0/2379269201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c00a94a0 0x7f60c006b000 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009870 tx=0x7f60ac02b990 comp rx=0 tx=0).stop 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60c6a00640 1 -- 172.21.15.67:0/2379269201 shutdown_connections 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.551+0000 7f60c6a00640 1 --2- 172.21.15.67:0/2379269201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c00a94a0 0x7f60c006b000 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.552+0000 7f60c6a00640 1 -- 172.21.15.67:0/2379269201 >> 172.21.15.67:0/2379269201 conn(0x7f60c00c63f0 msgr2=0x7f60c00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.552+0000 7f60c6a00640 1 -- 172.21.15.67:0/2379269201 shutdown_connections 2024-02-26T10:41:05.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.552+0000 7f60c6a00640 1 -- 172.21.15.67:0/2379269201 wait complete. 2024-02-26T10:41:05.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.553+0000 7f60c6a00640 1 Processor -- start 2024-02-26T10:41:05.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.553+0000 7f60c6a00640 1 -- start start 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.554+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60c006ac20 0x7f60c014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.554+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 0x7f60c0152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.554+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60c00a94a0 0x7f60c0149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.554+0000 7f60c6a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c0150590 con 0x7f60c006db00 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.554+0000 7f60c6a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c0150410 con 0x7f60c00a94a0 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.554+0000 7f60c6a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c015e520 con 0x7f60c006ac20 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 0x7f60c0152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 0x7f60c0152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54826/0 (socket says 172.21.15.67:54826) 2024-02-26T10:41:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 -- 172.21.15.67:0/4173652317 learned_addr learned my addr 172.21.15.67:0/4173652317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c5600640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60c006ac20 0x7f60c014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 -- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60c006ac20 msgr2=0x7f60c014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c6000640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60c00a94a0 0x7f60c0149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60c006ac20 0x7f60c014b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 -- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60c00a94a0 msgr2=0x7f60c0149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60c00a94a0 0x7f60c0149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c4c00640 1 -- 172.21.15.67:0/4173652317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c014c620 con 0x7f60c006db00 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.555+0000 7f60c5600640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60c006ac20 0x7f60c014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.556+0000 7f60c4c00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 0x7f60c0152390 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009870 tx=0x7f60ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.556+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac034040 con 0x7f60c006db00 2024-02-26T10:41:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.556+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac0433f0 con 0x7f60c006db00 2024-02-26T10:41:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.556+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c014bc90 con 0x7f60c006db00 2024-02-26T10:41:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.556+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0437e0 con 0x7f60c006db00 2024-02-26T10:41:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.556+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c01e8c20 con 0x7f60c006db00 2024-02-26T10:41:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.557+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0074a30 con 0x7f60c006db00 2024-02-26T10:41:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.558+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60ac04a060 con 0x7f60c006db00 2024-02-26T10:41:05.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.559+0000 7f60bd600640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f60a006b4d0 0x7f60a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:05.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.559+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f60ac0ac740 con 0x7f60c006db00 2024-02-26T10:41:05.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.560+0000 7f60c5600640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f60a006b4d0 0x7f60a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:05.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.561+0000 7f60c5600640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f60a006b4d0 0x7f60a006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f60b4003eb0 tx=0x7f60b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.564+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60ac03e070 con 0x7f60c006db00 2024-02-26T10:41:05.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.764+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60c006b9b0 con 0x7f60c006db00 2024-02-26T10:41:05.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.768+0000 7f60bd600640 1 -- 172.21.15.67:0/4173652317 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+20374 (secure 0 0 0) 0x7f60ac07b2f0 con 0x7f60c006db00 2024-02-26T10:41:05.768 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:05.768 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":59,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:02.772898+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f60a006b4d0 msgr2=0x7f60a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f60a006b4d0 0x7f60a006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f60b4003eb0 tx=0x7f60b4009040 comp rx=0 tx=0).stop 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 msgr2=0x7f60c0152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 0x7f60c0152390 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009870 tx=0x7f60ac0046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 shutdown_connections 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f60a006b4d0 0x7f60a006d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60c00a94a0 0x7f60c0149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c006db00 0x7f60c0152390 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 --2- 172.21.15.67:0/4173652317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60c006ac20 0x7f60c014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 >> 172.21.15.67:0/4173652317 conn(0x7f60c00c63f0 msgr2=0x7f60c00c1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 shutdown_connections 2024-02-26T10:41:05.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:05.771+0000 7f60c6a00640 1 -- 172.21.15.67:0/4173652317 wait complete. 2024-02-26T10:41:05.791 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:41:05.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:41:06.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.009+0000 7fbfb5600640 1 Processor -- start 2024-02-26T10:41:06.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.010+0000 7fbfb5600640 1 -- start start 2024-02-26T10:41:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.011+0000 7fbfb5600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb006ac20 0x7fbfb00a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.011+0000 7fbfb5600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbfb006b000 0x7fbfb006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.011+0000 7fbfb5600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb006b460 con 0x7fbfb0153610 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.011+0000 7fbfb5600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb00a9a80 con 0x7fbfb006b000 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.011+0000 7fbfb5600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb00a9d60 con 0x7fbfb006ac20 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbfb006b000 0x7fbfb006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbfb006b000 0x7fbfb006db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54836/0 (socket says 172.21.15.67:54836) 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 -- 172.21.15.67:0/423409682 learned_addr learned my addr 172.21.15.67:0/423409682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfafe00640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb006ac20 0x7fbfb00a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 -- 172.21.15.67:0/423409682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb006ac20 msgr2=0x7fbfb00a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb006ac20 0x7fbfb00a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 -- 172.21.15.67:0/423409682 >> v1:172.21.15.67:6789/0 conn(0x7fbfb0153610 legacy=0x7fbfb00abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfa7e00640 1 -- 172.21.15.67:0/423409682 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00ab750 con 0x7fbfb006b000 2024-02-26T10:41:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.012+0000 7fbfafe00640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb006ac20 0x7fbfb00a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.013+0000 7fbfa7e00640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb006db00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009870 tx=0x7fbf9802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94401a43a7fa859f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfaf400640 1 -- 172.21.15.67:0/423409682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98034070 con 0x7fbfb006b000 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfaf400640 1 -- 172.21.15.67:0/423409682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98030840 con 0x7fbfb006b000 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfaf400640 1 -- 172.21.15.67:0/423409682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98030c10 con 0x7fbfb006b000 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 -- 172.21.15.67:0/423409682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 msgr2=0x7fbfb006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb006db00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009870 tx=0x7fbf9802b980 comp rx=0 tx=0).stop 2024-02-26T10:41:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 -- 172.21.15.67:0/423409682 shutdown_connections 2024-02-26T10:41:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb006db00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 --2- 172.21.15.67:0/423409682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb006ac20 0x7fbfb00a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 -- 172.21.15.67:0/423409682 >> 172.21.15.67:0/423409682 conn(0x7fbfb00c63f0 msgr2=0x7fbfb00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 -- 172.21.15.67:0/423409682 shutdown_connections 2024-02-26T10:41:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.014+0000 7fbfb5600640 1 -- 172.21.15.67:0/423409682 wait complete. 2024-02-26T10:41:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.016+0000 7fbfb5600640 1 Processor -- start 2024-02-26T10:41:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.016+0000 7fbfb5600640 1 -- start start 2024-02-26T10:41:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfb5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb006ac20 0x7fbfb0155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfb5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb01def80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfb5600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb0153610 0x7fbfb01df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfb5600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb0150410 con 0x7fbfb006b000 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfb5600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb0153aa0 con 0x7fbfb006ac20 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfb5600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb0150590 con 0x7fbfb0153610 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfa7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb01def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfa7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb01def80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54846/0 (socket says 172.21.15.67:54846) 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.017+0000 7fbfa7e00640 1 -- 172.21.15.67:0/3395593663 learned_addr learned my addr 172.21.15.67:0/3395593663 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfafe00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb006ac20 0x7fbfb0155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfb4c00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb0153610 0x7fbfb01df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfa7e00640 1 -- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb0153610 msgr2=0x7fbfb01df4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfa7e00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb0153610 0x7fbfb01df4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfa7e00640 1 -- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb006ac20 msgr2=0x7fbfb0155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfa7e00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb006ac20 0x7fbfb0155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfa7e00640 1 -- 172.21.15.67:0/3395593663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00bc2c0 con 0x7fbfb006b000 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfafe00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb006ac20 0x7fbfb0155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.018+0000 7fbfa7e00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb01def80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009870 tx=0x7fbf98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.019+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98048070 con 0x7fbfb006b000 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.019+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98030640 con 0x7fbfb006b000 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.019+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb01dd5a0 con 0x7fbfb006b000 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.019+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98034040 con 0x7fbfb006b000 2024-02-26T10:41:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.019+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb01de780 con 0x7fbfb006b000 2024-02-26T10:41:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.020+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb01de570 con 0x7fbfb006b000 2024-02-26T10:41:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.021+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf9804b070 con 0x7fbfb006b000 2024-02-26T10:41:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.022+0000 7fbfacc00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbf8c06b4d0 0x7fbf8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.022+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fbf980ac6a0 con 0x7fbfb006b000 2024-02-26T10:41:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.022+0000 7fbfafe00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbf8c06b4d0 0x7fbf8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.024+0000 7fbfafe00640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbf8c06b4d0 0x7fbf8c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0006fd0 tx=0x7fbfa0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:06.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.027+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf9807b1b0 con 0x7fbfb006b000 2024-02-26T10:41:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.224+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb0074a30 con 0x7fbf8c06b4d0 2024-02-26T10:41:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.263+0000 7fbfacc00640 1 -- 172.21.15.67:0/3395593663 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490511 (secure 0 0 0) 0x7fbfb0074a30 con 0x7fbf8c06b4d0 2024-02-26T10:41:06.265 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbf8c06b4d0 msgr2=0x7fbf8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbf8c06b4d0 0x7fbf8c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0006fd0 tx=0x7fbfa0008040 comp rx=0 tx=0).stop 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 msgr2=0x7fbfb01def80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb01def80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009870 tx=0x7fbf98004730 comp rx=0 tx=0).stop 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 shutdown_connections 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfb0153610 0x7fbfb01df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb006b000 0x7fbfb01def80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbf8c06b4d0 0x7fbf8c06d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 --2- 172.21.15.67:0/3395593663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb006ac20 0x7fbfb0155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.273+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 >> 172.21.15.67:0/3395593663 conn(0x7fbfb00c63f0 msgr2=0x7fbfb0152920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.274+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 shutdown_connections 2024-02-26T10:41:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.274+0000 7fbfb5600640 1 -- 172.21.15.67:0/3395593663 wait complete. 2024-02-26T10:41:06.297 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-02-26T10:41:04.378919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":335,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231196,"kb_used_data":6708,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743524,"statfs":{"total":773094113280,"available":772857368576,"internally_reserved":0,"allocated":6868992,"data_stored":3627563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.640400"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.796917+0000","last_change":"2024-02-26T10:41:01.788509+0000","last_active":"2024-02-26T10:41:02.796917+0000","last_peered":"2024-02-26T10:41:02.796917+0000","last_clean":"2024-02-26T10:41:02.796917+0000","last_became_active":"2024-02-26T10:41:01.787843+0000","last_became_peered":"2024-02-26T10:41:01.787843+0000","last_unstale":"2024-02-26T10:41:02.796917+0000","last_undegraded":"2024-02-26T10:41:02.796917+0000","last_fullsized":"2024-02-26T10:41:02.796917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:44:11.908944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.772630+0000","last_change":"2024-02-26T10:40:45.640652+0000","last_active":"2024-02-26T10:41:01.772630+0000","last_peered":"2024-02-26T10:41:01.772630+0000","last_clean":"2024-02-26T10:41:01.772630+0000","last_became_active":"2024-02-26T10:40:45.637863+0000","last_became_peered":"2024-02-26T10:40:45.637863+0000","last_unstale":"2024-02-26T10:41:01.772630+0000","last_undegraded":"2024-02-26T10:41:01.772630+0000","last_fullsized":"2024-02-26T10:41:01.772630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:40.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.790925+0000","last_change":"2024-02-26T10:41:01.786722+0000","last_active":"2024-02-26T10:41:01.790925+0000","last_peered":"2024-02-26T10:41:01.790925+0000","last_clean":"2024-02-26T10:41:01.790925+0000","last_became_active":"2024-02-26T10:41:01.786408+0000","last_became_peered":"2024-02-26T10:41:01.786408+0000","last_unstale":"2024-02-26T10:41:01.790925+0000","last_undegraded":"2024-02-26T10:41:01.790925+0000","last_fullsized":"2024-02-26T10:41:01.790925+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:31.446787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798597+0000","last_change":"2024-02-26T10:40:45.635299+0000","last_active":"2024-02-26T10:41:02.798597+0000","last_peered":"2024-02-26T10:41:02.798597+0000","last_clean":"2024-02-26T10:41:02.798597+0000","last_became_active":"2024-02-26T10:40:45.634986+0000","last_became_peered":"2024-02-26T10:40:45.634986+0000","last_unstale":"2024-02-26T10:41:02.798597+0000","last_undegraded":"2024-02-26T10:41:02.798597+0000","last_fullsized":"2024-02-26T10:41:02.798597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:53.114125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240904+0000","last_change":"2024-02-26T10:41:01.802788+0000","last_active":"2024-02-26T10:41:03.240904+0000","last_peered":"2024-02-26T10:41:03.240904+0000","last_clean":"2024-02-26T10:41:03.240904+0000","last_became_active":"2024-02-26T10:41:01.802486+0000","last_became_peered":"2024-02-26T10:41:01.802486+0000","last_unstale":"2024-02-26T10:41:03.240904+0000","last_undegraded":"2024-02-26T10:41:03.240904+0000","last_fullsized":"2024-02-26T10:41:03.240904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:40.570174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240937+0000","last_change":"2024-02-26T10:40:45.652997+0000","last_active":"2024-02-26T10:41:03.240937+0000","last_peered":"2024-02-26T10:41:03.240937+0000","last_clean":"2024-02-26T10:41:03.240937+0000","last_became_active":"2024-02-26T10:40:45.652579+0000","last_became_peered":"2024-02-26T10:40:45.652579+0000","last_unstale":"2024-02-26T10:41:03.240937+0000","last_undegraded":"2024-02-26T10:41:03.240937+0000","last_fullsized":"2024-02-26T10:41:03.240937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:14.395806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793945+0000","last_change":"2024-02-26T10:41:01.783410+0000","last_active":"2024-02-26T10:41:02.793945+0000","last_peered":"2024-02-26T10:41:02.793945+0000","last_clean":"2024-02-26T10:41:02.793945+0000","last_became_active":"2024-02-26T10:41:01.783114+0000","last_became_peered":"2024-02-26T10:41:01.783114+0000","last_unstale":"2024-02-26T10:41:02.793945+0000","last_undegraded":"2024-02-26T10:41:02.793945+0000","last_fullsized":"2024-02-26T10:41:02.793945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:46.356779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113203+0000","last_change":"2024-02-26T10:40:45.625722+0000","last_active":"2024-02-26T10:41:03.113203+0000","last_peered":"2024-02-26T10:41:03.113203+0000","last_clean":"2024-02-26T10:41:03.113203+0000","last_became_active":"2024-02-26T10:40:45.625252+0000","last_became_peered":"2024-02-26T10:40:45.625252+0000","last_unstale":"2024-02-26T10:41:03.113203+0000","last_undegraded":"2024-02-26T10:41:03.113203+0000","last_fullsized":"2024-02-26T10:41:03.113203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112315+0000","last_change":"2024-02-26T10:41:01.792496+0000","last_active":"2024-02-26T10:41:03.112315+0000","last_peered":"2024-02-26T10:41:03.112315+0000","last_clean":"2024-02-26T10:41:03.112315+0000","last_became_active":"2024-02-26T10:41:01.792197+0000","last_became_peered":"2024-02-26T10:41:01.792197+0000","last_unstale":"2024-02-26T10:41:03.112315+0000","last_undegraded":"2024-02-26T10:41:03.112315+0000","last_fullsized":"2024-02-26T10:41:03.112315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:24.959706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.764722+0000","last_change":"2024-02-26T10:40:45.645804+0000","last_active":"2024-02-26T10:41:01.764722+0000","last_peered":"2024-02-26T10:41:01.764722+0000","last_clean":"2024-02-26T10:41:01.764722+0000","last_became_active":"2024-02-26T10:40:45.645530+0000","last_became_peered":"2024-02-26T10:40:45.645530+0000","last_unstale":"2024-02-26T10:41:01.764722+0000","last_undegraded":"2024-02-26T10:41:01.764722+0000","last_fullsized":"2024-02-26T10:41:01.764722+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:50.576149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.791932+0000","last_change":"2024-02-26T10:41:01.789486+0000","last_active":"2024-02-26T10:41:02.791932+0000","last_peered":"2024-02-26T10:41:02.791932+0000","last_clean":"2024-02-26T10:41:02.791932+0000","last_became_active":"2024-02-26T10:41:01.788181+0000","last_became_peered":"2024-02-26T10:41:01.788181+0000","last_unstale":"2024-02-26T10:41:02.791932+0000","last_undegraded":"2024-02-26T10:41:02.791932+0000","last_fullsized":"2024-02-26T10:41:02.791932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:49.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780774+0000","last_change":"2024-02-26T10:40:45.626504+0000","last_active":"2024-02-26T10:41:01.780774+0000","last_peered":"2024-02-26T10:41:01.780774+0000","last_clean":"2024-02-26T10:41:01.780774+0000","last_became_active":"2024-02-26T10:40:45.626190+0000","last_became_peered":"2024-02-26T10:40:45.626190+0000","last_unstale":"2024-02-26T10:41:01.780774+0000","last_undegraded":"2024-02-26T10:41:01.780774+0000","last_fullsized":"2024-02-26T10:41:01.780774+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.796455+0000","last_change":"2024-02-26T10:41:01.782509+0000","last_active":"2024-02-26T10:41:01.796455+0000","last_peered":"2024-02-26T10:41:01.796455+0000","last_clean":"2024-02-26T10:41:01.796455+0000","last_became_active":"2024-02-26T10:41:01.782223+0000","last_became_peered":"2024-02-26T10:41:01.782223+0000","last_unstale":"2024-02-26T10:41:01.796455+0000","last_undegraded":"2024-02-26T10:41:01.796455+0000","last_fullsized":"2024-02-26T10:41:01.796455+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:43.016082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365458+0000","last_change":"2024-02-26T10:40:45.647799+0000","last_active":"2024-02-26T10:41:03.365458+0000","last_peered":"2024-02-26T10:41:03.365458+0000","last_clean":"2024-02-26T10:41:03.365458+0000","last_became_active":"2024-02-26T10:40:45.647134+0000","last_became_peered":"2024-02-26T10:40:45.647134+0000","last_unstale":"2024-02-26T10:41:03.365458+0000","last_undegraded":"2024-02-26T10:41:03.365458+0000","last_fullsized":"2024-02-26T10:41:03.365458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:26.570068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792810+0000","last_change":"2024-02-26T10:41:01.786720+0000","last_active":"2024-02-26T10:41:02.792810+0000","last_peered":"2024-02-26T10:41:02.792810+0000","last_clean":"2024-02-26T10:41:02.792810+0000","last_became_active":"2024-02-26T10:41:01.784696+0000","last_became_peered":"2024-02-26T10:41:01.784696+0000","last_unstale":"2024-02-26T10:41:02.792810+0000","last_undegraded":"2024-02-26T10:41:02.792810+0000","last_fullsized":"2024-02-26T10:41:02.792810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:38:16.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793343+0000","last_change":"2024-02-26T10:40:45.638686+0000","last_active":"2024-02-26T10:41:02.793343+0000","last_peered":"2024-02-26T10:41:02.793343+0000","last_clean":"2024-02-26T10:41:02.793343+0000","last_became_active":"2024-02-26T10:40:45.638372+0000","last_became_peered":"2024-02-26T10:40:45.638372+0000","last_unstale":"2024-02-26T10:41:02.793343+0000","last_undegraded":"2024-02-26T10:41:02.793343+0000","last_fullsized":"2024-02-26T10:41:02.793343+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:09:51.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240236+0000","last_change":"2024-02-26T10:41:01.788651+0000","last_active":"2024-02-26T10:41:03.240236+0000","last_peered":"2024-02-26T10:41:03.240236+0000","last_clean":"2024-02-26T10:41:03.240236+0000","last_became_active":"2024-02-26T10:41:01.788044+0000","last_became_peered":"2024-02-26T10:41:01.788044+0000","last_unstale":"2024-02-26T10:41:03.240236+0000","last_undegraded":"2024-02-26T10:41:03.240236+0000","last_fullsized":"2024-02-26T10:41:03.240236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:10.745518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367135+0000","last_change":"2024-02-26T10:40:45.632997+0000","last_active":"2024-02-26T10:41:03.367135+0000","last_peered":"2024-02-26T10:41:03.367135+0000","last_clean":"2024-02-26T10:41:03.367135+0000","last_became_active":"2024-02-26T10:40:45.632665+0000","last_became_peered":"2024-02-26T10:40:45.632665+0000","last_unstale":"2024-02-26T10:41:03.367135+0000","last_undegraded":"2024-02-26T10:41:03.367135+0000","last_fullsized":"2024-02-26T10:41:03.367135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:52.054393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.786162+0000","last_change":"2024-02-26T10:41:01.777684+0000","last_active":"2024-02-26T10:41:01.786162+0000","last_peered":"2024-02-26T10:41:01.786162+0000","last_clean":"2024-02-26T10:41:01.786162+0000","last_became_active":"2024-02-26T10:41:01.777075+0000","last_became_peered":"2024-02-26T10:41:01.777075+0000","last_unstale":"2024-02-26T10:41:01.786162+0000","last_undegraded":"2024-02-26T10:41:01.786162+0000","last_fullsized":"2024-02-26T10:41:01.786162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:49.441345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.767240+0000","last_change":"2024-02-26T10:40:45.636607+0000","last_active":"2024-02-26T10:41:01.767240+0000","last_peered":"2024-02-26T10:41:01.767240+0000","last_clean":"2024-02-26T10:41:01.767240+0000","last_became_active":"2024-02-26T10:40:45.636328+0000","last_became_peered":"2024-02-26T10:40:45.636328+0000","last_unstale":"2024-02-26T10:41:01.767240+0000","last_undegraded":"2024-02-26T10:41:01.767240+0000","last_fullsized":"2024-02-26T10:41:01.767240+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:26.570068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799366+0000","last_change":"2024-02-26T10:41:01.791725+0000","last_active":"2024-02-26T10:41:01.799366+0000","last_peered":"2024-02-26T10:41:01.799366+0000","last_clean":"2024-02-26T10:41:01.799366+0000","last_became_active":"2024-02-26T10:41:01.780515+0000","last_became_peered":"2024-02-26T10:41:01.780515+0000","last_unstale":"2024-02-26T10:41:01.799366+0000","last_undegraded":"2024-02-26T10:41:01.799366+0000","last_fullsized":"2024-02-26T10:41:01.799366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:39.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.765243+0000","last_change":"2024-02-26T10:40:45.628536+0000","last_active":"2024-02-26T10:41:01.765243+0000","last_peered":"2024-02-26T10:41:01.765243+0000","last_clean":"2024-02-26T10:41:01.765243+0000","last_became_active":"2024-02-26T10:40:45.628287+0000","last_became_peered":"2024-02-26T10:40:45.628287+0000","last_unstale":"2024-02-26T10:41:01.765243+0000","last_undegraded":"2024-02-26T10:41:01.765243+0000","last_fullsized":"2024-02-26T10:41:01.765243+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:35.142103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367248+0000","last_change":"2024-02-26T10:41:01.782542+0000","last_active":"2024-02-26T10:41:03.367248+0000","last_peered":"2024-02-26T10:41:03.367248+0000","last_clean":"2024-02-26T10:41:03.367248+0000","last_became_active":"2024-02-26T10:41:01.782129+0000","last_became_peered":"2024-02-26T10:41:01.782129+0000","last_unstale":"2024-02-26T10:41:03.367248+0000","last_undegraded":"2024-02-26T10:41:03.367248+0000","last_fullsized":"2024-02-26T10:41:03.367248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:26:04.968703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113390+0000","last_change":"2024-02-26T10:40:45.644900+0000","last_active":"2024-02-26T10:41:03.113390+0000","last_peered":"2024-02-26T10:41:03.113390+0000","last_clean":"2024-02-26T10:41:03.113390+0000","last_became_active":"2024-02-26T10:40:45.644626+0000","last_became_peered":"2024-02-26T10:40:45.644626+0000","last_unstale":"2024-02-26T10:41:03.113390+0000","last_undegraded":"2024-02-26T10:41:03.113390+0000","last_fullsized":"2024-02-26T10:41:03.113390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.803350+0000","last_change":"2024-02-26T10:41:01.801985+0000","last_active":"2024-02-26T10:41:01.803350+0000","last_peered":"2024-02-26T10:41:01.803350+0000","last_clean":"2024-02-26T10:41:01.803350+0000","last_became_active":"2024-02-26T10:41:01.801187+0000","last_became_peered":"2024-02-26T10:41:01.801187+0000","last_unstale":"2024-02-26T10:41:01.803350+0000","last_undegraded":"2024-02-26T10:41:01.803350+0000","last_fullsized":"2024-02-26T10:41:01.803350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:10:07.551025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793188+0000","last_change":"2024-02-26T10:40:45.641831+0000","last_active":"2024-02-26T10:41:02.793188+0000","last_peered":"2024-02-26T10:41:02.793188+0000","last_clean":"2024-02-26T10:41:02.793188+0000","last_became_active":"2024-02-26T10:40:45.641331+0000","last_became_peered":"2024-02-26T10:40:45.641331+0000","last_unstale":"2024-02-26T10:41:02.793188+0000","last_undegraded":"2024-02-26T10:41:02.793188+0000","last_fullsized":"2024-02-26T10:41:02.793188+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:18:27.192800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365357+0000","last_change":"2024-02-26T10:41:01.783279+0000","last_active":"2024-02-26T10:41:03.365357+0000","last_peered":"2024-02-26T10:41:03.365357+0000","last_clean":"2024-02-26T10:41:03.365357+0000","last_became_active":"2024-02-26T10:41:01.782894+0000","last_became_peered":"2024-02-26T10:41:01.782894+0000","last_unstale":"2024-02-26T10:41:03.365357+0000","last_undegraded":"2024-02-26T10:41:03.365357+0000","last_fullsized":"2024-02-26T10:41:03.365357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:32.246950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365727+0000","last_change":"2024-02-26T10:40:45.645455+0000","last_active":"2024-02-26T10:41:03.365727+0000","last_peered":"2024-02-26T10:41:03.365727+0000","last_clean":"2024-02-26T10:41:03.365727+0000","last_became_active":"2024-02-26T10:40:45.645142+0000","last_became_peered":"2024-02-26T10:40:45.645142+0000","last_unstale":"2024-02-26T10:41:03.365727+0000","last_undegraded":"2024-02-26T10:41:03.365727+0000","last_fullsized":"2024-02-26T10:41:03.365727+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:03.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.802670+0000","last_change":"2024-02-26T10:41:01.798725+0000","last_active":"2024-02-26T10:41:01.802670+0000","last_peered":"2024-02-26T10:41:01.802670+0000","last_clean":"2024-02-26T10:41:01.802670+0000","last_became_active":"2024-02-26T10:41:01.797853+0000","last_became_peered":"2024-02-26T10:41:01.797853+0000","last_unstale":"2024-02-26T10:41:01.802670+0000","last_undegraded":"2024-02-26T10:41:01.802670+0000","last_fullsized":"2024-02-26T10:41:01.802670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:12:00.902313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365532+0000","last_change":"2024-02-26T10:40:45.648027+0000","last_active":"2024-02-26T10:41:03.365532+0000","last_peered":"2024-02-26T10:41:03.365532+0000","last_clean":"2024-02-26T10:41:03.365532+0000","last_became_active":"2024-02-26T10:40:45.647523+0000","last_became_peered":"2024-02-26T10:40:45.647523+0000","last_unstale":"2024-02-26T10:41:03.365532+0000","last_undegraded":"2024-02-26T10:41:03.365532+0000","last_fullsized":"2024-02-26T10:41:03.365532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:33.097875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.785315+0000","last_change":"2024-02-26T10:41:01.774898+0000","last_active":"2024-02-26T10:41:01.785315+0000","last_peered":"2024-02-26T10:41:01.785315+0000","last_clean":"2024-02-26T10:41:01.785315+0000","last_became_active":"2024-02-26T10:41:01.774608+0000","last_became_peered":"2024-02-26T10:41:01.774608+0000","last_unstale":"2024-02-26T10:41:01.785315+0000","last_undegraded":"2024-02-26T10:41:01.785315+0000","last_fullsized":"2024-02-26T10:41:01.785315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:27.207753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.773425+0000","last_change":"2024-02-26T10:40:45.639856+0000","last_active":"2024-02-26T10:41:01.773425+0000","last_peered":"2024-02-26T10:41:01.773425+0000","last_clean":"2024-02-26T10:41:01.773425+0000","last_became_active":"2024-02-26T10:40:45.639346+0000","last_became_peered":"2024-02-26T10:40:45.639346+0000","last_unstale":"2024-02-26T10:41:01.773425+0000","last_undegraded":"2024-02-26T10:41:01.773425+0000","last_fullsized":"2024-02-26T10:41:01.773425+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:59.659271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792233+0000","last_change":"2024-02-26T10:41:01.787816+0000","last_active":"2024-02-26T10:41:02.792233+0000","last_peered":"2024-02-26T10:41:02.792233+0000","last_clean":"2024-02-26T10:41:02.792233+0000","last_became_active":"2024-02-26T10:41:01.787431+0000","last_became_peered":"2024-02-26T10:41:01.787431+0000","last_unstale":"2024-02-26T10:41:02.792233+0000","last_undegraded":"2024-02-26T10:41:02.792233+0000","last_fullsized":"2024-02-26T10:41:02.792233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:26:04.968703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.799110+0000","last_change":"2024-02-26T10:40:45.639209+0000","last_active":"2024-02-26T10:41:02.799110+0000","last_peered":"2024-02-26T10:41:02.799110+0000","last_clean":"2024-02-26T10:41:02.799110+0000","last_became_active":"2024-02-26T10:40:45.638129+0000","last_became_peered":"2024-02-26T10:40:45.638129+0000","last_unstale":"2024-02-26T10:41:02.799110+0000","last_undegraded":"2024-02-26T10:41:02.799110+0000","last_fullsized":"2024-02-26T10:41:02.799110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:08.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.800176+0000","last_change":"2024-02-26T10:41:01.797205+0000","last_active":"2024-02-26T10:41:01.800176+0000","last_peered":"2024-02-26T10:41:01.800176+0000","last_clean":"2024-02-26T10:41:01.800176+0000","last_became_active":"2024-02-26T10:41:01.796937+0000","last_became_peered":"2024-02-26T10:41:01.796937+0000","last_unstale":"2024-02-26T10:41:01.800176+0000","last_undegraded":"2024-02-26T10:41:01.800176+0000","last_fullsized":"2024-02-26T10:41:01.800176+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:51.649294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241170+0000","last_change":"2024-02-26T10:40:45.649827+0000","last_active":"2024-02-26T10:41:03.241170+0000","last_peered":"2024-02-26T10:41:03.241170+0000","last_clean":"2024-02-26T10:41:03.241170+0000","last_became_active":"2024-02-26T10:40:45.649273+0000","last_became_peered":"2024-02-26T10:40:45.649273+0000","last_unstale":"2024-02-26T10:41:03.241170+0000","last_undegraded":"2024-02-26T10:41:03.241170+0000","last_fullsized":"2024-02-26T10:41:03.241170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:33.671100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799345+0000","last_change":"2024-02-26T10:41:01.791632+0000","last_active":"2024-02-26T10:41:01.799345+0000","last_peered":"2024-02-26T10:41:01.799345+0000","last_clean":"2024-02-26T10:41:01.799345+0000","last_became_active":"2024-02-26T10:41:01.780138+0000","last_became_peered":"2024-02-26T10:41:01.780138+0000","last_unstale":"2024-02-26T10:41:01.799345+0000","last_undegraded":"2024-02-26T10:41:01.799345+0000","last_fullsized":"2024-02-26T10:41:01.799345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:23.135572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.770672+0000","last_change":"2024-02-26T10:40:45.626981+0000","last_active":"2024-02-26T10:41:01.770672+0000","last_peered":"2024-02-26T10:41:01.770672+0000","last_clean":"2024-02-26T10:41:01.770672+0000","last_became_active":"2024-02-26T10:40:45.626693+0000","last_became_peered":"2024-02-26T10:40:45.626693+0000","last_unstale":"2024-02-26T10:41:01.770672+0000","last_undegraded":"2024-02-26T10:41:01.770672+0000","last_fullsized":"2024-02-26T10:41:01.770672+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:37.225724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.789422+0000","last_change":"2024-02-26T10:41:01.784430+0000","last_active":"2024-02-26T10:41:01.789422+0000","last_peered":"2024-02-26T10:41:01.789422+0000","last_clean":"2024-02-26T10:41:01.789422+0000","last_became_active":"2024-02-26T10:41:01.784160+0000","last_became_peered":"2024-02-26T10:41:01.784160+0000","last_unstale":"2024-02-26T10:41:01.789422+0000","last_undegraded":"2024-02-26T10:41:01.789422+0000","last_fullsized":"2024-02-26T10:41:01.789422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:52:47.004046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795288+0000","last_change":"2024-02-26T10:40:45.634499+0000","last_active":"2024-02-26T10:41:02.795288+0000","last_peered":"2024-02-26T10:41:02.795288+0000","last_clean":"2024-02-26T10:41:02.795288+0000","last_became_active":"2024-02-26T10:40:45.633868+0000","last_became_peered":"2024-02-26T10:40:45.633868+0000","last_unstale":"2024-02-26T10:41:02.795288+0000","last_undegraded":"2024-02-26T10:41:02.795288+0000","last_fullsized":"2024-02-26T10:41:02.795288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:18:54.827479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239261+0000","last_change":"2024-02-26T10:41:01.796543+0000","last_active":"2024-02-26T10:41:03.239261+0000","last_peered":"2024-02-26T10:41:03.239261+0000","last_clean":"2024-02-26T10:41:03.239261+0000","last_became_active":"2024-02-26T10:41:01.796229+0000","last_became_peered":"2024-02-26T10:41:01.796229+0000","last_unstale":"2024-02-26T10:41:03.239261+0000","last_undegraded":"2024-02-26T10:41:03.239261+0000","last_fullsized":"2024-02-26T10:41:03.239261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:22.919759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240001+0000","last_change":"2024-02-26T10:40:45.650672+0000","last_active":"2024-02-26T10:41:03.240001+0000","last_peered":"2024-02-26T10:41:03.240001+0000","last_clean":"2024-02-26T10:41:03.240001+0000","last_became_active":"2024-02-26T10:40:45.649874+0000","last_became_peered":"2024-02-26T10:40:45.649874+0000","last_unstale":"2024-02-26T10:41:03.240001+0000","last_undegraded":"2024-02-26T10:41:03.240001+0000","last_fullsized":"2024-02-26T10:41:03.240001+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239041+0000","last_change":"2024-02-26T10:41:01.795687+0000","last_active":"2024-02-26T10:41:03.239041+0000","last_peered":"2024-02-26T10:41:03.239041+0000","last_clean":"2024-02-26T10:41:03.239041+0000","last_became_active":"2024-02-26T10:41:01.792930+0000","last_became_peered":"2024-02-26T10:41:01.792930+0000","last_unstale":"2024-02-26T10:41:03.239041+0000","last_undegraded":"2024-02-26T10:41:03.239041+0000","last_fullsized":"2024-02-26T10:41:03.239041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:58.363793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793899+0000","last_change":"2024-02-26T10:40:45.636991+0000","last_active":"2024-02-26T10:41:02.793899+0000","last_peered":"2024-02-26T10:41:02.793899+0000","last_clean":"2024-02-26T10:41:02.793899+0000","last_became_active":"2024-02-26T10:40:45.634121+0000","last_became_peered":"2024-02-26T10:40:45.634121+0000","last_unstale":"2024-02-26T10:41:02.793899+0000","last_undegraded":"2024-02-26T10:41:02.793899+0000","last_fullsized":"2024-02-26T10:41:02.793899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:23:35.604849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366669+0000","last_change":"2024-02-26T10:41:01.790805+0000","last_active":"2024-02-26T10:41:03.366669+0000","last_peered":"2024-02-26T10:41:03.366669+0000","last_clean":"2024-02-26T10:41:03.366669+0000","last_became_active":"2024-02-26T10:41:01.790540+0000","last_became_peered":"2024-02-26T10:41:01.790540+0000","last_unstale":"2024-02-26T10:41:03.366669+0000","last_undegraded":"2024-02-26T10:41:03.366669+0000","last_fullsized":"2024-02-26T10:41:03.366669+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:50.908155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239446+0000","last_change":"2024-02-26T10:40:45.642494+0000","last_active":"2024-02-26T10:41:03.239446+0000","last_peered":"2024-02-26T10:41:03.239446+0000","last_clean":"2024-02-26T10:41:03.239446+0000","last_became_active":"2024-02-26T10:40:45.641655+0000","last_became_peered":"2024-02-26T10:40:45.641655+0000","last_unstale":"2024-02-26T10:41:03.239446+0000","last_undegraded":"2024-02-26T10:41:03.239446+0000","last_fullsized":"2024-02-26T10:41:03.239446+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:15.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366026+0000","last_change":"2024-02-26T10:41:01.781103+0000","last_active":"2024-02-26T10:41:03.366026+0000","last_peered":"2024-02-26T10:41:03.366026+0000","last_clean":"2024-02-26T10:41:03.366026+0000","last_became_active":"2024-02-26T10:41:01.780775+0000","last_became_peered":"2024-02-26T10:41:01.780775+0000","last_unstale":"2024-02-26T10:41:03.366026+0000","last_undegraded":"2024-02-26T10:41:03.366026+0000","last_fullsized":"2024-02-26T10:41:03.366026+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:48.562416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792389+0000","last_change":"2024-02-26T10:40:45.646927+0000","last_active":"2024-02-26T10:41:02.792389+0000","last_peered":"2024-02-26T10:41:02.792389+0000","last_clean":"2024-02-26T10:41:02.792389+0000","last_became_active":"2024-02-26T10:40:45.646598+0000","last_became_peered":"2024-02-26T10:40:45.646598+0000","last_unstale":"2024-02-26T10:41:02.792389+0000","last_undegraded":"2024-02-26T10:41:02.792389+0000","last_fullsized":"2024-02-26T10:41:02.792389+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:02.887116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.797350+0000","last_change":"2024-02-26T10:41:01.795359+0000","last_active":"2024-02-26T10:41:02.797350+0000","last_peered":"2024-02-26T10:41:02.797350+0000","last_clean":"2024-02-26T10:41:02.797350+0000","last_became_active":"2024-02-26T10:41:01.794707+0000","last_became_peered":"2024-02-26T10:41:01.794707+0000","last_unstale":"2024-02-26T10:41:02.797350+0000","last_undegraded":"2024-02-26T10:41:02.797350+0000","last_fullsized":"2024-02-26T10:41:02.797350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:26:56.076176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.772821+0000","last_change":"2024-02-26T10:40:45.637507+0000","last_active":"2024-02-26T10:41:01.772821+0000","last_peered":"2024-02-26T10:41:01.772821+0000","last_clean":"2024-02-26T10:41:01.772821+0000","last_became_active":"2024-02-26T10:40:45.631723+0000","last_became_peered":"2024-02-26T10:40:45.631723+0000","last_unstale":"2024-02-26T10:41:01.772821+0000","last_undegraded":"2024-02-26T10:41:01.772821+0000","last_fullsized":"2024-02-26T10:41:01.772821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:07.605390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366854+0000","last_change":"2024-02-26T10:41:01.778438+0000","last_active":"2024-02-26T10:41:03.366854+0000","last_peered":"2024-02-26T10:41:03.366854+0000","last_clean":"2024-02-26T10:41:03.366854+0000","last_became_active":"2024-02-26T10:41:01.778150+0000","last_became_peered":"2024-02-26T10:41:01.778150+0000","last_unstale":"2024-02-26T10:41:03.366854+0000","last_undegraded":"2024-02-26T10:41:03.366854+0000","last_fullsized":"2024-02-26T10:41:03.366854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:30.561314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114055+0000","last_change":"2024-02-26T10:40:45.633049+0000","last_active":"2024-02-26T10:41:03.114055+0000","last_peered":"2024-02-26T10:41:03.114055+0000","last_clean":"2024-02-26T10:41:03.114055+0000","last_became_active":"2024-02-26T10:40:45.632003+0000","last_became_peered":"2024-02-26T10:40:45.632003+0000","last_unstale":"2024-02-26T10:41:03.114055+0000","last_undegraded":"2024-02-26T10:41:03.114055+0000","last_fullsized":"2024-02-26T10:41:03.114055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366992+0000","last_change":"2024-02-26T10:41:01.791451+0000","last_active":"2024-02-26T10:41:03.366992+0000","last_peered":"2024-02-26T10:41:03.366992+0000","last_clean":"2024-02-26T10:41:03.366992+0000","last_became_active":"2024-02-26T10:41:01.791147+0000","last_became_peered":"2024-02-26T10:41:01.791147+0000","last_unstale":"2024-02-26T10:41:03.366992+0000","last_undegraded":"2024-02-26T10:41:03.366992+0000","last_fullsized":"2024-02-26T10:41:03.366992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:49.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781015+0000","last_change":"2024-02-26T10:40:45.639683+0000","last_active":"2024-02-26T10:41:01.781015+0000","last_peered":"2024-02-26T10:41:01.781015+0000","last_clean":"2024-02-26T10:41:01.781015+0000","last_became_active":"2024-02-26T10:40:45.639388+0000","last_became_peered":"2024-02-26T10:40:45.639388+0000","last_unstale":"2024-02-26T10:41:01.781015+0000","last_undegraded":"2024-02-26T10:41:01.781015+0000","last_fullsized":"2024-02-26T10:41:01.781015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113853+0000","last_change":"2024-02-26T10:41:01.774167+0000","last_active":"2024-02-26T10:41:03.113853+0000","last_peered":"2024-02-26T10:41:03.113853+0000","last_clean":"2024-02-26T10:41:03.113853+0000","last_became_active":"2024-02-26T10:41:01.773876+0000","last_became_peered":"2024-02-26T10:41:01.773876+0000","last_unstale":"2024-02-26T10:41:03.113853+0000","last_undegraded":"2024-02-26T10:41:03.113853+0000","last_fullsized":"2024-02-26T10:41:03.113853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:25.572381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.770960+0000","last_change":"2024-02-26T10:40:45.632119+0000","last_active":"2024-02-26T10:41:01.770960+0000","last_peered":"2024-02-26T10:41:01.770960+0000","last_clean":"2024-02-26T10:41:01.770960+0000","last_became_active":"2024-02-26T10:40:45.631803+0000","last_became_peered":"2024-02-26T10:40:45.631803+0000","last_unstale":"2024-02-26T10:41:01.770960+0000","last_undegraded":"2024-02-26T10:41:01.770960+0000","last_fullsized":"2024-02-26T10:41:01.770960+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:42.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112544+0000","last_change":"2024-02-26T10:41:01.780670+0000","last_active":"2024-02-26T10:41:03.112544+0000","last_peered":"2024-02-26T10:41:03.112544+0000","last_clean":"2024-02-26T10:41:03.112544+0000","last_became_active":"2024-02-26T10:41:01.780364+0000","last_became_peered":"2024-02-26T10:41:01.780364+0000","last_unstale":"2024-02-26T10:41:03.112544+0000","last_undegraded":"2024-02-26T10:41:03.112544+0000","last_fullsized":"2024-02-26T10:41:03.112544+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:17:09.279632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795972+0000","last_change":"2024-02-26T10:40:45.636100+0000","last_active":"2024-02-26T10:41:02.795972+0000","last_peered":"2024-02-26T10:41:02.795972+0000","last_clean":"2024-02-26T10:41:02.795972+0000","last_became_active":"2024-02-26T10:40:45.635761+0000","last_became_peered":"2024-02-26T10:40:45.635761+0000","last_unstale":"2024-02-26T10:41:02.795972+0000","last_undegraded":"2024-02-26T10:41:02.795972+0000","last_fullsized":"2024-02-26T10:41:02.795972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:42.198286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793536+0000","last_change":"2024-02-26T10:41:01.787501+0000","last_active":"2024-02-26T10:41:02.793536+0000","last_peered":"2024-02-26T10:41:02.793536+0000","last_clean":"2024-02-26T10:41:02.793536+0000","last_became_active":"2024-02-26T10:41:01.787201+0000","last_became_peered":"2024-02-26T10:41:01.787201+0000","last_unstale":"2024-02-26T10:41:02.793536+0000","last_undegraded":"2024-02-26T10:41:02.793536+0000","last_fullsized":"2024-02-26T10:41:02.793536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:28.417034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239298+0000","last_change":"2024-02-26T10:40:45.649382+0000","last_active":"2024-02-26T10:41:03.239298+0000","last_peered":"2024-02-26T10:41:03.239298+0000","last_clean":"2024-02-26T10:41:03.239298+0000","last_became_active":"2024-02-26T10:40:45.649076+0000","last_became_peered":"2024-02-26T10:40:45.649076+0000","last_unstale":"2024-02-26T10:41:03.239298+0000","last_undegraded":"2024-02-26T10:41:03.239298+0000","last_fullsized":"2024-02-26T10:41:03.239298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795988+0000","last_change":"2024-02-26T10:41:01.791971+0000","last_active":"2024-02-26T10:41:02.795988+0000","last_peered":"2024-02-26T10:41:02.795988+0000","last_clean":"2024-02-26T10:41:02.795988+0000","last_became_active":"2024-02-26T10:41:01.790980+0000","last_became_peered":"2024-02-26T10:41:01.790980+0000","last_unstale":"2024-02-26T10:41:02.795988+0000","last_undegraded":"2024-02-26T10:41:02.795988+0000","last_fullsized":"2024-02-26T10:41:02.795988+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:28:26.576969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763732+0000","last_change":"2024-02-26T10:40:45.625890+0000","last_active":"2024-02-26T10:41:01.763732+0000","last_peered":"2024-02-26T10:41:01.763732+0000","last_clean":"2024-02-26T10:41:01.763732+0000","last_became_active":"2024-02-26T10:40:45.625334+0000","last_became_peered":"2024-02-26T10:40:45.625334+0000","last_unstale":"2024-02-26T10:41:01.763732+0000","last_undegraded":"2024-02-26T10:41:01.763732+0000","last_fullsized":"2024-02-26T10:41:01.763732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:38:29.332505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112585+0000","last_change":"2024-02-26T10:41:01.778110+0000","last_active":"2024-02-26T10:41:03.112585+0000","last_peered":"2024-02-26T10:41:03.112585+0000","last_clean":"2024-02-26T10:41:03.112585+0000","last_became_active":"2024-02-26T10:41:01.777811+0000","last_became_peered":"2024-02-26T10:41:01.777811+0000","last_unstale":"2024-02-26T10:41:03.112585+0000","last_undegraded":"2024-02-26T10:41:03.112585+0000","last_fullsized":"2024-02-26T10:41:03.112585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:10.992986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.773748+0000","last_change":"2024-02-26T10:40:45.626985+0000","last_active":"2024-02-26T10:41:01.773748+0000","last_peered":"2024-02-26T10:41:01.773748+0000","last_clean":"2024-02-26T10:41:01.773748+0000","last_became_active":"2024-02-26T10:40:45.626657+0000","last_became_peered":"2024-02-26T10:40:45.626657+0000","last_unstale":"2024-02-26T10:41:01.773748+0000","last_undegraded":"2024-02-26T10:41:01.773748+0000","last_fullsized":"2024-02-26T10:41:01.773748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:15.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793019+0000","last_change":"2024-02-26T10:41:01.795830+0000","last_active":"2024-02-26T10:41:02.793019+0000","last_peered":"2024-02-26T10:41:02.793019+0000","last_clean":"2024-02-26T10:41:02.793019+0000","last_became_active":"2024-02-26T10:41:01.795529+0000","last_became_peered":"2024-02-26T10:41:01.795529+0000","last_unstale":"2024-02-26T10:41:02.793019+0000","last_undegraded":"2024-02-26T10:41:02.793019+0000","last_fullsized":"2024-02-26T10:41:02.793019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:23.235157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366718+0000","last_change":"2024-02-26T10:41:00.794647+0000","last_active":"2024-02-26T10:41:03.366718+0000","last_peered":"2024-02-26T10:41:03.366718+0000","last_clean":"2024-02-26T10:41:03.366718+0000","last_became_active":"2024-02-26T10:41:00.793948+0000","last_became_peered":"2024-02-26T10:41:00.793948+0000","last_unstale":"2024-02-26T10:41:03.366718+0000","last_undegraded":"2024-02-26T10:41:03.366718+0000","last_fullsized":"2024-02-26T10:41:03.366718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:15.501040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.782937+0000","last_change":"2024-02-26T10:40:45.643239+0000","last_active":"2024-02-26T10:41:01.782937+0000","last_peered":"2024-02-26T10:41:01.782937+0000","last_clean":"2024-02-26T10:41:01.782937+0000","last_became_active":"2024-02-26T10:40:45.642924+0000","last_became_peered":"2024-02-26T10:40:45.642924+0000","last_unstale":"2024-02-26T10:41:01.782937+0000","last_undegraded":"2024-02-26T10:41:01.782937+0000","last_fullsized":"2024-02-26T10:41:01.782937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":24,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.783336+0000","last_change":"2024-02-26T10:40:44.610146+0000","last_active":"2024-02-26T10:41:01.783336+0000","last_peered":"2024-02-26T10:41:01.783336+0000","last_clean":"2024-02-26T10:41:01.783336+0000","last_became_active":"2024-02-26T10:40:44.609886+0000","last_became_peered":"2024-02-26T10:40:44.609886+0000","last_unstale":"2024-02-26T10:41:01.783336+0000","last_undegraded":"2024-02-26T10:41:01.783336+0000","last_fullsized":"2024-02-26T10:41:01.783336+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:55.475057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240693+0000","last_change":"2024-02-26T10:41:01.788873+0000","last_active":"2024-02-26T10:41:03.240693+0000","last_peered":"2024-02-26T10:41:03.240693+0000","last_clean":"2024-02-26T10:41:03.240693+0000","last_became_active":"2024-02-26T10:41:01.788368+0000","last_became_peered":"2024-02-26T10:41:01.788368+0000","last_unstale":"2024-02-26T10:41:03.240693+0000","last_undegraded":"2024-02-26T10:41:03.240693+0000","last_fullsized":"2024-02-26T10:41:03.240693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:17:24.101378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793325+0000","last_change":"2024-02-26T10:41:00.793345+0000","last_active":"2024-02-26T10:41:02.793325+0000","last_peered":"2024-02-26T10:41:02.793325+0000","last_clean":"2024-02-26T10:41:02.793325+0000","last_became_active":"2024-02-26T10:41:00.792909+0000","last_became_peered":"2024-02-26T10:41:00.792909+0000","last_unstale":"2024-02-26T10:41:02.793325+0000","last_undegraded":"2024-02-26T10:41:02.793325+0000","last_fullsized":"2024-02-26T10:41:02.793325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:21.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.769335+0000","last_change":"2024-02-26T10:40:45.645160+0000","last_active":"2024-02-26T10:41:01.769335+0000","last_peered":"2024-02-26T10:41:01.769335+0000","last_clean":"2024-02-26T10:41:01.769335+0000","last_became_active":"2024-02-26T10:40:45.644902+0000","last_became_peered":"2024-02-26T10:40:45.644902+0000","last_unstale":"2024-02-26T10:41:01.769335+0000","last_undegraded":"2024-02-26T10:41:01.769335+0000","last_fullsized":"2024-02-26T10:41:01.769335+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:53:58.631698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798774+0000","last_change":"2024-02-26T10:40:44.617860+0000","last_active":"2024-02-26T10:41:02.798774+0000","last_peered":"2024-02-26T10:41:02.798774+0000","last_clean":"2024-02-26T10:41:02.798774+0000","last_became_active":"2024-02-26T10:40:44.617592+0000","last_became_peered":"2024-02-26T10:40:44.617592+0000","last_unstale":"2024-02-26T10:41:02.798774+0000","last_undegraded":"2024-02-26T10:41:02.798774+0000","last_fullsized":"2024-02-26T10:41:02.798774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:16.970692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241028+0000","last_change":"2024-02-26T10:41:01.798892+0000","last_active":"2024-02-26T10:41:03.241028+0000","last_peered":"2024-02-26T10:41:03.241028+0000","last_clean":"2024-02-26T10:41:03.241028+0000","last_became_active":"2024-02-26T10:41:01.798614+0000","last_became_peered":"2024-02-26T10:41:01.798614+0000","last_unstale":"2024-02-26T10:41:03.241028+0000","last_undegraded":"2024-02-26T10:41:03.241028+0000","last_fullsized":"2024-02-26T10:41:03.241028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:13.201328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241213+0000","last_change":"2024-02-26T10:41:00.795196+0000","last_active":"2024-02-26T10:41:03.241213+0000","last_peered":"2024-02-26T10:41:03.241213+0000","last_clean":"2024-02-26T10:41:03.241213+0000","last_became_active":"2024-02-26T10:41:00.794870+0000","last_became_peered":"2024-02-26T10:41:00.794870+0000","last_unstale":"2024-02-26T10:41:03.241213+0000","last_undegraded":"2024-02-26T10:41:03.241213+0000","last_fullsized":"2024-02-26T10:41:03.241213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:16.987475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780415+0000","last_change":"2024-02-26T10:40:45.621813+0000","last_active":"2024-02-26T10:41:01.780415+0000","last_peered":"2024-02-26T10:41:01.780415+0000","last_clean":"2024-02-26T10:41:01.780415+0000","last_became_active":"2024-02-26T10:40:45.621532+0000","last_became_peered":"2024-02-26T10:40:45.621532+0000","last_unstale":"2024-02-26T10:41:01.780415+0000","last_undegraded":"2024-02-26T10:41:01.780415+0000","last_fullsized":"2024-02-26T10:41:01.780415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.784527+0000","last_change":"2024-02-26T10:40:44.594679+0000","last_active":"2024-02-26T10:41:01.784527+0000","last_peered":"2024-02-26T10:41:01.784527+0000","last_clean":"2024-02-26T10:41:01.784527+0000","last_became_active":"2024-02-26T10:40:44.593817+0000","last_became_peered":"2024-02-26T10:40:44.593817+0000","last_unstale":"2024-02-26T10:41:01.784527+0000","last_undegraded":"2024-02-26T10:41:01.784527+0000","last_fullsized":"2024-02-26T10:41:01.784527+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:47.713723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799302+0000","last_change":"2024-02-26T10:41:01.786081+0000","last_active":"2024-02-26T10:41:01.799302+0000","last_peered":"2024-02-26T10:41:01.799302+0000","last_clean":"2024-02-26T10:41:01.799302+0000","last_became_active":"2024-02-26T10:41:01.779677+0000","last_became_peered":"2024-02-26T10:41:01.779677+0000","last_unstale":"2024-02-26T10:41:01.799302+0000","last_undegraded":"2024-02-26T10:41:01.799302+0000","last_fullsized":"2024-02-26T10:41:01.799302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:59.497223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.800337+0000","last_change":"2024-02-26T10:41:00.779600+0000","last_active":"2024-02-26T10:41:01.800337+0000","last_peered":"2024-02-26T10:41:01.800337+0000","last_clean":"2024-02-26T10:41:01.800337+0000","last_became_active":"2024-02-26T10:41:00.778132+0000","last_became_peered":"2024-02-26T10:41:00.778132+0000","last_unstale":"2024-02-26T10:41:01.800337+0000","last_undegraded":"2024-02-26T10:41:01.800337+0000","last_fullsized":"2024-02-26T10:41:01.800337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:16.987475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781144+0000","last_change":"2024-02-26T10:40:45.630558+0000","last_active":"2024-02-26T10:41:01.781144+0000","last_peered":"2024-02-26T10:41:01.781144+0000","last_clean":"2024-02-26T10:41:01.781144+0000","last_became_active":"2024-02-26T10:40:45.630319+0000","last_became_peered":"2024-02-26T10:40:45.630319+0000","last_unstale":"2024-02-26T10:41:01.781144+0000","last_undegraded":"2024-02-26T10:41:01.781144+0000","last_fullsized":"2024-02-26T10:41:01.781144+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":24,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113500+0000","last_change":"2024-02-26T10:40:44.616740+0000","last_active":"2024-02-26T10:41:03.113500+0000","last_peered":"2024-02-26T10:41:03.113500+0000","last_clean":"2024-02-26T10:41:03.113500+0000","last_became_active":"2024-02-26T10:40:44.616416+0000","last_became_peered":"2024-02-26T10:40:44.616416+0000","last_unstale":"2024-02-26T10:41:03.113500+0000","last_undegraded":"2024-02-26T10:41:03.113500+0000","last_fullsized":"2024-02-26T10:41:03.113500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:58.112139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.791976+0000","last_change":"2024-02-26T10:41:01.779451+0000","last_active":"2024-02-26T10:41:01.791976+0000","last_peered":"2024-02-26T10:41:01.791976+0000","last_clean":"2024-02-26T10:41:01.791976+0000","last_became_active":"2024-02-26T10:41:01.779125+0000","last_became_peered":"2024-02-26T10:41:01.779125+0000","last_unstale":"2024-02-26T10:41:01.791976+0000","last_undegraded":"2024-02-26T10:41:01.791976+0000","last_fullsized":"2024-02-26T10:41:01.791976+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:09:55.280668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.762926+0000","last_change":"2024-02-26T10:41:00.771434+0000","last_active":"2024-02-26T10:41:01.762926+0000","last_peered":"2024-02-26T10:41:01.762926+0000","last_clean":"2024-02-26T10:41:01.762926+0000","last_became_active":"2024-02-26T10:41:00.770953+0000","last_became_peered":"2024-02-26T10:41:00.770953+0000","last_unstale":"2024-02-26T10:41:01.762926+0000","last_undegraded":"2024-02-26T10:41:01.762926+0000","last_fullsized":"2024-02-26T10:41:01.762926+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:14.265905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794747+0000","last_change":"2024-02-26T10:40:45.631974+0000","last_active":"2024-02-26T10:41:02.794747+0000","last_peered":"2024-02-26T10:41:02.794747+0000","last_clean":"2024-02-26T10:41:02.794747+0000","last_became_active":"2024-02-26T10:40:45.631691+0000","last_became_peered":"2024-02-26T10:40:45.631691+0000","last_unstale":"2024-02-26T10:41:02.794747+0000","last_undegraded":"2024-02-26T10:41:02.794747+0000","last_fullsized":"2024-02-26T10:41:02.794747+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:57.553070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366385+0000","last_change":"2024-02-26T10:40:44.614734+0000","last_active":"2024-02-26T10:41:03.366385+0000","last_peered":"2024-02-26T10:41:03.366385+0000","last_clean":"2024-02-26T10:41:03.366385+0000","last_became_active":"2024-02-26T10:40:44.612533+0000","last_became_peered":"2024-02-26T10:40:44.612533+0000","last_unstale":"2024-02-26T10:41:03.366385+0000","last_undegraded":"2024-02-26T10:41:03.366385+0000","last_fullsized":"2024-02-26T10:41:03.366385+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:00:39.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.802963+0000","last_change":"2024-02-26T10:41:01.799273+0000","last_active":"2024-02-26T10:41:01.802963+0000","last_peered":"2024-02-26T10:41:01.802963+0000","last_clean":"2024-02-26T10:41:01.802963+0000","last_became_active":"2024-02-26T10:41:01.798562+0000","last_became_peered":"2024-02-26T10:41:01.798562+0000","last_unstale":"2024-02-26T10:41:01.802963+0000","last_undegraded":"2024-02-26T10:41:01.802963+0000","last_fullsized":"2024-02-26T10:41:01.802963+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:23:51.770356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763637+0000","last_change":"2024-02-26T10:41:00.767901+0000","last_active":"2024-02-26T10:41:01.763637+0000","last_peered":"2024-02-26T10:41:01.763637+0000","last_clean":"2024-02-26T10:41:01.763637+0000","last_became_active":"2024-02-26T10:41:00.767605+0000","last_became_peered":"2024-02-26T10:41:00.767605+0000","last_unstale":"2024-02-26T10:41:01.763637+0000","last_undegraded":"2024-02-26T10:41:01.763637+0000","last_fullsized":"2024-02-26T10:41:01.763637+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:06.779347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795426+0000","last_change":"2024-02-26T10:40:45.645152+0000","last_active":"2024-02-26T10:41:02.795426+0000","last_peered":"2024-02-26T10:41:02.795426+0000","last_clean":"2024-02-26T10:41:02.795426+0000","last_became_active":"2024-02-26T10:40:45.644904+0000","last_became_peered":"2024-02-26T10:40:45.644904+0000","last_unstale":"2024-02-26T10:41:02.795426+0000","last_undegraded":"2024-02-26T10:41:02.795426+0000","last_fullsized":"2024-02-26T10:41:02.795426+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:43.820459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781142+0000","last_change":"2024-02-26T10:40:44.608899+0000","last_active":"2024-02-26T10:41:01.781142+0000","last_peered":"2024-02-26T10:41:01.781142+0000","last_clean":"2024-02-26T10:41:01.781142+0000","last_became_active":"2024-02-26T10:40:44.608621+0000","last_became_peered":"2024-02-26T10:40:44.608621+0000","last_unstale":"2024-02-26T10:41:01.781142+0000","last_undegraded":"2024-02-26T10:41:01.781142+0000","last_fullsized":"2024-02-26T10:41:01.781142+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:51.049847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112730+0000","last_change":"2024-02-26T10:41:01.786178+0000","last_active":"2024-02-26T10:41:03.112730+0000","last_peered":"2024-02-26T10:41:03.112730+0000","last_clean":"2024-02-26T10:41:03.112730+0000","last_became_active":"2024-02-26T10:41:01.785908+0000","last_became_peered":"2024-02-26T10:41:01.785908+0000","last_unstale":"2024-02-26T10:41:03.112730+0000","last_undegraded":"2024-02-26T10:41:03.112730+0000","last_fullsized":"2024-02-26T10:41:03.112730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:58.363793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795285+0000","last_change":"2024-02-26T10:41:00.788397+0000","last_active":"2024-02-26T10:41:02.795285+0000","last_peered":"2024-02-26T10:41:02.795285+0000","last_clean":"2024-02-26T10:41:02.795285+0000","last_became_active":"2024-02-26T10:41:00.787968+0000","last_became_peered":"2024-02-26T10:41:00.787968+0000","last_unstale":"2024-02-26T10:41:02.795285+0000","last_undegraded":"2024-02-26T10:41:02.795285+0000","last_fullsized":"2024-02-26T10:41:02.795285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:12.185042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792608+0000","last_change":"2024-02-26T10:40:45.643761+0000","last_active":"2024-02-26T10:41:02.792608+0000","last_peered":"2024-02-26T10:41:02.792608+0000","last_clean":"2024-02-26T10:41:02.792608+0000","last_became_active":"2024-02-26T10:40:45.643356+0000","last_became_peered":"2024-02-26T10:40:45.643356+0000","last_unstale":"2024-02-26T10:41:02.792608+0000","last_undegraded":"2024-02-26T10:41:02.792608+0000","last_fullsized":"2024-02-26T10:41:02.792608+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:30.965210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113044+0000","last_change":"2024-02-26T10:40:44.621252+0000","last_active":"2024-02-26T10:41:03.113044+0000","last_peered":"2024-02-26T10:41:03.113044+0000","last_clean":"2024-02-26T10:41:03.113044+0000","last_became_active":"2024-02-26T10:40:44.620990+0000","last_became_peered":"2024-02-26T10:40:44.620990+0000","last_unstale":"2024-02-26T10:41:03.113044+0000","last_undegraded":"2024-02-26T10:41:03.113044+0000","last_fullsized":"2024-02-26T10:41:03.113044+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:32.271293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366236+0000","last_change":"2024-02-26T10:41:01.792324+0000","last_active":"2024-02-26T10:41:03.366236+0000","last_peered":"2024-02-26T10:41:03.366236+0000","last_clean":"2024-02-26T10:41:03.366236+0000","last_became_active":"2024-02-26T10:41:01.792045+0000","last_became_peered":"2024-02-26T10:41:01.792045+0000","last_unstale":"2024-02-26T10:41:03.366236+0000","last_undegraded":"2024-02-26T10:41:03.366236+0000","last_fullsized":"2024-02-26T10:41:03.366236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:59.265674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792996+0000","last_change":"2024-02-26T10:41:00.790542+0000","last_active":"2024-02-26T10:41:02.792996+0000","last_peered":"2024-02-26T10:41:02.792996+0000","last_clean":"2024-02-26T10:41:02.792996+0000","last_became_active":"2024-02-26T10:41:00.789842+0000","last_became_peered":"2024-02-26T10:41:00.789842+0000","last_unstale":"2024-02-26T10:41:02.792996+0000","last_undegraded":"2024-02-26T10:41:02.792996+0000","last_fullsized":"2024-02-26T10:41:02.792996+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:48.425059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794173+0000","last_change":"2024-02-26T10:40:45.647365+0000","last_active":"2024-02-26T10:41:02.794173+0000","last_peered":"2024-02-26T10:41:02.794173+0000","last_clean":"2024-02-26T10:41:02.794173+0000","last_became_active":"2024-02-26T10:40:45.638659+0000","last_became_peered":"2024-02-26T10:40:45.638659+0000","last_unstale":"2024-02-26T10:41:02.794173+0000","last_undegraded":"2024-02-26T10:41:02.794173+0000","last_fullsized":"2024-02-26T10:41:02.794173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:35:53.706078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365749+0000","last_change":"2024-02-26T10:40:44.615854+0000","last_active":"2024-02-26T10:41:03.365749+0000","last_peered":"2024-02-26T10:41:03.365749+0000","last_clean":"2024-02-26T10:41:03.365749+0000","last_became_active":"2024-02-26T10:40:44.615511+0000","last_became_peered":"2024-02-26T10:40:44.615511+0000","last_unstale":"2024-02-26T10:41:03.365749+0000","last_undegraded":"2024-02-26T10:41:03.365749+0000","last_fullsized":"2024-02-26T10:41:03.365749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:04:43.874952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793203+0000","last_change":"2024-02-26T10:41:01.783755+0000","last_active":"2024-02-26T10:41:02.793203+0000","last_peered":"2024-02-26T10:41:02.793203+0000","last_clean":"2024-02-26T10:41:02.793203+0000","last_became_active":"2024-02-26T10:41:01.783486+0000","last_became_peered":"2024-02-26T10:41:01.783486+0000","last_unstale":"2024-02-26T10:41:02.793203+0000","last_undegraded":"2024-02-26T10:41:02.793203+0000","last_fullsized":"2024-02-26T10:41:02.793203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:01:09.228302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798279+0000","last_change":"2024-02-26T10:41:00.788941+0000","last_active":"2024-02-26T10:41:02.798279+0000","last_peered":"2024-02-26T10:41:02.798279+0000","last_clean":"2024-02-26T10:41:02.798279+0000","last_became_active":"2024-02-26T10:41:00.788657+0000","last_became_peered":"2024-02-26T10:41:00.788657+0000","last_unstale":"2024-02-26T10:41:02.798279+0000","last_undegraded":"2024-02-26T10:41:02.798279+0000","last_fullsized":"2024-02-26T10:41:02.798279+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:39.553888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367486+0000","last_change":"2024-02-26T10:40:45.645033+0000","last_active":"2024-02-26T10:41:03.367486+0000","last_peered":"2024-02-26T10:41:03.367486+0000","last_clean":"2024-02-26T10:41:03.367486+0000","last_became_active":"2024-02-26T10:40:45.644738+0000","last_became_peered":"2024-02-26T10:40:45.644738+0000","last_unstale":"2024-02-26T10:41:03.367486+0000","last_undegraded":"2024-02-26T10:41:03.367486+0000","last_fullsized":"2024-02-26T10:41:03.367486+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:53:58.631698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241843+0000","last_change":"2024-02-26T10:40:44.619564+0000","last_active":"2024-02-26T10:41:03.241843+0000","last_peered":"2024-02-26T10:41:03.241843+0000","last_clean":"2024-02-26T10:41:03.241843+0000","last_became_active":"2024-02-26T10:40:44.619054+0000","last_became_peered":"2024-02-26T10:40:44.619054+0000","last_unstale":"2024-02-26T10:41:03.241843+0000","last_undegraded":"2024-02-26T10:41:03.241843+0000","last_fullsized":"2024-02-26T10:41:03.241843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:55.475057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799223+0000","last_change":"2024-02-26T10:41:01.788092+0000","last_active":"2024-02-26T10:41:01.799223+0000","last_peered":"2024-02-26T10:41:01.799223+0000","last_clean":"2024-02-26T10:41:01.799223+0000","last_became_active":"2024-02-26T10:41:01.786793+0000","last_became_peered":"2024-02-26T10:41:01.786793+0000","last_unstale":"2024-02-26T10:41:01.799223+0000","last_undegraded":"2024-02-26T10:41:01.799223+0000","last_fullsized":"2024-02-26T10:41:01.799223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:46:52.532082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"58'6","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.803437+0000","last_change":"2024-02-26T10:41:00.770606+0000","last_active":"2024-02-26T10:41:01.803437+0000","last_peered":"2024-02-26T10:41:01.803437+0000","last_clean":"2024-02-26T10:41:01.803437+0000","last_became_active":"2024-02-26T10:41:00.770049+0000","last_became_peered":"2024-02-26T10:41:00.770049+0000","last_unstale":"2024-02-26T10:41:01.803437+0000","last_undegraded":"2024-02-26T10:41:01.803437+0000","last_fullsized":"2024-02-26T10:41:01.803437+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:21.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241520+0000","last_change":"2024-02-26T10:40:45.651473+0000","last_active":"2024-02-26T10:41:03.241520+0000","last_peered":"2024-02-26T10:41:03.241520+0000","last_clean":"2024-02-26T10:41:03.241520+0000","last_became_active":"2024-02-26T10:40:45.651136+0000","last_became_peered":"2024-02-26T10:40:45.651136+0000","last_unstale":"2024-02-26T10:41:03.241520+0000","last_undegraded":"2024-02-26T10:41:03.241520+0000","last_fullsized":"2024-02-26T10:41:03.241520+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798922+0000","last_change":"2024-02-26T10:40:44.613640+0000","last_active":"2024-02-26T10:41:02.798922+0000","last_peered":"2024-02-26T10:41:02.798922+0000","last_clean":"2024-02-26T10:41:02.798922+0000","last_became_active":"2024-02-26T10:40:44.613419+0000","last_became_peered":"2024-02-26T10:40:44.613419+0000","last_unstale":"2024-02-26T10:41:02.798922+0000","last_undegraded":"2024-02-26T10:41:02.798922+0000","last_fullsized":"2024-02-26T10:41:02.798922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:57.206752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113873+0000","last_change":"2024-02-26T10:41:01.796297+0000","last_active":"2024-02-26T10:41:03.113873+0000","last_peered":"2024-02-26T10:41:03.113873+0000","last_clean":"2024-02-26T10:41:03.113873+0000","last_became_active":"2024-02-26T10:41:01.795214+0000","last_became_peered":"2024-02-26T10:41:01.795214+0000","last_unstale":"2024-02-26T10:41:03.113873+0000","last_undegraded":"2024-02-26T10:41:03.113873+0000","last_fullsized":"2024-02-26T10:41:03.113873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:39:29.874066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366995+0000","last_change":"2024-02-26T10:41:00.789707+0000","last_active":"2024-02-26T10:41:03.366995+0000","last_peered":"2024-02-26T10:41:03.366995+0000","last_clean":"2024-02-26T10:41:03.366995+0000","last_became_active":"2024-02-26T10:41:00.787710+0000","last_became_peered":"2024-02-26T10:41:00.787710+0000","last_unstale":"2024-02-26T10:41:03.366995+0000","last_undegraded":"2024-02-26T10:41:03.366995+0000","last_fullsized":"2024-02-26T10:41:03.366995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:52:45.987760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.766459+0000","last_change":"2024-02-26T10:40:45.640605+0000","last_active":"2024-02-26T10:41:01.766459+0000","last_peered":"2024-02-26T10:41:01.766459+0000","last_clean":"2024-02-26T10:41:01.766459+0000","last_became_active":"2024-02-26T10:40:45.640078+0000","last_became_peered":"2024-02-26T10:40:45.640078+0000","last_unstale":"2024-02-26T10:41:01.766459+0000","last_undegraded":"2024-02-26T10:41:01.766459+0000","last_fullsized":"2024-02-26T10:41:01.766459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:28.520753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":25,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241233+0000","last_change":"2024-02-26T10:40:44.618998+0000","last_active":"2024-02-26T10:41:03.241233+0000","last_peered":"2024-02-26T10:41:03.241233+0000","last_clean":"2024-02-26T10:41:03.241233+0000","last_became_active":"2024-02-26T10:40:44.618710+0000","last_became_peered":"2024-02-26T10:40:44.618710+0000","last_unstale":"2024-02-26T10:41:03.241233+0000","last_undegraded":"2024-02-26T10:41:03.241233+0000","last_fullsized":"2024-02-26T10:41:03.241233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:55.190540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241129+0000","last_change":"2024-02-26T10:41:01.799011+0000","last_active":"2024-02-26T10:41:03.241129+0000","last_peered":"2024-02-26T10:41:03.241129+0000","last_clean":"2024-02-26T10:41:03.241129+0000","last_became_active":"2024-02-26T10:41:01.798726+0000","last_became_peered":"2024-02-26T10:41:01.798726+0000","last_unstale":"2024-02-26T10:41:03.241129+0000","last_undegraded":"2024-02-26T10:41:03.241129+0000","last_fullsized":"2024-02-26T10:41:03.241129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:56:24.277610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793687+0000","last_change":"2024-02-26T10:41:00.792723+0000","last_active":"2024-02-26T10:41:02.793687+0000","last_peered":"2024-02-26T10:41:02.793687+0000","last_clean":"2024-02-26T10:41:02.793687+0000","last_became_active":"2024-02-26T10:41:00.792217+0000","last_became_peered":"2024-02-26T10:41:00.792217+0000","last_unstale":"2024-02-26T10:41:02.793687+0000","last_undegraded":"2024-02-26T10:41:02.793687+0000","last_fullsized":"2024-02-26T10:41:02.793687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:19.984955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240905+0000","last_change":"2024-02-26T10:40:45.652659+0000","last_active":"2024-02-26T10:41:03.240905+0000","last_peered":"2024-02-26T10:41:03.240905+0000","last_clean":"2024-02-26T10:41:03.240905+0000","last_became_active":"2024-02-26T10:40:45.652312+0000","last_became_peered":"2024-02-26T10:40:45.652312+0000","last_unstale":"2024-02-26T10:41:03.240905+0000","last_undegraded":"2024-02-26T10:41:03.240905+0000","last_fullsized":"2024-02-26T10:41:03.240905+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'9","reported_seq":31,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367191+0000","last_change":"2024-02-26T10:40:44.615861+0000","last_active":"2024-02-26T10:41:03.367191+0000","last_peered":"2024-02-26T10:41:03.367191+0000","last_clean":"2024-02-26T10:41:03.367191+0000","last_became_active":"2024-02-26T10:40:44.615538+0000","last_became_peered":"2024-02-26T10:40:44.615538+0000","last_unstale":"2024-02-26T10:41:03.367191+0000","last_undegraded":"2024-02-26T10:41:03.367191+0000","last_fullsized":"2024-02-26T10:41:03.367191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:29.960665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3115,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.802711+0000","last_change":"2024-02-26T10:41:01.799282+0000","last_active":"2024-02-26T10:41:01.802711+0000","last_peered":"2024-02-26T10:41:01.802711+0000","last_clean":"2024-02-26T10:41:01.802711+0000","last_became_active":"2024-02-26T10:41:01.797979+0000","last_became_peered":"2024-02-26T10:41:01.797979+0000","last_unstale":"2024-02-26T10:41:01.802711+0000","last_undegraded":"2024-02-26T10:41:01.802711+0000","last_fullsized":"2024-02-26T10:41:01.802711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:27:13.718577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112873+0000","last_change":"2024-02-26T10:41:00.788343+0000","last_active":"2024-02-26T10:41:03.112873+0000","last_peered":"2024-02-26T10:41:03.112873+0000","last_clean":"2024-02-26T10:41:03.112873+0000","last_became_active":"2024-02-26T10:41:00.788053+0000","last_became_peered":"2024-02-26T10:41:00.788053+0000","last_unstale":"2024-02-26T10:41:03.112873+0000","last_undegraded":"2024-02-26T10:41:03.112873+0000","last_fullsized":"2024-02-26T10:41:03.112873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:50.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112728+0000","last_change":"2024-02-26T10:40:45.628631+0000","last_active":"2024-02-26T10:41:03.112728+0000","last_peered":"2024-02-26T10:41:03.112728+0000","last_clean":"2024-02-26T10:41:03.112728+0000","last_became_active":"2024-02-26T10:40:45.628321+0000","last_became_peered":"2024-02-26T10:40:45.628321+0000","last_unstale":"2024-02-26T10:41:03.112728+0000","last_undegraded":"2024-02-26T10:41:03.112728+0000","last_fullsized":"2024-02-26T10:41:03.112728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":24,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795703+0000","last_change":"2024-02-26T10:40:44.617278+0000","last_active":"2024-02-26T10:41:02.795703+0000","last_peered":"2024-02-26T10:41:02.795703+0000","last_clean":"2024-02-26T10:41:02.795703+0000","last_became_active":"2024-02-26T10:40:44.610922+0000","last_became_peered":"2024-02-26T10:40:44.610922+0000","last_unstale":"2024-02-26T10:41:02.795703+0000","last_undegraded":"2024-02-26T10:41:02.795703+0000","last_fullsized":"2024-02-26T10:41:02.795703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:31.392364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799559+0000","last_change":"2024-02-26T10:41:01.792387+0000","last_active":"2024-02-26T10:41:01.799559+0000","last_peered":"2024-02-26T10:41:01.799559+0000","last_clean":"2024-02-26T10:41:01.799559+0000","last_became_active":"2024-02-26T10:41:01.791270+0000","last_became_peered":"2024-02-26T10:41:01.791270+0000","last_unstale":"2024-02-26T10:41:01.799559+0000","last_undegraded":"2024-02-26T10:41:01.799559+0000","last_fullsized":"2024-02-26T10:41:01.799559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:51.889988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365462+0000","last_change":"2024-02-26T10:41:00.779242+0000","last_active":"2024-02-26T10:41:03.365462+0000","last_peered":"2024-02-26T10:41:03.365462+0000","last_clean":"2024-02-26T10:41:03.365462+0000","last_became_active":"2024-02-26T10:41:00.779002+0000","last_became_peered":"2024-02-26T10:41:00.779002+0000","last_unstale":"2024-02-26T10:41:03.365462+0000","last_undegraded":"2024-02-26T10:41:03.365462+0000","last_fullsized":"2024-02-26T10:41:03.365462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:33.486965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240258+0000","last_change":"2024-02-26T10:40:45.644578+0000","last_active":"2024-02-26T10:41:03.240258+0000","last_peered":"2024-02-26T10:41:03.240258+0000","last_clean":"2024-02-26T10:41:03.240258+0000","last_became_active":"2024-02-26T10:40:45.644211+0000","last_became_peered":"2024-02-26T10:40:45.644211+0000","last_unstale":"2024-02-26T10:41:03.240258+0000","last_undegraded":"2024-02-26T10:41:03.240258+0000","last_fullsized":"2024-02-26T10:41:03.240258+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793443+0000","last_change":"2024-02-26T10:40:44.618437+0000","last_active":"2024-02-26T10:41:02.793443+0000","last_peered":"2024-02-26T10:41:02.793443+0000","last_clean":"2024-02-26T10:41:02.793443+0000","last_became_active":"2024-02-26T10:40:44.617307+0000","last_became_peered":"2024-02-26T10:40:44.617307+0000","last_unstale":"2024-02-26T10:41:02.793443+0000","last_undegraded":"2024-02-26T10:41:02.793443+0000","last_fullsized":"2024-02-26T10:41:02.793443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:15.126480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239609+0000","last_change":"2024-02-26T10:41:01.803637+0000","last_active":"2024-02-26T10:41:03.239609+0000","last_peered":"2024-02-26T10:41:03.239609+0000","last_clean":"2024-02-26T10:41:03.239609+0000","last_became_active":"2024-02-26T10:41:01.803307+0000","last_became_peered":"2024-02-26T10:41:01.803307+0000","last_unstale":"2024-02-26T10:41:03.239609+0000","last_undegraded":"2024-02-26T10:41:03.239609+0000","last_fullsized":"2024-02-26T10:41:03.239609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:25.572381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"58'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.801904+0000","last_change":"2024-02-26T10:41:00.786760+0000","last_active":"2024-02-26T10:41:01.801904+0000","last_peered":"2024-02-26T10:41:01.801904+0000","last_clean":"2024-02-26T10:41:01.801904+0000","last_became_active":"2024-02-26T10:41:00.786438+0000","last_became_peered":"2024-02-26T10:41:00.786438+0000","last_unstale":"2024-02-26T10:41:01.801904+0000","last_undegraded":"2024-02-26T10:41:01.801904+0000","last_fullsized":"2024-02-26T10:41:01.801904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:22.952631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365734+0000","last_change":"2024-02-26T10:40:45.643600+0000","last_active":"2024-02-26T10:41:03.365734+0000","last_peered":"2024-02-26T10:41:03.365734+0000","last_clean":"2024-02-26T10:41:03.365734+0000","last_became_active":"2024-02-26T10:40:45.643190+0000","last_became_peered":"2024-02-26T10:40:45.643190+0000","last_unstale":"2024-02-26T10:41:03.365734+0000","last_undegraded":"2024-02-26T10:41:03.365734+0000","last_fullsized":"2024-02-26T10:41:03.365734+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:35.142103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113071+0000","last_change":"2024-02-26T10:40:44.615356+0000","last_active":"2024-02-26T10:41:03.113071+0000","last_peered":"2024-02-26T10:41:03.113071+0000","last_clean":"2024-02-26T10:41:03.113071+0000","last_became_active":"2024-02-26T10:40:44.615072+0000","last_became_peered":"2024-02-26T10:40:44.615072+0000","last_unstale":"2024-02-26T10:41:03.113071+0000","last_undegraded":"2024-02-26T10:41:03.113071+0000","last_fullsized":"2024-02-26T10:41:03.113071+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:06.798865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113151+0000","last_change":"2024-02-26T10:41:01.780694+0000","last_active":"2024-02-26T10:41:03.113151+0000","last_peered":"2024-02-26T10:41:03.113151+0000","last_clean":"2024-02-26T10:41:03.113151+0000","last_became_active":"2024-02-26T10:41:01.780248+0000","last_became_peered":"2024-02-26T10:41:01.780248+0000","last_unstale":"2024-02-26T10:41:03.113151+0000","last_undegraded":"2024-02-26T10:41:03.113151+0000","last_fullsized":"2024-02-26T10:41:03.113151+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:22.919759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793161+0000","last_change":"2024-02-26T10:41:00.790030+0000","last_active":"2024-02-26T10:41:02.793161+0000","last_peered":"2024-02-26T10:41:02.793161+0000","last_clean":"2024-02-26T10:41:02.793161+0000","last_became_active":"2024-02-26T10:41:00.789473+0000","last_became_peered":"2024-02-26T10:41:00.789473+0000","last_unstale":"2024-02-26T10:41:02.793161+0000","last_undegraded":"2024-02-26T10:41:02.793161+0000","last_fullsized":"2024-02-26T10:41:02.793161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:15.501040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239826+0000","last_change":"2024-02-26T10:40:45.653780+0000","last_active":"2024-02-26T10:41:03.239826+0000","last_peered":"2024-02-26T10:41:03.239826+0000","last_clean":"2024-02-26T10:41:03.239826+0000","last_became_active":"2024-02-26T10:40:45.653336+0000","last_became_peered":"2024-02-26T10:40:45.653336+0000","last_unstale":"2024-02-26T10:41:03.239826+0000","last_undegraded":"2024-02-26T10:41:03.239826+0000","last_fullsized":"2024-02-26T10:41:03.239826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:49:13.271700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":30,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239158+0000","last_change":"2024-02-26T10:40:44.621458+0000","last_active":"2024-02-26T10:41:03.239158+0000","last_peered":"2024-02-26T10:41:03.239158+0000","last_clean":"2024-02-26T10:41:03.239158+0000","last_became_active":"2024-02-26T10:40:44.620425+0000","last_became_peered":"2024-02-26T10:40:44.620425+0000","last_unstale":"2024-02-26T10:41:03.239158+0000","last_undegraded":"2024-02-26T10:41:03.239158+0000","last_fullsized":"2024-02-26T10:41:03.239158+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:47.713723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.786181+0000","last_change":"2024-02-26T10:41:01.777865+0000","last_active":"2024-02-26T10:41:01.786181+0000","last_peered":"2024-02-26T10:41:01.786181+0000","last_clean":"2024-02-26T10:41:01.786181+0000","last_became_active":"2024-02-26T10:41:01.777419+0000","last_became_peered":"2024-02-26T10:41:01.777419+0000","last_unstale":"2024-02-26T10:41:01.786181+0000","last_undegraded":"2024-02-26T10:41:01.786181+0000","last_fullsized":"2024-02-26T10:41:01.786181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:16.517326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792414+0000","last_change":"2024-02-26T10:41:00.795096+0000","last_active":"2024-02-26T10:41:02.792414+0000","last_peered":"2024-02-26T10:41:02.792414+0000","last_clean":"2024-02-26T10:41:02.792414+0000","last_became_active":"2024-02-26T10:41:00.794058+0000","last_became_peered":"2024-02-26T10:41:00.794058+0000","last_unstale":"2024-02-26T10:41:02.792414+0000","last_undegraded":"2024-02-26T10:41:02.792414+0000","last_fullsized":"2024-02-26T10:41:02.792414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:30.738848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241175+0000","last_change":"2024-02-26T10:40:45.654175+0000","last_active":"2024-02-26T10:41:03.241175+0000","last_peered":"2024-02-26T10:41:03.241175+0000","last_clean":"2024-02-26T10:41:03.241175+0000","last_became_active":"2024-02-26T10:40:45.653092+0000","last_became_peered":"2024-02-26T10:40:45.653092+0000","last_unstale":"2024-02-26T10:41:03.241175+0000","last_undegraded":"2024-02-26T10:41:03.241175+0000","last_fullsized":"2024-02-26T10:41:03.241175+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:48.803196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.799457+0000","last_change":"2024-02-26T10:40:44.614563+0000","last_active":"2024-02-26T10:41:02.799457+0000","last_peered":"2024-02-26T10:41:02.799457+0000","last_clean":"2024-02-26T10:41:02.799457+0000","last_became_active":"2024-02-26T10:40:44.614281+0000","last_became_peered":"2024-02-26T10:40:44.614281+0000","last_unstale":"2024-02-26T10:41:02.799457+0000","last_undegraded":"2024-02-26T10:41:02.799457+0000","last_fullsized":"2024-02-26T10:41:02.799457+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:58.146019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781917+0000","last_change":"2024-02-26T10:41:01.778881+0000","last_active":"2024-02-26T10:41:01.781917+0000","last_peered":"2024-02-26T10:41:01.781917+0000","last_clean":"2024-02-26T10:41:01.781917+0000","last_became_active":"2024-02-26T10:41:01.778567+0000","last_became_peered":"2024-02-26T10:41:01.778567+0000","last_unstale":"2024-02-26T10:41:01.781917+0000","last_undegraded":"2024-02-26T10:41:01.781917+0000","last_fullsized":"2024-02-26T10:41:01.781917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:30:26.138159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240358+0000","last_change":"2024-02-26T10:41:00.791881+0000","last_active":"2024-02-26T10:41:03.240358+0000","last_peered":"2024-02-26T10:41:03.240358+0000","last_clean":"2024-02-26T10:41:03.240358+0000","last_became_active":"2024-02-26T10:41:00.790958+0000","last_became_peered":"2024-02-26T10:41:00.790958+0000","last_unstale":"2024-02-26T10:41:03.240358+0000","last_undegraded":"2024-02-26T10:41:03.240358+0000","last_fullsized":"2024-02-26T10:41:03.240358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:50.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114128+0000","last_change":"2024-02-26T10:40:45.628322+0000","last_active":"2024-02-26T10:41:03.114128+0000","last_peered":"2024-02-26T10:41:03.114128+0000","last_clean":"2024-02-26T10:41:03.114128+0000","last_became_active":"2024-02-26T10:40:45.624341+0000","last_became_peered":"2024-02-26T10:40:45.624341+0000","last_unstale":"2024-02-26T10:41:03.114128+0000","last_undegraded":"2024-02-26T10:41:03.114128+0000","last_fullsized":"2024-02-26T10:41:03.114128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.782961+0000","last_change":"2024-02-26T10:40:44.613428+0000","last_active":"2024-02-26T10:41:01.782961+0000","last_peered":"2024-02-26T10:41:01.782961+0000","last_clean":"2024-02-26T10:41:01.782961+0000","last_became_active":"2024-02-26T10:40:44.613145+0000","last_became_peered":"2024-02-26T10:40:44.613145+0000","last_unstale":"2024-02-26T10:41:01.782961+0000","last_undegraded":"2024-02-26T10:41:01.782961+0000","last_fullsized":"2024-02-26T10:41:01.782961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:55.190540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367545+0000","last_change":"2024-02-26T10:41:01.782129+0000","last_active":"2024-02-26T10:41:03.367545+0000","last_peered":"2024-02-26T10:41:03.367545+0000","last_clean":"2024-02-26T10:41:03.367545+0000","last_became_active":"2024-02-26T10:41:01.780673+0000","last_became_peered":"2024-02-26T10:41:01.780673+0000","last_unstale":"2024-02-26T10:41:03.367545+0000","last_undegraded":"2024-02-26T10:41:03.367545+0000","last_fullsized":"2024-02-26T10:41:03.367545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:19.088038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792713+0000","last_change":"2024-02-26T10:41:00.792372+0000","last_active":"2024-02-26T10:41:02.792713+0000","last_peered":"2024-02-26T10:41:02.792713+0000","last_clean":"2024-02-26T10:41:02.792713+0000","last_became_active":"2024-02-26T10:41:00.792068+0000","last_became_peered":"2024-02-26T10:41:00.792068+0000","last_unstale":"2024-02-26T10:41:02.792713+0000","last_undegraded":"2024-02-26T10:41:02.792713+0000","last_fullsized":"2024-02-26T10:41:02.792713+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:30.430501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795588+0000","last_change":"2024-02-26T10:40:45.637902+0000","last_active":"2024-02-26T10:41:02.795588+0000","last_peered":"2024-02-26T10:41:02.795588+0000","last_clean":"2024-02-26T10:41:02.795588+0000","last_became_active":"2024-02-26T10:40:45.636813+0000","last_became_peered":"2024-02-26T10:40:45.636813+0000","last_unstale":"2024-02-26T10:41:02.795588+0000","last_undegraded":"2024-02-26T10:41:02.795588+0000","last_fullsized":"2024-02-26T10:41:02.795588+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:06.754252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113595+0000","last_change":"2024-02-26T10:40:44.616165+0000","last_active":"2024-02-26T10:41:03.113595+0000","last_peered":"2024-02-26T10:41:03.113595+0000","last_clean":"2024-02-26T10:41:03.113595+0000","last_became_active":"2024-02-26T10:40:44.614801+0000","last_became_peered":"2024-02-26T10:40:44.614801+0000","last_unstale":"2024-02-26T10:41:03.113595+0000","last_undegraded":"2024-02-26T10:41:03.113595+0000","last_fullsized":"2024-02-26T10:41:03.113595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:17.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241714+0000","last_change":"2024-02-26T10:41:01.791218+0000","last_active":"2024-02-26T10:41:03.241714+0000","last_peered":"2024-02-26T10:41:03.241714+0000","last_clean":"2024-02-26T10:41:03.241714+0000","last_became_active":"2024-02-26T10:41:01.790919+0000","last_became_peered":"2024-02-26T10:41:01.790919+0000","last_unstale":"2024-02-26T10:41:03.241714+0000","last_undegraded":"2024-02-26T10:41:03.241714+0000","last_fullsized":"2024-02-26T10:41:03.241714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:58.921535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367154+0000","last_change":"2024-02-26T10:41:00.794191+0000","last_active":"2024-02-26T10:41:03.367154+0000","last_peered":"2024-02-26T10:41:03.367154+0000","last_clean":"2024-02-26T10:41:03.367154+0000","last_became_active":"2024-02-26T10:41:00.793888+0000","last_became_peered":"2024-02-26T10:41:00.793888+0000","last_unstale":"2024-02-26T10:41:03.367154+0000","last_undegraded":"2024-02-26T10:41:03.367154+0000","last_fullsized":"2024-02-26T10:41:03.367154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:48.425059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241056+0000","last_change":"2024-02-26T10:40:45.655962+0000","last_active":"2024-02-26T10:41:03.241056+0000","last_peered":"2024-02-26T10:41:03.241056+0000","last_clean":"2024-02-26T10:41:03.241056+0000","last_became_active":"2024-02-26T10:40:45.655701+0000","last_became_peered":"2024-02-26T10:40:45.655701+0000","last_unstale":"2024-02-26T10:41:03.241056+0000","last_undegraded":"2024-02-26T10:41:03.241056+0000","last_fullsized":"2024-02-26T10:41:03.241056+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780514+0000","last_change":"2024-02-26T10:40:44.609742+0000","last_active":"2024-02-26T10:41:01.780514+0000","last_peered":"2024-02-26T10:41:01.780514+0000","last_clean":"2024-02-26T10:41:01.780514+0000","last_became_active":"2024-02-26T10:40:44.609452+0000","last_became_peered":"2024-02-26T10:40:44.609452+0000","last_unstale":"2024-02-26T10:41:01.780514+0000","last_undegraded":"2024-02-26T10:41:01.780514+0000","last_fullsized":"2024-02-26T10:41:01.780514+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:55.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792994+0000","last_change":"2024-02-26T10:41:01.798444+0000","last_active":"2024-02-26T10:41:02.792994+0000","last_peered":"2024-02-26T10:41:02.792994+0000","last_clean":"2024-02-26T10:41:02.792994+0000","last_became_active":"2024-02-26T10:41:01.797201+0000","last_became_peered":"2024-02-26T10:41:01.797201+0000","last_unstale":"2024-02-26T10:41:02.792994+0000","last_undegraded":"2024-02-26T10:41:02.792994+0000","last_fullsized":"2024-02-26T10:41:02.792994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:07.040156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112620+0000","last_change":"2024-02-26T10:41:00.788831+0000","last_active":"2024-02-26T10:41:03.112620+0000","last_peered":"2024-02-26T10:41:03.112620+0000","last_clean":"2024-02-26T10:41:03.112620+0000","last_became_active":"2024-02-26T10:41:00.787849+0000","last_became_peered":"2024-02-26T10:41:00.787849+0000","last_unstale":"2024-02-26T10:41:03.112620+0000","last_undegraded":"2024-02-26T10:41:03.112620+0000","last_fullsized":"2024-02-26T10:41:03.112620+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:16.987475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112857+0000","last_change":"2024-02-26T10:40:45.631443+0000","last_active":"2024-02-26T10:41:03.112857+0000","last_peered":"2024-02-26T10:41:03.112857+0000","last_clean":"2024-02-26T10:41:03.112857+0000","last_became_active":"2024-02-26T10:40:45.631119+0000","last_became_peered":"2024-02-26T10:40:45.631119+0000","last_unstale":"2024-02-26T10:41:03.112857+0000","last_undegraded":"2024-02-26T10:41:03.112857+0000","last_fullsized":"2024-02-26T10:41:03.112857+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":24,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.796159+0000","last_change":"2024-02-26T10:40:44.616723+0000","last_active":"2024-02-26T10:41:02.796159+0000","last_peered":"2024-02-26T10:41:02.796159+0000","last_clean":"2024-02-26T10:41:02.796159+0000","last_became_active":"2024-02-26T10:40:44.616480+0000","last_became_peered":"2024-02-26T10:40:44.616480+0000","last_unstale":"2024-02-26T10:41:02.796159+0000","last_undegraded":"2024-02-26T10:41:02.796159+0000","last_fullsized":"2024-02-26T10:41:02.796159+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:42.095622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793711+0000","last_change":"2024-02-26T10:41:01.795973+0000","last_active":"2024-02-26T10:41:02.793711+0000","last_peered":"2024-02-26T10:41:02.793711+0000","last_clean":"2024-02-26T10:41:02.793711+0000","last_became_active":"2024-02-26T10:41:01.795683+0000","last_became_peered":"2024-02-26T10:41:01.795683+0000","last_unstale":"2024-02-26T10:41:02.793711+0000","last_undegraded":"2024-02-26T10:41:02.793711+0000","last_fullsized":"2024-02-26T10:41:02.793711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:58.927024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.773689+0000","last_change":"2024-02-26T10:41:00.778570+0000","last_active":"2024-02-26T10:41:01.773689+0000","last_peered":"2024-02-26T10:41:01.773689+0000","last_clean":"2024-02-26T10:41:01.773689+0000","last_became_active":"2024-02-26T10:41:00.778299+0000","last_became_peered":"2024-02-26T10:41:00.778299+0000","last_unstale":"2024-02-26T10:41:01.773689+0000","last_undegraded":"2024-02-26T10:41:01.773689+0000","last_fullsized":"2024-02-26T10:41:01.773689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:27.400748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.779431+0000","last_change":"2024-02-26T10:40:45.639556+0000","last_active":"2024-02-26T10:41:01.779431+0000","last_peered":"2024-02-26T10:41:01.779431+0000","last_clean":"2024-02-26T10:41:01.779431+0000","last_became_active":"2024-02-26T10:40:45.639271+0000","last_became_peered":"2024-02-26T10:40:45.639271+0000","last_unstale":"2024-02-26T10:41:01.779431+0000","last_undegraded":"2024-02-26T10:41:01.779431+0000","last_fullsized":"2024-02-26T10:41:01.779431+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763918+0000","last_change":"2024-02-26T10:40:44.612287+0000","last_active":"2024-02-26T10:41:01.763918+0000","last_peered":"2024-02-26T10:41:01.763918+0000","last_clean":"2024-02-26T10:41:01.763918+0000","last_became_active":"2024-02-26T10:40:44.611972+0000","last_became_peered":"2024-02-26T10:40:44.611972+0000","last_unstale":"2024-02-26T10:41:01.763918+0000","last_undegraded":"2024-02-26T10:41:01.763918+0000","last_fullsized":"2024-02-26T10:41:01.763918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:38.390162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794866+0000","last_change":"2024-02-26T10:41:01.789881+0000","last_active":"2024-02-26T10:41:02.794866+0000","last_peered":"2024-02-26T10:41:02.794866+0000","last_clean":"2024-02-26T10:41:02.794866+0000","last_became_active":"2024-02-26T10:41:01.789504+0000","last_became_peered":"2024-02-26T10:41:01.789504+0000","last_unstale":"2024-02-26T10:41:02.794866+0000","last_undegraded":"2024-02-26T10:41:02.794866+0000","last_fullsized":"2024-02-26T10:41:02.794866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:07.205812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793427+0000","last_change":"2024-02-26T10:41:00.790257+0000","last_active":"2024-02-26T10:41:02.793427+0000","last_peered":"2024-02-26T10:41:02.793427+0000","last_clean":"2024-02-26T10:41:02.793427+0000","last_became_active":"2024-02-26T10:41:00.789638+0000","last_became_peered":"2024-02-26T10:41:00.789638+0000","last_unstale":"2024-02-26T10:41:02.793427+0000","last_undegraded":"2024-02-26T10:41:02.793427+0000","last_fullsized":"2024-02-26T10:41:02.793427+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:52:45.987760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.775895+0000","last_change":"2024-02-26T10:40:45.644041+0000","last_active":"2024-02-26T10:41:01.775895+0000","last_peered":"2024-02-26T10:41:01.775895+0000","last_clean":"2024-02-26T10:41:01.775895+0000","last_became_active":"2024-02-26T10:40:45.640298+0000","last_became_peered":"2024-02-26T10:40:45.640298+0000","last_unstale":"2024-02-26T10:41:01.775895+0000","last_undegraded":"2024-02-26T10:41:01.775895+0000","last_fullsized":"2024-02-26T10:41:01.775895+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:49:13.271700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.111963+0000","last_change":"2024-02-26T10:40:44.611018+0000","last_active":"2024-02-26T10:41:03.111963+0000","last_peered":"2024-02-26T10:41:03.111963+0000","last_clean":"2024-02-26T10:41:03.111963+0000","last_became_active":"2024-02-26T10:40:44.610676+0000","last_became_peered":"2024-02-26T10:40:44.610676+0000","last_unstale":"2024-02-26T10:41:03.111963+0000","last_undegraded":"2024-02-26T10:41:03.111963+0000","last_fullsized":"2024-02-26T10:41:03.111963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:46:52.342064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240045+0000","last_change":"2024-02-26T10:41:01.799263+0000","last_active":"2024-02-26T10:41:03.240045+0000","last_peered":"2024-02-26T10:41:03.240045+0000","last_clean":"2024-02-26T10:41:03.240045+0000","last_became_active":"2024-02-26T10:41:01.798213+0000","last_became_peered":"2024-02-26T10:41:01.798213+0000","last_unstale":"2024-02-26T10:41:03.240045+0000","last_undegraded":"2024-02-26T10:41:03.240045+0000","last_fullsized":"2024-02-26T10:41:03.240045+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:39:29.874066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.770595+0000","last_change":"2024-02-26T10:41:00.775411+0000","last_active":"2024-02-26T10:41:01.770595+0000","last_peered":"2024-02-26T10:41:01.770595+0000","last_clean":"2024-02-26T10:41:01.770595+0000","last_became_active":"2024-02-26T10:41:00.774584+0000","last_became_peered":"2024-02-26T10:41:00.774584+0000","last_unstale":"2024-02-26T10:41:01.770595+0000","last_undegraded":"2024-02-26T10:41:01.770595+0000","last_fullsized":"2024-02-26T10:41:01.770595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:56.754194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114326+0000","last_change":"2024-02-26T10:40:45.641308+0000","last_active":"2024-02-26T10:41:03.114326+0000","last_peered":"2024-02-26T10:41:03.114326+0000","last_clean":"2024-02-26T10:41:03.114326+0000","last_became_active":"2024-02-26T10:40:45.641045+0000","last_became_peered":"2024-02-26T10:40:45.641045+0000","last_unstale":"2024-02-26T10:41:03.114326+0000","last_undegraded":"2024-02-26T10:41:03.114326+0000","last_fullsized":"2024-02-26T10:41:03.114326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.769480+0000","last_change":"2024-02-26T10:40:44.609096+0000","last_active":"2024-02-26T10:41:01.769480+0000","last_peered":"2024-02-26T10:41:01.769480+0000","last_clean":"2024-02-26T10:41:01.769480+0000","last_became_active":"2024-02-26T10:40:44.608811+0000","last_became_peered":"2024-02-26T10:40:44.608811+0000","last_unstale":"2024-02-26T10:41:01.769480+0000","last_undegraded":"2024-02-26T10:41:01.769480+0000","last_fullsized":"2024-02-26T10:41:01.769480+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:00:39.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240751+0000","last_change":"2024-02-26T10:41:01.797991+0000","last_active":"2024-02-26T10:41:03.240751+0000","last_peered":"2024-02-26T10:41:03.240751+0000","last_clean":"2024-02-26T10:41:03.240751+0000","last_became_active":"2024-02-26T10:41:01.797620+0000","last_became_peered":"2024-02-26T10:41:01.797620+0000","last_unstale":"2024-02-26T10:41:03.240751+0000","last_undegraded":"2024-02-26T10:41:03.240751+0000","last_fullsized":"2024-02-26T10:41:03.240751+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:11:20.045564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780200+0000","last_change":"2024-02-26T10:41:00.773110+0000","last_active":"2024-02-26T10:41:01.780200+0000","last_peered":"2024-02-26T10:41:01.780200+0000","last_clean":"2024-02-26T10:41:01.780200+0000","last_became_active":"2024-02-26T10:41:00.772804+0000","last_became_peered":"2024-02-26T10:41:00.772804+0000","last_unstale":"2024-02-26T10:41:01.780200+0000","last_undegraded":"2024-02-26T10:41:01.780200+0000","last_fullsized":"2024-02-26T10:41:01.780200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:45.340492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114161+0000","last_change":"2024-02-26T10:40:45.642661+0000","last_active":"2024-02-26T10:41:03.114161+0000","last_peered":"2024-02-26T10:41:03.114161+0000","last_clean":"2024-02-26T10:41:03.114161+0000","last_became_active":"2024-02-26T10:40:45.640611+0000","last_became_peered":"2024-02-26T10:40:45.640611+0000","last_unstale":"2024-02-26T10:41:03.114161+0000","last_undegraded":"2024-02-26T10:41:03.114161+0000","last_fullsized":"2024-02-26T10:41:03.114161+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:14.395806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.783087+0000","last_change":"2024-02-26T10:40:44.612941+0000","last_active":"2024-02-26T10:41:01.783087+0000","last_peered":"2024-02-26T10:41:01.783087+0000","last_clean":"2024-02-26T10:41:01.783087+0000","last_became_active":"2024-02-26T10:40:44.612610+0000","last_became_peered":"2024-02-26T10:40:44.612610+0000","last_unstale":"2024-02-26T10:41:01.783087+0000","last_undegraded":"2024-02-26T10:41:01.783087+0000","last_fullsized":"2024-02-26T10:41:01.783087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:06.688227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.803343+0000","last_change":"2024-02-26T10:41:01.801264+0000","last_active":"2024-02-26T10:41:01.803343+0000","last_peered":"2024-02-26T10:41:01.803343+0000","last_clean":"2024-02-26T10:41:01.803343+0000","last_became_active":"2024-02-26T10:41:01.800910+0000","last_became_peered":"2024-02-26T10:41:01.800910+0000","last_unstale":"2024-02-26T10:41:01.803343+0000","last_undegraded":"2024-02-26T10:41:01.803343+0000","last_fullsized":"2024-02-26T10:41:01.803343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:49.607021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793600+0000","last_change":"2024-02-26T10:41:00.792170+0000","last_active":"2024-02-26T10:41:02.793600+0000","last_peered":"2024-02-26T10:41:02.793600+0000","last_clean":"2024-02-26T10:41:02.793600+0000","last_became_active":"2024-02-26T10:41:00.791870+0000","last_became_peered":"2024-02-26T10:41:00.791870+0000","last_unstale":"2024-02-26T10:41:02.793600+0000","last_undegraded":"2024-02-26T10:41:02.793600+0000","last_fullsized":"2024-02-26T10:41:02.793600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:12:42.026861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.765253+0000","last_change":"2024-02-26T10:40:45.640358+0000","last_active":"2024-02-26T10:41:01.765253+0000","last_peered":"2024-02-26T10:41:01.765253+0000","last_clean":"2024-02-26T10:41:01.765253+0000","last_became_active":"2024-02-26T10:40:45.639707+0000","last_became_peered":"2024-02-26T10:40:45.639707+0000","last_unstale":"2024-02-26T10:41:01.765253+0000","last_undegraded":"2024-02-26T10:41:01.765253+0000","last_fullsized":"2024-02-26T10:41:01.765253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:03.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792881+0000","last_change":"2024-02-26T10:40:44.619084+0000","last_active":"2024-02-26T10:41:02.792881+0000","last_peered":"2024-02-26T10:41:02.792881+0000","last_clean":"2024-02-26T10:41:02.792881+0000","last_became_active":"2024-02-26T10:40:44.618715+0000","last_became_peered":"2024-02-26T10:40:44.618715+0000","last_unstale":"2024-02-26T10:41:02.792881+0000","last_undegraded":"2024-02-26T10:41:02.792881+0000","last_fullsized":"2024-02-26T10:41:02.792881+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:58.496066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114020+0000","last_change":"2024-02-26T10:41:01.790983+0000","last_active":"2024-02-26T10:41:03.114020+0000","last_peered":"2024-02-26T10:41:03.114020+0000","last_clean":"2024-02-26T10:41:03.114020+0000","last_became_active":"2024-02-26T10:41:01.790642+0000","last_became_peered":"2024-02-26T10:41:01.790642+0000","last_unstale":"2024-02-26T10:41:03.114020+0000","last_undegraded":"2024-02-26T10:41:03.114020+0000","last_fullsized":"2024-02-26T10:41:03.114020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:10.745518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114154+0000","last_change":"2024-02-26T10:41:00.789535+0000","last_active":"2024-02-26T10:41:03.114154+0000","last_peered":"2024-02-26T10:41:03.114154+0000","last_clean":"2024-02-26T10:41:03.114154+0000","last_became_active":"2024-02-26T10:41:00.789225+0000","last_became_peered":"2024-02-26T10:41:00.789225+0000","last_unstale":"2024-02-26T10:41:03.114154+0000","last_undegraded":"2024-02-26T10:41:03.114154+0000","last_fullsized":"2024-02-26T10:41:03.114154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:45.340492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781273+0000","last_change":"2024-02-26T10:40:45.630528+0000","last_active":"2024-02-26T10:41:01.781273+0000","last_peered":"2024-02-26T10:41:01.781273+0000","last_clean":"2024-02-26T10:41:01.781273+0000","last_became_active":"2024-02-26T10:40:45.630230+0000","last_became_peered":"2024-02-26T10:40:45.630230+0000","last_unstale":"2024-02-26T10:41:01.781273+0000","last_undegraded":"2024-02-26T10:41:01.781273+0000","last_fullsized":"2024-02-26T10:41:01.781273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113685+0000","last_change":"2024-02-26T10:40:44.619432+0000","last_active":"2024-02-26T10:41:03.113685+0000","last_peered":"2024-02-26T10:41:03.113685+0000","last_clean":"2024-02-26T10:41:03.113685+0000","last_became_active":"2024-02-26T10:40:44.619141+0000","last_became_peered":"2024-02-26T10:40:44.619141+0000","last_unstale":"2024-02-26T10:41:03.113685+0000","last_undegraded":"2024-02-26T10:41:03.113685+0000","last_fullsized":"2024-02-26T10:41:03.113685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:05.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.790251+0000","last_change":"2024-02-26T10:41:01.786301+0000","last_active":"2024-02-26T10:41:01.790251+0000","last_peered":"2024-02-26T10:41:01.790251+0000","last_clean":"2024-02-26T10:41:01.790251+0000","last_became_active":"2024-02-26T10:41:01.785990+0000","last_became_peered":"2024-02-26T10:41:01.785990+0000","last_unstale":"2024-02-26T10:41:01.790251+0000","last_undegraded":"2024-02-26T10:41:01.790251+0000","last_fullsized":"2024-02-26T10:41:01.790251+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:21.001241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365351+0000","last_change":"2024-02-26T10:41:00.777167+0000","last_active":"2024-02-26T10:41:03.365351+0000","last_peered":"2024-02-26T10:41:03.365351+0000","last_clean":"2024-02-26T10:41:03.365351+0000","last_became_active":"2024-02-26T10:41:00.776713+0000","last_became_peered":"2024-02-26T10:41:00.776713+0000","last_unstale":"2024-02-26T10:41:03.365351+0000","last_undegraded":"2024-02-26T10:41:03.365351+0000","last_fullsized":"2024-02-26T10:41:03.365351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:21.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794975+0000","last_change":"2024-02-26T10:40:45.634981+0000","last_active":"2024-02-26T10:41:02.794975+0000","last_peered":"2024-02-26T10:41:02.794975+0000","last_clean":"2024-02-26T10:41:02.794975+0000","last_became_active":"2024-02-26T10:40:45.634638+0000","last_became_peered":"2024-02-26T10:40:45.634638+0000","last_unstale":"2024-02-26T10:41:02.794975+0000","last_undegraded":"2024-02-26T10:41:02.794975+0000","last_fullsized":"2024-02-26T10:41:02.794975+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:01:22.095627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.762921+0000","last_change":"2024-02-26T10:40:44.599450+0000","last_active":"2024-02-26T10:41:01.762921+0000","last_peered":"2024-02-26T10:41:01.762921+0000","last_clean":"2024-02-26T10:41:01.762921+0000","last_became_active":"2024-02-26T10:40:44.598802+0000","last_became_peered":"2024-02-26T10:40:44.598802+0000","last_unstale":"2024-02-26T10:41:01.762921+0000","last_undegraded":"2024-02-26T10:41:01.762921+0000","last_fullsized":"2024-02-26T10:41:01.762921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:58.496066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792411+0000","last_change":"2024-02-26T10:41:01.785467+0000","last_active":"2024-02-26T10:41:02.792411+0000","last_peered":"2024-02-26T10:41:02.792411+0000","last_clean":"2024-02-26T10:41:02.792411+0000","last_became_active":"2024-02-26T10:41:01.784587+0000","last_became_peered":"2024-02-26T10:41:01.784587+0000","last_unstale":"2024-02-26T10:41:02.792411+0000","last_undegraded":"2024-02-26T10:41:02.792411+0000","last_fullsized":"2024-02-26T10:41:02.792411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:19.088038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.791877+0000","last_change":"2024-02-26T10:41:00.793641+0000","last_active":"2024-02-26T10:41:02.791877+0000","last_peered":"2024-02-26T10:41:02.791877+0000","last_clean":"2024-02-26T10:41:02.791877+0000","last_became_active":"2024-02-26T10:41:00.793308+0000","last_became_peered":"2024-02-26T10:41:00.793308+0000","last_unstale":"2024-02-26T10:41:02.791877+0000","last_undegraded":"2024-02-26T10:41:02.791877+0000","last_fullsized":"2024-02-26T10:41:02.791877+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:17:14.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240555+0000","last_change":"2024-02-26T10:40:45.650467+0000","last_active":"2024-02-26T10:41:03.240555+0000","last_peered":"2024-02-26T10:41:03.240555+0000","last_clean":"2024-02-26T10:41:03.240555+0000","last_became_active":"2024-02-26T10:40:45.649189+0000","last_became_peered":"2024-02-26T10:40:45.649189+0000","last_unstale":"2024-02-26T10:41:03.240555+0000","last_undegraded":"2024-02-26T10:41:03.240555+0000","last_fullsized":"2024-02-26T10:41:03.240555+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366185+0000","last_change":"2024-02-26T10:40:44.613063+0000","last_active":"2024-02-26T10:41:03.366185+0000","last_peered":"2024-02-26T10:41:03.366185+0000","last_clean":"2024-02-26T10:41:03.366185+0000","last_became_active":"2024-02-26T10:40:44.612792+0000","last_became_peered":"2024-02-26T10:40:44.612792+0000","last_unstale":"2024-02-26T10:41:03.366185+0000","last_undegraded":"2024-02-26T10:41:03.366185+0000","last_fullsized":"2024-02-26T10:41:03.366185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:38.390162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.801870+0000","last_change":"2024-02-26T10:41:01.793419+0000","last_active":"2024-02-26T10:41:01.801870+0000","last_peered":"2024-02-26T10:41:01.801870+0000","last_clean":"2024-02-26T10:41:01.801870+0000","last_became_active":"2024-02-26T10:41:01.785140+0000","last_became_peered":"2024-02-26T10:41:01.785140+0000","last_unstale":"2024-02-26T10:41:01.801870+0000","last_undegraded":"2024-02-26T10:41:01.801870+0000","last_fullsized":"2024-02-26T10:41:01.801870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:34:32.359358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763665+0000","last_change":"2024-02-26T10:41:00.790834+0000","last_active":"2024-02-26T10:41:01.763665+0000","last_peered":"2024-02-26T10:41:01.763665+0000","last_clean":"2024-02-26T10:41:01.763665+0000","last_became_active":"2024-02-26T10:41:00.790521+0000","last_became_peered":"2024-02-26T10:41:00.790521+0000","last_unstale":"2024-02-26T10:41:01.763665+0000","last_undegraded":"2024-02-26T10:41:01.763665+0000","last_fullsized":"2024-02-26T10:41:01.763665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:38.428822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.764538+0000","last_change":"2024-02-26T10:40:45.636109+0000","last_active":"2024-02-26T10:41:01.764538+0000","last_peered":"2024-02-26T10:41:01.764538+0000","last_clean":"2024-02-26T10:41:01.764538+0000","last_became_active":"2024-02-26T10:40:45.635783+0000","last_became_peered":"2024-02-26T10:40:45.635783+0000","last_unstale":"2024-02-26T10:41:01.764538+0000","last_undegraded":"2024-02-26T10:41:01.764538+0000","last_fullsized":"2024-02-26T10:41:01.764538+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:45:18.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763240+0000","last_change":"2024-02-26T10:40:44.595757+0000","last_active":"2024-02-26T10:41:01.763240+0000","last_peered":"2024-02-26T10:41:01.763240+0000","last_clean":"2024-02-26T10:41:01.763240+0000","last_became_active":"2024-02-26T10:40:44.595491+0000","last_became_peered":"2024-02-26T10:40:44.595491+0000","last_unstale":"2024-02-26T10:41:01.763240+0000","last_undegraded":"2024-02-26T10:41:01.763240+0000","last_fullsized":"2024-02-26T10:41:01.763240+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:15.126480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792698+0000","last_change":"2024-02-26T10:41:01.790344+0000","last_active":"2024-02-26T10:41:02.792698+0000","last_peered":"2024-02-26T10:41:02.792698+0000","last_clean":"2024-02-26T10:41:02.792698+0000","last_became_active":"2024-02-26T10:41:01.787549+0000","last_became_peered":"2024-02-26T10:41:01.787549+0000","last_unstale":"2024-02-26T10:41:02.792698+0000","last_undegraded":"2024-02-26T10:41:02.792698+0000","last_fullsized":"2024-02-26T10:41:02.792698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:22:57.319850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763599+0000","last_change":"2024-02-26T10:41:00.777425+0000","last_active":"2024-02-26T10:41:01.763599+0000","last_peered":"2024-02-26T10:41:01.763599+0000","last_clean":"2024-02-26T10:41:01.763599+0000","last_became_active":"2024-02-26T10:41:00.777106+0000","last_became_peered":"2024-02-26T10:41:00.777106+0000","last_unstale":"2024-02-26T10:41:01.763599+0000","last_undegraded":"2024-02-26T10:41:01.763599+0000","last_fullsized":"2024-02-26T10:41:01.763599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:33.486965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794390+0000","last_change":"2024-02-26T10:40:45.647115+0000","last_active":"2024-02-26T10:41:02.794390+0000","last_peered":"2024-02-26T10:41:02.794390+0000","last_clean":"2024-02-26T10:41:02.794390+0000","last_became_active":"2024-02-26T10:40:45.637035+0000","last_became_peered":"2024-02-26T10:40:45.637035+0000","last_unstale":"2024-02-26T10:41:02.794390+0000","last_undegraded":"2024-02-26T10:41:02.794390+0000","last_fullsized":"2024-02-26T10:41:02.794390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:26.126929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794174+0000","last_change":"2024-02-26T10:40:44.620365+0000","last_active":"2024-02-26T10:41:02.794174+0000","last_peered":"2024-02-26T10:41:02.794174+0000","last_clean":"2024-02-26T10:41:02.794174+0000","last_became_active":"2024-02-26T10:40:44.619424+0000","last_became_peered":"2024-02-26T10:40:44.619424+0000","last_unstale":"2024-02-26T10:41:02.794174+0000","last_undegraded":"2024-02-26T10:41:02.794174+0000","last_fullsized":"2024-02-26T10:41:02.794174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:15.076898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793576+0000","last_change":"2024-02-26T10:41:01.794921+0000","last_active":"2024-02-26T10:41:02.793576+0000","last_peered":"2024-02-26T10:41:02.793576+0000","last_clean":"2024-02-26T10:41:02.793576+0000","last_became_active":"2024-02-26T10:41:01.794650+0000","last_became_peered":"2024-02-26T10:41:01.794650+0000","last_unstale":"2024-02-26T10:41:02.793576+0000","last_undegraded":"2024-02-26T10:41:02.793576+0000","last_fullsized":"2024-02-26T10:41:02.793576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:35.543250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792894+0000","last_change":"2024-02-26T10:41:00.786808+0000","last_active":"2024-02-26T10:41:02.792894+0000","last_peered":"2024-02-26T10:41:02.792894+0000","last_clean":"2024-02-26T10:41:02.792894+0000","last_became_active":"2024-02-26T10:41:00.784517+0000","last_became_peered":"2024-02-26T10:41:00.784517+0000","last_unstale":"2024-02-26T10:41:02.792894+0000","last_undegraded":"2024-02-26T10:41:02.792894+0000","last_fullsized":"2024-02-26T10:41:02.792894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:33.486965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793600+0000","last_change":"2024-02-26T10:40:45.633488+0000","last_active":"2024-02-26T10:41:02.793600+0000","last_peered":"2024-02-26T10:41:02.793600+0000","last_clean":"2024-02-26T10:41:02.793600+0000","last_became_active":"2024-02-26T10:40:45.633157+0000","last_became_peered":"2024-02-26T10:40:45.633157+0000","last_unstale":"2024-02-26T10:41:02.793600+0000","last_undegraded":"2024-02-26T10:41:02.793600+0000","last_fullsized":"2024-02-26T10:41:02.793600+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:33.441514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792164+0000","last_change":"2024-02-26T10:40:44.614389+0000","last_active":"2024-02-26T10:41:02.792164+0000","last_peered":"2024-02-26T10:41:02.792164+0000","last_clean":"2024-02-26T10:41:02.792164+0000","last_became_active":"2024-02-26T10:40:44.613960+0000","last_became_peered":"2024-02-26T10:40:44.613960+0000","last_unstale":"2024-02-26T10:41:02.792164+0000","last_undegraded":"2024-02-26T10:41:02.792164+0000","last_fullsized":"2024-02-26T10:41:02.792164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:09:07.764711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":242,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239191+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:41:03.239191+0000","last_peered":"2024-02-26T10:41:03.239191+0000","last_clean":"2024-02-26T10:41:03.239191+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:41:03.239191+0000","last_undegraded":"2024-02-26T10:41:03.239191+0000","last_fullsized":"2024-02-26T10:41:03.239191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3913,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":894522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":310627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":704512,"data_stored":304120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":306563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":303546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":712704,"data_stored":305046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":716800,"data_stored":305696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542179,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":897443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:41:06.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:41:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.558+0000 7fed91e00640 1 Processor -- start 2024-02-26T10:41:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.558+0000 7fed91e00640 1 -- start start 2024-02-26T10:41:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed91e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed8c0c2870 0x7fed8c155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed91e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed8c071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed91e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c155c00 con 0x7fed8c14f3d0 2024-02-26T10:41:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed91e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c06dd80 con 0x7fed8c0c2870 2024-02-26T10:41:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed91e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c155ee0 con 0x7fed8c1556a0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed8aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fed8c14f3d0 0x7fed8c06e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37438/0 (socket says 172.21.15.67:37438) 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.560+0000 7fed8aa00640 1 -- 172.21.15.67:0/397483893 learned_addr learned my addr 172.21.15.67:0/397483893 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8b400640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed8c071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099360632 0 0) 0x7fed8c155c00 con 0x7fed8c14f3d0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed83e00640 1 --2- 172.21.15.67:0/397483893 >> v2:172.21.15.82:3300/0 conn(0x7fed8c0c2870 0x7fed8c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed70003520 con 0x7fed8c14f3d0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 320488857 0 0) 0x7fed70003520 con 0x7fed8c14f3d0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 msgr2=0x7fed8c071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed8c071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 >> v2:172.21.15.82:3300/0 conn(0x7fed8c0c2870 msgr2=0x7fed8c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 --2- 172.21.15.67:0/397483893 >> v2:172.21.15.82:3300/0 conn(0x7fed8c0c2870 0x7fed8c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c06dfd0 con 0x7fed8c14f3d0 2024-02-26T10:41:06.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fed7c001670 con 0x7fed8c14f3d0 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 >> v1:172.21.15.67:6789/0 conn(0x7fed8c14f3d0 legacy=0x7fed8c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed700039b0 0x7fed70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.561+0000 7fed8a000640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed70004d40 0x7fed700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c06dd80 con 0x7fed70004d40 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c155ee0 con 0x7fed700039b0 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7000b9c0 con 0x7fed8c1556a0 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8b400640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed70004d40 0x7fed700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8aa00640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed700039b0 0x7fed70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed83e00640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8b400640 1 -- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 msgr2=0x7fed70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8b400640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8b400640 1 -- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed700039b0 msgr2=0x7fed70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:06.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8b400640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed700039b0 0x7fed70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed8b400640 1 -- 172.21.15.67:0/397483893 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70003520 con 0x7fed70004d40 2024-02-26T10:41:06.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.562+0000 7fed83e00640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c1556a0 0x7fed70004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed8b400640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed70004d40 0x7fed700051c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fed84009fc0 tx=0x7fed70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888978c5d08bcdf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84039070 con 0x7fed70004d40 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8402cc20 con 0x7fed70004d40 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed8a000640 1 -- 172.21.15.67:0/397483893 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84009c10 con 0x7fed70004d40 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed91e00640 1 -- 172.21.15.67:0/397483893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed70004d40 msgr2=0x7fed700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed91e00640 1 --2- 172.21.15.67:0/397483893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed70004d40 0x7fed700051c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fed84009fc0 tx=0x7fed70005830 comp rx=0 tx=0).stop 2024-02-26T10:41:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.564+0000 7fed8b400640 1 -- 172.21.15.67:0/397483893 reap_dead start 2024-02-26T10:41:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.565+0000 7fed91e00640 1 -- 172.21.15.67:0/397483893 shutdown_connections 2024-02-26T10:41:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.565+0000 7fed91e00640 1 -- 172.21.15.67:0/397483893 >> 172.21.15.67:0/397483893 conn(0x7fed8c0a84a0 msgr2=0x7fed8c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.565+0000 7fed91e00640 1 -- 172.21.15.67:0/397483893 shutdown_connections 2024-02-26T10:41:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.565+0000 7fed91e00640 1 -- 172.21.15.67:0/397483893 wait complete. 2024-02-26T10:41:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.566+0000 7fed91e00640 1 Processor -- start 2024-02-26T10:41:06.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 -- start start 2024-02-26T10:41:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed70004d40 0x7fed8c06ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c152780 0x7fed8c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 0x7fed8c0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0c2cb0 con 0x7fed8c14a400 2024-02-26T10:41:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0c1810 con 0x7fed70004d40 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.567+0000 7fed91e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0c2e30 con 0x7fed8c152780 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 0x7fed8c0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 0x7fed8c0c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54860/0 (socket says 172.21.15.67:54860) 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 -- 172.21.15.67:0/2656766300 learned_addr learned my addr 172.21.15.67:0/2656766300 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed83e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c152780 0x7fed8c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8aa00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed70004d40 0x7fed8c06ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 -- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c152780 msgr2=0x7fed8c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c152780 0x7fed8c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 -- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed70004d40 msgr2=0x7fed8c06ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed70004d40 0x7fed8c06ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8b400640 1 -- 172.21.15.67:0/2656766300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0c41f0 con 0x7fed8c14a400 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed83e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c152780 0x7fed8c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.568+0000 7fed8aa00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed70004d40 0x7fed8c06ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.569+0000 7fed8b400640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 0x7fed8c0c56b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fed84002230 tx=0x7fed84009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.569+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84039030 con 0x7fed8c14a400 2024-02-26T10:41:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.569+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed84043070 con 0x7fed8c14a400 2024-02-26T10:41:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.569+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84009c10 con 0x7fed8c14a400 2024-02-26T10:41:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.569+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c150b70 con 0x7fed8c14a400 2024-02-26T10:41:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.569+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c0c5190 con 0x7fed8c14a400 2024-02-26T10:41:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.570+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c074a30 con 0x7fed8c14a400 2024-02-26T10:41:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.571+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed84005d10 con 0x7fed8c14a400 2024-02-26T10:41:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.572+0000 7fed83400640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fed6806b500 0x7fed6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.572+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fed840ac720 con 0x7fed8c14a400 2024-02-26T10:41:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.573+0000 7fed8aa00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fed6806b500 0x7fed6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:06.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.574+0000 7fed8aa00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fed6806b500 0x7fed6806d9c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000f60 tx=0x7fed7c000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:06.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.577+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed8407b230 con 0x7fed8c14a400 2024-02-26T10:41:06.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.772+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed8c0c44b0 con 0x7fed6806b500 2024-02-26T10:41:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.806+0000 7fed83400640 1 -- 172.21.15.67:0/2656766300 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490511 (secure 0 0 0) 0x7fed8c0c44b0 con 0x7fed6806b500 2024-02-26T10:41:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fed6806b500 msgr2=0x7fed6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fed6806b500 0x7fed6806d9c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000f60 tx=0x7fed7c000ed0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 msgr2=0x7fed8c0c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 0x7fed8c0c56b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fed84002230 tx=0x7fed84009750 comp rx=0 tx=0).stop 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 shutdown_connections 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed8c14a400 0x7fed8c0c56b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed8c152780 0x7fed8c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fed6806b500 0x7fed6806d9c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 --2- 172.21.15.67:0/2656766300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed70004d40 0x7fed8c06ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 >> 172.21.15.67:0/2656766300 conn(0x7fed8c0a84a0 msgr2=0x7fed8c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 shutdown_connections 2024-02-26T10:41:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:06.814+0000 7fed91e00640 1 -- 172.21.15.67:0/2656766300 wait complete. 2024-02-26T10:41:06.837 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-02-26T10:41:06.380159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":335,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231196,"kb_used_data":6708,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743524,"statfs":{"total":773094113280,"available":772857368576,"internally_reserved":0,"allocated":6868992,"data_stored":3627563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002548"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.796917+0000","last_change":"2024-02-26T10:41:01.788509+0000","last_active":"2024-02-26T10:41:02.796917+0000","last_peered":"2024-02-26T10:41:02.796917+0000","last_clean":"2024-02-26T10:41:02.796917+0000","last_became_active":"2024-02-26T10:41:01.787843+0000","last_became_peered":"2024-02-26T10:41:01.787843+0000","last_unstale":"2024-02-26T10:41:02.796917+0000","last_undegraded":"2024-02-26T10:41:02.796917+0000","last_fullsized":"2024-02-26T10:41:02.796917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:44:11.908944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.772630+0000","last_change":"2024-02-26T10:40:45.640652+0000","last_active":"2024-02-26T10:41:01.772630+0000","last_peered":"2024-02-26T10:41:01.772630+0000","last_clean":"2024-02-26T10:41:01.772630+0000","last_became_active":"2024-02-26T10:40:45.637863+0000","last_became_peered":"2024-02-26T10:40:45.637863+0000","last_unstale":"2024-02-26T10:41:01.772630+0000","last_undegraded":"2024-02-26T10:41:01.772630+0000","last_fullsized":"2024-02-26T10:41:01.772630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:41:40.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.790925+0000","last_change":"2024-02-26T10:41:01.786722+0000","last_active":"2024-02-26T10:41:01.790925+0000","last_peered":"2024-02-26T10:41:01.790925+0000","last_clean":"2024-02-26T10:41:01.790925+0000","last_became_active":"2024-02-26T10:41:01.786408+0000","last_became_peered":"2024-02-26T10:41:01.786408+0000","last_unstale":"2024-02-26T10:41:01.790925+0000","last_undegraded":"2024-02-26T10:41:01.790925+0000","last_fullsized":"2024-02-26T10:41:01.790925+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:31.446787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798597+0000","last_change":"2024-02-26T10:40:45.635299+0000","last_active":"2024-02-26T10:41:02.798597+0000","last_peered":"2024-02-26T10:41:02.798597+0000","last_clean":"2024-02-26T10:41:02.798597+0000","last_became_active":"2024-02-26T10:40:45.634986+0000","last_became_peered":"2024-02-26T10:40:45.634986+0000","last_unstale":"2024-02-26T10:41:02.798597+0000","last_undegraded":"2024-02-26T10:41:02.798597+0000","last_fullsized":"2024-02-26T10:41:02.798597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:53.114125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240904+0000","last_change":"2024-02-26T10:41:01.802788+0000","last_active":"2024-02-26T10:41:03.240904+0000","last_peered":"2024-02-26T10:41:03.240904+0000","last_clean":"2024-02-26T10:41:03.240904+0000","last_became_active":"2024-02-26T10:41:01.802486+0000","last_became_peered":"2024-02-26T10:41:01.802486+0000","last_unstale":"2024-02-26T10:41:03.240904+0000","last_undegraded":"2024-02-26T10:41:03.240904+0000","last_fullsized":"2024-02-26T10:41:03.240904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:40.570174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240937+0000","last_change":"2024-02-26T10:40:45.652997+0000","last_active":"2024-02-26T10:41:03.240937+0000","last_peered":"2024-02-26T10:41:03.240937+0000","last_clean":"2024-02-26T10:41:03.240937+0000","last_became_active":"2024-02-26T10:40:45.652579+0000","last_became_peered":"2024-02-26T10:40:45.652579+0000","last_unstale":"2024-02-26T10:41:03.240937+0000","last_undegraded":"2024-02-26T10:41:03.240937+0000","last_fullsized":"2024-02-26T10:41:03.240937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:14.395806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793945+0000","last_change":"2024-02-26T10:41:01.783410+0000","last_active":"2024-02-26T10:41:02.793945+0000","last_peered":"2024-02-26T10:41:02.793945+0000","last_clean":"2024-02-26T10:41:02.793945+0000","last_became_active":"2024-02-26T10:41:01.783114+0000","last_became_peered":"2024-02-26T10:41:01.783114+0000","last_unstale":"2024-02-26T10:41:02.793945+0000","last_undegraded":"2024-02-26T10:41:02.793945+0000","last_fullsized":"2024-02-26T10:41:02.793945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:46.356779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113203+0000","last_change":"2024-02-26T10:40:45.625722+0000","last_active":"2024-02-26T10:41:03.113203+0000","last_peered":"2024-02-26T10:41:03.113203+0000","last_clean":"2024-02-26T10:41:03.113203+0000","last_became_active":"2024-02-26T10:40:45.625252+0000","last_became_peered":"2024-02-26T10:40:45.625252+0000","last_unstale":"2024-02-26T10:41:03.113203+0000","last_undegraded":"2024-02-26T10:41:03.113203+0000","last_fullsized":"2024-02-26T10:41:03.113203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112315+0000","last_change":"2024-02-26T10:41:01.792496+0000","last_active":"2024-02-26T10:41:03.112315+0000","last_peered":"2024-02-26T10:41:03.112315+0000","last_clean":"2024-02-26T10:41:03.112315+0000","last_became_active":"2024-02-26T10:41:01.792197+0000","last_became_peered":"2024-02-26T10:41:01.792197+0000","last_unstale":"2024-02-26T10:41:03.112315+0000","last_undegraded":"2024-02-26T10:41:03.112315+0000","last_fullsized":"2024-02-26T10:41:03.112315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:24.959706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.764722+0000","last_change":"2024-02-26T10:40:45.645804+0000","last_active":"2024-02-26T10:41:01.764722+0000","last_peered":"2024-02-26T10:41:01.764722+0000","last_clean":"2024-02-26T10:41:01.764722+0000","last_became_active":"2024-02-26T10:40:45.645530+0000","last_became_peered":"2024-02-26T10:40:45.645530+0000","last_unstale":"2024-02-26T10:41:01.764722+0000","last_undegraded":"2024-02-26T10:41:01.764722+0000","last_fullsized":"2024-02-26T10:41:01.764722+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:02:50.576149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.791932+0000","last_change":"2024-02-26T10:41:01.789486+0000","last_active":"2024-02-26T10:41:02.791932+0000","last_peered":"2024-02-26T10:41:02.791932+0000","last_clean":"2024-02-26T10:41:02.791932+0000","last_became_active":"2024-02-26T10:41:01.788181+0000","last_became_peered":"2024-02-26T10:41:01.788181+0000","last_unstale":"2024-02-26T10:41:02.791932+0000","last_undegraded":"2024-02-26T10:41:02.791932+0000","last_fullsized":"2024-02-26T10:41:02.791932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:49.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780774+0000","last_change":"2024-02-26T10:40:45.626504+0000","last_active":"2024-02-26T10:41:01.780774+0000","last_peered":"2024-02-26T10:41:01.780774+0000","last_clean":"2024-02-26T10:41:01.780774+0000","last_became_active":"2024-02-26T10:40:45.626190+0000","last_became_peered":"2024-02-26T10:40:45.626190+0000","last_unstale":"2024-02-26T10:41:01.780774+0000","last_undegraded":"2024-02-26T10:41:01.780774+0000","last_fullsized":"2024-02-26T10:41:01.780774+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.796455+0000","last_change":"2024-02-26T10:41:01.782509+0000","last_active":"2024-02-26T10:41:01.796455+0000","last_peered":"2024-02-26T10:41:01.796455+0000","last_clean":"2024-02-26T10:41:01.796455+0000","last_became_active":"2024-02-26T10:41:01.782223+0000","last_became_peered":"2024-02-26T10:41:01.782223+0000","last_unstale":"2024-02-26T10:41:01.796455+0000","last_undegraded":"2024-02-26T10:41:01.796455+0000","last_fullsized":"2024-02-26T10:41:01.796455+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:23:43.016082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365458+0000","last_change":"2024-02-26T10:40:45.647799+0000","last_active":"2024-02-26T10:41:03.365458+0000","last_peered":"2024-02-26T10:41:03.365458+0000","last_clean":"2024-02-26T10:41:03.365458+0000","last_became_active":"2024-02-26T10:40:45.647134+0000","last_became_peered":"2024-02-26T10:40:45.647134+0000","last_unstale":"2024-02-26T10:41:03.365458+0000","last_undegraded":"2024-02-26T10:41:03.365458+0000","last_fullsized":"2024-02-26T10:41:03.365458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:26.570068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792810+0000","last_change":"2024-02-26T10:41:01.786720+0000","last_active":"2024-02-26T10:41:02.792810+0000","last_peered":"2024-02-26T10:41:02.792810+0000","last_clean":"2024-02-26T10:41:02.792810+0000","last_became_active":"2024-02-26T10:41:01.784696+0000","last_became_peered":"2024-02-26T10:41:01.784696+0000","last_unstale":"2024-02-26T10:41:02.792810+0000","last_undegraded":"2024-02-26T10:41:02.792810+0000","last_fullsized":"2024-02-26T10:41:02.792810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:38:16.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793343+0000","last_change":"2024-02-26T10:40:45.638686+0000","last_active":"2024-02-26T10:41:02.793343+0000","last_peered":"2024-02-26T10:41:02.793343+0000","last_clean":"2024-02-26T10:41:02.793343+0000","last_became_active":"2024-02-26T10:40:45.638372+0000","last_became_peered":"2024-02-26T10:40:45.638372+0000","last_unstale":"2024-02-26T10:41:02.793343+0000","last_undegraded":"2024-02-26T10:41:02.793343+0000","last_fullsized":"2024-02-26T10:41:02.793343+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:09:51.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240236+0000","last_change":"2024-02-26T10:41:01.788651+0000","last_active":"2024-02-26T10:41:03.240236+0000","last_peered":"2024-02-26T10:41:03.240236+0000","last_clean":"2024-02-26T10:41:03.240236+0000","last_became_active":"2024-02-26T10:41:01.788044+0000","last_became_peered":"2024-02-26T10:41:01.788044+0000","last_unstale":"2024-02-26T10:41:03.240236+0000","last_undegraded":"2024-02-26T10:41:03.240236+0000","last_fullsized":"2024-02-26T10:41:03.240236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:10.745518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367135+0000","last_change":"2024-02-26T10:40:45.632997+0000","last_active":"2024-02-26T10:41:03.367135+0000","last_peered":"2024-02-26T10:41:03.367135+0000","last_clean":"2024-02-26T10:41:03.367135+0000","last_became_active":"2024-02-26T10:40:45.632665+0000","last_became_peered":"2024-02-26T10:40:45.632665+0000","last_unstale":"2024-02-26T10:41:03.367135+0000","last_undegraded":"2024-02-26T10:41:03.367135+0000","last_fullsized":"2024-02-26T10:41:03.367135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:52.054393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.786162+0000","last_change":"2024-02-26T10:41:01.777684+0000","last_active":"2024-02-26T10:41:01.786162+0000","last_peered":"2024-02-26T10:41:01.786162+0000","last_clean":"2024-02-26T10:41:01.786162+0000","last_became_active":"2024-02-26T10:41:01.777075+0000","last_became_peered":"2024-02-26T10:41:01.777075+0000","last_unstale":"2024-02-26T10:41:01.786162+0000","last_undegraded":"2024-02-26T10:41:01.786162+0000","last_fullsized":"2024-02-26T10:41:01.786162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:49.441345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.767240+0000","last_change":"2024-02-26T10:40:45.636607+0000","last_active":"2024-02-26T10:41:01.767240+0000","last_peered":"2024-02-26T10:41:01.767240+0000","last_clean":"2024-02-26T10:41:01.767240+0000","last_became_active":"2024-02-26T10:40:45.636328+0000","last_became_peered":"2024-02-26T10:40:45.636328+0000","last_unstale":"2024-02-26T10:41:01.767240+0000","last_undegraded":"2024-02-26T10:41:01.767240+0000","last_fullsized":"2024-02-26T10:41:01.767240+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:47:26.570068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799366+0000","last_change":"2024-02-26T10:41:01.791725+0000","last_active":"2024-02-26T10:41:01.799366+0000","last_peered":"2024-02-26T10:41:01.799366+0000","last_clean":"2024-02-26T10:41:01.799366+0000","last_became_active":"2024-02-26T10:41:01.780515+0000","last_became_peered":"2024-02-26T10:41:01.780515+0000","last_unstale":"2024-02-26T10:41:01.799366+0000","last_undegraded":"2024-02-26T10:41:01.799366+0000","last_fullsized":"2024-02-26T10:41:01.799366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:17:39.047299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.765243+0000","last_change":"2024-02-26T10:40:45.628536+0000","last_active":"2024-02-26T10:41:01.765243+0000","last_peered":"2024-02-26T10:41:01.765243+0000","last_clean":"2024-02-26T10:41:01.765243+0000","last_became_active":"2024-02-26T10:40:45.628287+0000","last_became_peered":"2024-02-26T10:40:45.628287+0000","last_unstale":"2024-02-26T10:41:01.765243+0000","last_undegraded":"2024-02-26T10:41:01.765243+0000","last_fullsized":"2024-02-26T10:41:01.765243+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:35.142103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367248+0000","last_change":"2024-02-26T10:41:01.782542+0000","last_active":"2024-02-26T10:41:03.367248+0000","last_peered":"2024-02-26T10:41:03.367248+0000","last_clean":"2024-02-26T10:41:03.367248+0000","last_became_active":"2024-02-26T10:41:01.782129+0000","last_became_peered":"2024-02-26T10:41:01.782129+0000","last_unstale":"2024-02-26T10:41:03.367248+0000","last_undegraded":"2024-02-26T10:41:03.367248+0000","last_fullsized":"2024-02-26T10:41:03.367248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:26:04.968703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113390+0000","last_change":"2024-02-26T10:40:45.644900+0000","last_active":"2024-02-26T10:41:03.113390+0000","last_peered":"2024-02-26T10:41:03.113390+0000","last_clean":"2024-02-26T10:41:03.113390+0000","last_became_active":"2024-02-26T10:40:45.644626+0000","last_became_peered":"2024-02-26T10:40:45.644626+0000","last_unstale":"2024-02-26T10:41:03.113390+0000","last_undegraded":"2024-02-26T10:41:03.113390+0000","last_fullsized":"2024-02-26T10:41:03.113390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.803350+0000","last_change":"2024-02-26T10:41:01.801985+0000","last_active":"2024-02-26T10:41:01.803350+0000","last_peered":"2024-02-26T10:41:01.803350+0000","last_clean":"2024-02-26T10:41:01.803350+0000","last_became_active":"2024-02-26T10:41:01.801187+0000","last_became_peered":"2024-02-26T10:41:01.801187+0000","last_unstale":"2024-02-26T10:41:01.803350+0000","last_undegraded":"2024-02-26T10:41:01.803350+0000","last_fullsized":"2024-02-26T10:41:01.803350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:10:07.551025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793188+0000","last_change":"2024-02-26T10:40:45.641831+0000","last_active":"2024-02-26T10:41:02.793188+0000","last_peered":"2024-02-26T10:41:02.793188+0000","last_clean":"2024-02-26T10:41:02.793188+0000","last_became_active":"2024-02-26T10:40:45.641331+0000","last_became_peered":"2024-02-26T10:40:45.641331+0000","last_unstale":"2024-02-26T10:41:02.793188+0000","last_undegraded":"2024-02-26T10:41:02.793188+0000","last_fullsized":"2024-02-26T10:41:02.793188+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:18:27.192800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365357+0000","last_change":"2024-02-26T10:41:01.783279+0000","last_active":"2024-02-26T10:41:03.365357+0000","last_peered":"2024-02-26T10:41:03.365357+0000","last_clean":"2024-02-26T10:41:03.365357+0000","last_became_active":"2024-02-26T10:41:01.782894+0000","last_became_peered":"2024-02-26T10:41:01.782894+0000","last_unstale":"2024-02-26T10:41:03.365357+0000","last_undegraded":"2024-02-26T10:41:03.365357+0000","last_fullsized":"2024-02-26T10:41:03.365357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:32.246950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365727+0000","last_change":"2024-02-26T10:40:45.645455+0000","last_active":"2024-02-26T10:41:03.365727+0000","last_peered":"2024-02-26T10:41:03.365727+0000","last_clean":"2024-02-26T10:41:03.365727+0000","last_became_active":"2024-02-26T10:40:45.645142+0000","last_became_peered":"2024-02-26T10:40:45.645142+0000","last_unstale":"2024-02-26T10:41:03.365727+0000","last_undegraded":"2024-02-26T10:41:03.365727+0000","last_fullsized":"2024-02-26T10:41:03.365727+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:03.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.802670+0000","last_change":"2024-02-26T10:41:01.798725+0000","last_active":"2024-02-26T10:41:01.802670+0000","last_peered":"2024-02-26T10:41:01.802670+0000","last_clean":"2024-02-26T10:41:01.802670+0000","last_became_active":"2024-02-26T10:41:01.797853+0000","last_became_peered":"2024-02-26T10:41:01.797853+0000","last_unstale":"2024-02-26T10:41:01.802670+0000","last_undegraded":"2024-02-26T10:41:01.802670+0000","last_fullsized":"2024-02-26T10:41:01.802670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:12:00.902313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365532+0000","last_change":"2024-02-26T10:40:45.648027+0000","last_active":"2024-02-26T10:41:03.365532+0000","last_peered":"2024-02-26T10:41:03.365532+0000","last_clean":"2024-02-26T10:41:03.365532+0000","last_became_active":"2024-02-26T10:40:45.647523+0000","last_became_peered":"2024-02-26T10:40:45.647523+0000","last_unstale":"2024-02-26T10:41:03.365532+0000","last_undegraded":"2024-02-26T10:41:03.365532+0000","last_fullsized":"2024-02-26T10:41:03.365532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:33.097875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.785315+0000","last_change":"2024-02-26T10:41:01.774898+0000","last_active":"2024-02-26T10:41:01.785315+0000","last_peered":"2024-02-26T10:41:01.785315+0000","last_clean":"2024-02-26T10:41:01.785315+0000","last_became_active":"2024-02-26T10:41:01.774608+0000","last_became_peered":"2024-02-26T10:41:01.774608+0000","last_unstale":"2024-02-26T10:41:01.785315+0000","last_undegraded":"2024-02-26T10:41:01.785315+0000","last_fullsized":"2024-02-26T10:41:01.785315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:19:27.207753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.773425+0000","last_change":"2024-02-26T10:40:45.639856+0000","last_active":"2024-02-26T10:41:01.773425+0000","last_peered":"2024-02-26T10:41:01.773425+0000","last_clean":"2024-02-26T10:41:01.773425+0000","last_became_active":"2024-02-26T10:40:45.639346+0000","last_became_peered":"2024-02-26T10:40:45.639346+0000","last_unstale":"2024-02-26T10:41:01.773425+0000","last_undegraded":"2024-02-26T10:41:01.773425+0000","last_fullsized":"2024-02-26T10:41:01.773425+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:59.659271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792233+0000","last_change":"2024-02-26T10:41:01.787816+0000","last_active":"2024-02-26T10:41:02.792233+0000","last_peered":"2024-02-26T10:41:02.792233+0000","last_clean":"2024-02-26T10:41:02.792233+0000","last_became_active":"2024-02-26T10:41:01.787431+0000","last_became_peered":"2024-02-26T10:41:01.787431+0000","last_unstale":"2024-02-26T10:41:02.792233+0000","last_undegraded":"2024-02-26T10:41:02.792233+0000","last_fullsized":"2024-02-26T10:41:02.792233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:26:04.968703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.799110+0000","last_change":"2024-02-26T10:40:45.639209+0000","last_active":"2024-02-26T10:41:02.799110+0000","last_peered":"2024-02-26T10:41:02.799110+0000","last_clean":"2024-02-26T10:41:02.799110+0000","last_became_active":"2024-02-26T10:40:45.638129+0000","last_became_peered":"2024-02-26T10:40:45.638129+0000","last_unstale":"2024-02-26T10:41:02.799110+0000","last_undegraded":"2024-02-26T10:41:02.799110+0000","last_fullsized":"2024-02-26T10:41:02.799110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:08.794199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.800176+0000","last_change":"2024-02-26T10:41:01.797205+0000","last_active":"2024-02-26T10:41:01.800176+0000","last_peered":"2024-02-26T10:41:01.800176+0000","last_clean":"2024-02-26T10:41:01.800176+0000","last_became_active":"2024-02-26T10:41:01.796937+0000","last_became_peered":"2024-02-26T10:41:01.796937+0000","last_unstale":"2024-02-26T10:41:01.800176+0000","last_undegraded":"2024-02-26T10:41:01.800176+0000","last_fullsized":"2024-02-26T10:41:01.800176+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:51.649294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241170+0000","last_change":"2024-02-26T10:40:45.649827+0000","last_active":"2024-02-26T10:41:03.241170+0000","last_peered":"2024-02-26T10:41:03.241170+0000","last_clean":"2024-02-26T10:41:03.241170+0000","last_became_active":"2024-02-26T10:40:45.649273+0000","last_became_peered":"2024-02-26T10:40:45.649273+0000","last_unstale":"2024-02-26T10:41:03.241170+0000","last_undegraded":"2024-02-26T10:41:03.241170+0000","last_fullsized":"2024-02-26T10:41:03.241170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:33.671100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799345+0000","last_change":"2024-02-26T10:41:01.791632+0000","last_active":"2024-02-26T10:41:01.799345+0000","last_peered":"2024-02-26T10:41:01.799345+0000","last_clean":"2024-02-26T10:41:01.799345+0000","last_became_active":"2024-02-26T10:41:01.780138+0000","last_became_peered":"2024-02-26T10:41:01.780138+0000","last_unstale":"2024-02-26T10:41:01.799345+0000","last_undegraded":"2024-02-26T10:41:01.799345+0000","last_fullsized":"2024-02-26T10:41:01.799345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:15:23.135572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.770672+0000","last_change":"2024-02-26T10:40:45.626981+0000","last_active":"2024-02-26T10:41:01.770672+0000","last_peered":"2024-02-26T10:41:01.770672+0000","last_clean":"2024-02-26T10:41:01.770672+0000","last_became_active":"2024-02-26T10:40:45.626693+0000","last_became_peered":"2024-02-26T10:40:45.626693+0000","last_unstale":"2024-02-26T10:41:01.770672+0000","last_undegraded":"2024-02-26T10:41:01.770672+0000","last_fullsized":"2024-02-26T10:41:01.770672+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:43:37.225724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.789422+0000","last_change":"2024-02-26T10:41:01.784430+0000","last_active":"2024-02-26T10:41:01.789422+0000","last_peered":"2024-02-26T10:41:01.789422+0000","last_clean":"2024-02-26T10:41:01.789422+0000","last_became_active":"2024-02-26T10:41:01.784160+0000","last_became_peered":"2024-02-26T10:41:01.784160+0000","last_unstale":"2024-02-26T10:41:01.789422+0000","last_undegraded":"2024-02-26T10:41:01.789422+0000","last_fullsized":"2024-02-26T10:41:01.789422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:52:47.004046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795288+0000","last_change":"2024-02-26T10:40:45.634499+0000","last_active":"2024-02-26T10:41:02.795288+0000","last_peered":"2024-02-26T10:41:02.795288+0000","last_clean":"2024-02-26T10:41:02.795288+0000","last_became_active":"2024-02-26T10:40:45.633868+0000","last_became_peered":"2024-02-26T10:40:45.633868+0000","last_unstale":"2024-02-26T10:41:02.795288+0000","last_undegraded":"2024-02-26T10:41:02.795288+0000","last_fullsized":"2024-02-26T10:41:02.795288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:18:54.827479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239261+0000","last_change":"2024-02-26T10:41:01.796543+0000","last_active":"2024-02-26T10:41:03.239261+0000","last_peered":"2024-02-26T10:41:03.239261+0000","last_clean":"2024-02-26T10:41:03.239261+0000","last_became_active":"2024-02-26T10:41:01.796229+0000","last_became_peered":"2024-02-26T10:41:01.796229+0000","last_unstale":"2024-02-26T10:41:03.239261+0000","last_undegraded":"2024-02-26T10:41:03.239261+0000","last_fullsized":"2024-02-26T10:41:03.239261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:22.919759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240001+0000","last_change":"2024-02-26T10:40:45.650672+0000","last_active":"2024-02-26T10:41:03.240001+0000","last_peered":"2024-02-26T10:41:03.240001+0000","last_clean":"2024-02-26T10:41:03.240001+0000","last_became_active":"2024-02-26T10:40:45.649874+0000","last_became_peered":"2024-02-26T10:40:45.649874+0000","last_unstale":"2024-02-26T10:41:03.240001+0000","last_undegraded":"2024-02-26T10:41:03.240001+0000","last_fullsized":"2024-02-26T10:41:03.240001+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239041+0000","last_change":"2024-02-26T10:41:01.795687+0000","last_active":"2024-02-26T10:41:03.239041+0000","last_peered":"2024-02-26T10:41:03.239041+0000","last_clean":"2024-02-26T10:41:03.239041+0000","last_became_active":"2024-02-26T10:41:01.792930+0000","last_became_peered":"2024-02-26T10:41:01.792930+0000","last_unstale":"2024-02-26T10:41:03.239041+0000","last_undegraded":"2024-02-26T10:41:03.239041+0000","last_fullsized":"2024-02-26T10:41:03.239041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:58.363793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793899+0000","last_change":"2024-02-26T10:40:45.636991+0000","last_active":"2024-02-26T10:41:02.793899+0000","last_peered":"2024-02-26T10:41:02.793899+0000","last_clean":"2024-02-26T10:41:02.793899+0000","last_became_active":"2024-02-26T10:40:45.634121+0000","last_became_peered":"2024-02-26T10:40:45.634121+0000","last_unstale":"2024-02-26T10:41:02.793899+0000","last_undegraded":"2024-02-26T10:41:02.793899+0000","last_fullsized":"2024-02-26T10:41:02.793899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:23:35.604849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366669+0000","last_change":"2024-02-26T10:41:01.790805+0000","last_active":"2024-02-26T10:41:03.366669+0000","last_peered":"2024-02-26T10:41:03.366669+0000","last_clean":"2024-02-26T10:41:03.366669+0000","last_became_active":"2024-02-26T10:41:01.790540+0000","last_became_peered":"2024-02-26T10:41:01.790540+0000","last_unstale":"2024-02-26T10:41:03.366669+0000","last_undegraded":"2024-02-26T10:41:03.366669+0000","last_fullsized":"2024-02-26T10:41:03.366669+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:50.908155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239446+0000","last_change":"2024-02-26T10:40:45.642494+0000","last_active":"2024-02-26T10:41:03.239446+0000","last_peered":"2024-02-26T10:41:03.239446+0000","last_clean":"2024-02-26T10:41:03.239446+0000","last_became_active":"2024-02-26T10:40:45.641655+0000","last_became_peered":"2024-02-26T10:40:45.641655+0000","last_unstale":"2024-02-26T10:41:03.239446+0000","last_undegraded":"2024-02-26T10:41:03.239446+0000","last_fullsized":"2024-02-26T10:41:03.239446+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:15.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366026+0000","last_change":"2024-02-26T10:41:01.781103+0000","last_active":"2024-02-26T10:41:03.366026+0000","last_peered":"2024-02-26T10:41:03.366026+0000","last_clean":"2024-02-26T10:41:03.366026+0000","last_became_active":"2024-02-26T10:41:01.780775+0000","last_became_peered":"2024-02-26T10:41:01.780775+0000","last_unstale":"2024-02-26T10:41:03.366026+0000","last_undegraded":"2024-02-26T10:41:03.366026+0000","last_fullsized":"2024-02-26T10:41:03.366026+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:48.562416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792389+0000","last_change":"2024-02-26T10:40:45.646927+0000","last_active":"2024-02-26T10:41:02.792389+0000","last_peered":"2024-02-26T10:41:02.792389+0000","last_clean":"2024-02-26T10:41:02.792389+0000","last_became_active":"2024-02-26T10:40:45.646598+0000","last_became_peered":"2024-02-26T10:40:45.646598+0000","last_unstale":"2024-02-26T10:41:02.792389+0000","last_undegraded":"2024-02-26T10:41:02.792389+0000","last_fullsized":"2024-02-26T10:41:02.792389+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:04:02.887116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.797350+0000","last_change":"2024-02-26T10:41:01.795359+0000","last_active":"2024-02-26T10:41:02.797350+0000","last_peered":"2024-02-26T10:41:02.797350+0000","last_clean":"2024-02-26T10:41:02.797350+0000","last_became_active":"2024-02-26T10:41:01.794707+0000","last_became_peered":"2024-02-26T10:41:01.794707+0000","last_unstale":"2024-02-26T10:41:02.797350+0000","last_undegraded":"2024-02-26T10:41:02.797350+0000","last_fullsized":"2024-02-26T10:41:02.797350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:26:56.076176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.772821+0000","last_change":"2024-02-26T10:40:45.637507+0000","last_active":"2024-02-26T10:41:01.772821+0000","last_peered":"2024-02-26T10:41:01.772821+0000","last_clean":"2024-02-26T10:41:01.772821+0000","last_became_active":"2024-02-26T10:40:45.631723+0000","last_became_peered":"2024-02-26T10:40:45.631723+0000","last_unstale":"2024-02-26T10:41:01.772821+0000","last_undegraded":"2024-02-26T10:41:01.772821+0000","last_fullsized":"2024-02-26T10:41:01.772821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:14:07.605390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366854+0000","last_change":"2024-02-26T10:41:01.778438+0000","last_active":"2024-02-26T10:41:03.366854+0000","last_peered":"2024-02-26T10:41:03.366854+0000","last_clean":"2024-02-26T10:41:03.366854+0000","last_became_active":"2024-02-26T10:41:01.778150+0000","last_became_peered":"2024-02-26T10:41:01.778150+0000","last_unstale":"2024-02-26T10:41:03.366854+0000","last_undegraded":"2024-02-26T10:41:03.366854+0000","last_fullsized":"2024-02-26T10:41:03.366854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:30.561314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114055+0000","last_change":"2024-02-26T10:40:45.633049+0000","last_active":"2024-02-26T10:41:03.114055+0000","last_peered":"2024-02-26T10:41:03.114055+0000","last_clean":"2024-02-26T10:41:03.114055+0000","last_became_active":"2024-02-26T10:40:45.632003+0000","last_became_peered":"2024-02-26T10:40:45.632003+0000","last_unstale":"2024-02-26T10:41:03.114055+0000","last_undegraded":"2024-02-26T10:41:03.114055+0000","last_fullsized":"2024-02-26T10:41:03.114055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366992+0000","last_change":"2024-02-26T10:41:01.791451+0000","last_active":"2024-02-26T10:41:03.366992+0000","last_peered":"2024-02-26T10:41:03.366992+0000","last_clean":"2024-02-26T10:41:03.366992+0000","last_became_active":"2024-02-26T10:41:01.791147+0000","last_became_peered":"2024-02-26T10:41:01.791147+0000","last_unstale":"2024-02-26T10:41:03.366992+0000","last_undegraded":"2024-02-26T10:41:03.366992+0000","last_fullsized":"2024-02-26T10:41:03.366992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:40:49.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781015+0000","last_change":"2024-02-26T10:40:45.639683+0000","last_active":"2024-02-26T10:41:01.781015+0000","last_peered":"2024-02-26T10:41:01.781015+0000","last_clean":"2024-02-26T10:41:01.781015+0000","last_became_active":"2024-02-26T10:40:45.639388+0000","last_became_peered":"2024-02-26T10:40:45.639388+0000","last_unstale":"2024-02-26T10:41:01.781015+0000","last_undegraded":"2024-02-26T10:41:01.781015+0000","last_fullsized":"2024-02-26T10:41:01.781015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113853+0000","last_change":"2024-02-26T10:41:01.774167+0000","last_active":"2024-02-26T10:41:03.113853+0000","last_peered":"2024-02-26T10:41:03.113853+0000","last_clean":"2024-02-26T10:41:03.113853+0000","last_became_active":"2024-02-26T10:41:01.773876+0000","last_became_peered":"2024-02-26T10:41:01.773876+0000","last_unstale":"2024-02-26T10:41:03.113853+0000","last_undegraded":"2024-02-26T10:41:03.113853+0000","last_fullsized":"2024-02-26T10:41:03.113853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:25.572381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.770960+0000","last_change":"2024-02-26T10:40:45.632119+0000","last_active":"2024-02-26T10:41:01.770960+0000","last_peered":"2024-02-26T10:41:01.770960+0000","last_clean":"2024-02-26T10:41:01.770960+0000","last_became_active":"2024-02-26T10:40:45.631803+0000","last_became_peered":"2024-02-26T10:40:45.631803+0000","last_unstale":"2024-02-26T10:41:01.770960+0000","last_undegraded":"2024-02-26T10:41:01.770960+0000","last_fullsized":"2024-02-26T10:41:01.770960+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:42.673290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112544+0000","last_change":"2024-02-26T10:41:01.780670+0000","last_active":"2024-02-26T10:41:03.112544+0000","last_peered":"2024-02-26T10:41:03.112544+0000","last_clean":"2024-02-26T10:41:03.112544+0000","last_became_active":"2024-02-26T10:41:01.780364+0000","last_became_peered":"2024-02-26T10:41:01.780364+0000","last_unstale":"2024-02-26T10:41:03.112544+0000","last_undegraded":"2024-02-26T10:41:03.112544+0000","last_fullsized":"2024-02-26T10:41:03.112544+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:17:09.279632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795972+0000","last_change":"2024-02-26T10:40:45.636100+0000","last_active":"2024-02-26T10:41:02.795972+0000","last_peered":"2024-02-26T10:41:02.795972+0000","last_clean":"2024-02-26T10:41:02.795972+0000","last_became_active":"2024-02-26T10:40:45.635761+0000","last_became_peered":"2024-02-26T10:40:45.635761+0000","last_unstale":"2024-02-26T10:41:02.795972+0000","last_undegraded":"2024-02-26T10:41:02.795972+0000","last_fullsized":"2024-02-26T10:41:02.795972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:42.198286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793536+0000","last_change":"2024-02-26T10:41:01.787501+0000","last_active":"2024-02-26T10:41:02.793536+0000","last_peered":"2024-02-26T10:41:02.793536+0000","last_clean":"2024-02-26T10:41:02.793536+0000","last_became_active":"2024-02-26T10:41:01.787201+0000","last_became_peered":"2024-02-26T10:41:01.787201+0000","last_unstale":"2024-02-26T10:41:02.793536+0000","last_undegraded":"2024-02-26T10:41:02.793536+0000","last_fullsized":"2024-02-26T10:41:02.793536+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:28.417034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239298+0000","last_change":"2024-02-26T10:40:45.649382+0000","last_active":"2024-02-26T10:41:03.239298+0000","last_peered":"2024-02-26T10:41:03.239298+0000","last_clean":"2024-02-26T10:41:03.239298+0000","last_became_active":"2024-02-26T10:40:45.649076+0000","last_became_peered":"2024-02-26T10:40:45.649076+0000","last_unstale":"2024-02-26T10:41:03.239298+0000","last_undegraded":"2024-02-26T10:41:03.239298+0000","last_fullsized":"2024-02-26T10:41:03.239298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795988+0000","last_change":"2024-02-26T10:41:01.791971+0000","last_active":"2024-02-26T10:41:02.795988+0000","last_peered":"2024-02-26T10:41:02.795988+0000","last_clean":"2024-02-26T10:41:02.795988+0000","last_became_active":"2024-02-26T10:41:01.790980+0000","last_became_peered":"2024-02-26T10:41:01.790980+0000","last_unstale":"2024-02-26T10:41:02.795988+0000","last_undegraded":"2024-02-26T10:41:02.795988+0000","last_fullsized":"2024-02-26T10:41:02.795988+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:28:26.576969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763732+0000","last_change":"2024-02-26T10:40:45.625890+0000","last_active":"2024-02-26T10:41:01.763732+0000","last_peered":"2024-02-26T10:41:01.763732+0000","last_clean":"2024-02-26T10:41:01.763732+0000","last_became_active":"2024-02-26T10:40:45.625334+0000","last_became_peered":"2024-02-26T10:40:45.625334+0000","last_unstale":"2024-02-26T10:41:01.763732+0000","last_undegraded":"2024-02-26T10:41:01.763732+0000","last_fullsized":"2024-02-26T10:41:01.763732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:38:29.332505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112585+0000","last_change":"2024-02-26T10:41:01.778110+0000","last_active":"2024-02-26T10:41:03.112585+0000","last_peered":"2024-02-26T10:41:03.112585+0000","last_clean":"2024-02-26T10:41:03.112585+0000","last_became_active":"2024-02-26T10:41:01.777811+0000","last_became_peered":"2024-02-26T10:41:01.777811+0000","last_unstale":"2024-02-26T10:41:03.112585+0000","last_undegraded":"2024-02-26T10:41:03.112585+0000","last_fullsized":"2024-02-26T10:41:03.112585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:19:10.992986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.773748+0000","last_change":"2024-02-26T10:40:45.626985+0000","last_active":"2024-02-26T10:41:01.773748+0000","last_peered":"2024-02-26T10:41:01.773748+0000","last_clean":"2024-02-26T10:41:01.773748+0000","last_became_active":"2024-02-26T10:40:45.626657+0000","last_became_peered":"2024-02-26T10:40:45.626657+0000","last_unstale":"2024-02-26T10:41:01.773748+0000","last_undegraded":"2024-02-26T10:41:01.773748+0000","last_fullsized":"2024-02-26T10:41:01.773748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:15.589627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793019+0000","last_change":"2024-02-26T10:41:01.795830+0000","last_active":"2024-02-26T10:41:02.793019+0000","last_peered":"2024-02-26T10:41:02.793019+0000","last_clean":"2024-02-26T10:41:02.793019+0000","last_became_active":"2024-02-26T10:41:01.795529+0000","last_became_peered":"2024-02-26T10:41:01.795529+0000","last_unstale":"2024-02-26T10:41:02.793019+0000","last_undegraded":"2024-02-26T10:41:02.793019+0000","last_fullsized":"2024-02-26T10:41:02.793019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:35:23.235157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366718+0000","last_change":"2024-02-26T10:41:00.794647+0000","last_active":"2024-02-26T10:41:03.366718+0000","last_peered":"2024-02-26T10:41:03.366718+0000","last_clean":"2024-02-26T10:41:03.366718+0000","last_became_active":"2024-02-26T10:41:00.793948+0000","last_became_peered":"2024-02-26T10:41:00.793948+0000","last_unstale":"2024-02-26T10:41:03.366718+0000","last_undegraded":"2024-02-26T10:41:03.366718+0000","last_fullsized":"2024-02-26T10:41:03.366718+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:15.501040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.782937+0000","last_change":"2024-02-26T10:40:45.643239+0000","last_active":"2024-02-26T10:41:01.782937+0000","last_peered":"2024-02-26T10:41:01.782937+0000","last_clean":"2024-02-26T10:41:01.782937+0000","last_became_active":"2024-02-26T10:40:45.642924+0000","last_became_peered":"2024-02-26T10:40:45.642924+0000","last_unstale":"2024-02-26T10:41:01.782937+0000","last_undegraded":"2024-02-26T10:41:01.782937+0000","last_fullsized":"2024-02-26T10:41:01.782937+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":24,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.783336+0000","last_change":"2024-02-26T10:40:44.610146+0000","last_active":"2024-02-26T10:41:01.783336+0000","last_peered":"2024-02-26T10:41:01.783336+0000","last_clean":"2024-02-26T10:41:01.783336+0000","last_became_active":"2024-02-26T10:40:44.609886+0000","last_became_peered":"2024-02-26T10:40:44.609886+0000","last_unstale":"2024-02-26T10:41:01.783336+0000","last_undegraded":"2024-02-26T10:41:01.783336+0000","last_fullsized":"2024-02-26T10:41:01.783336+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:55.475057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240693+0000","last_change":"2024-02-26T10:41:01.788873+0000","last_active":"2024-02-26T10:41:03.240693+0000","last_peered":"2024-02-26T10:41:03.240693+0000","last_clean":"2024-02-26T10:41:03.240693+0000","last_became_active":"2024-02-26T10:41:01.788368+0000","last_became_peered":"2024-02-26T10:41:01.788368+0000","last_unstale":"2024-02-26T10:41:03.240693+0000","last_undegraded":"2024-02-26T10:41:03.240693+0000","last_fullsized":"2024-02-26T10:41:03.240693+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:17:24.101378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793325+0000","last_change":"2024-02-26T10:41:00.793345+0000","last_active":"2024-02-26T10:41:02.793325+0000","last_peered":"2024-02-26T10:41:02.793325+0000","last_clean":"2024-02-26T10:41:02.793325+0000","last_became_active":"2024-02-26T10:41:00.792909+0000","last_became_peered":"2024-02-26T10:41:00.792909+0000","last_unstale":"2024-02-26T10:41:02.793325+0000","last_undegraded":"2024-02-26T10:41:02.793325+0000","last_fullsized":"2024-02-26T10:41:02.793325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:21.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.769335+0000","last_change":"2024-02-26T10:40:45.645160+0000","last_active":"2024-02-26T10:41:01.769335+0000","last_peered":"2024-02-26T10:41:01.769335+0000","last_clean":"2024-02-26T10:41:01.769335+0000","last_became_active":"2024-02-26T10:40:45.644902+0000","last_became_peered":"2024-02-26T10:40:45.644902+0000","last_unstale":"2024-02-26T10:41:01.769335+0000","last_undegraded":"2024-02-26T10:41:01.769335+0000","last_fullsized":"2024-02-26T10:41:01.769335+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:53:58.631698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798774+0000","last_change":"2024-02-26T10:40:44.617860+0000","last_active":"2024-02-26T10:41:02.798774+0000","last_peered":"2024-02-26T10:41:02.798774+0000","last_clean":"2024-02-26T10:41:02.798774+0000","last_became_active":"2024-02-26T10:40:44.617592+0000","last_became_peered":"2024-02-26T10:40:44.617592+0000","last_unstale":"2024-02-26T10:41:02.798774+0000","last_undegraded":"2024-02-26T10:41:02.798774+0000","last_fullsized":"2024-02-26T10:41:02.798774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:16.970692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241028+0000","last_change":"2024-02-26T10:41:01.798892+0000","last_active":"2024-02-26T10:41:03.241028+0000","last_peered":"2024-02-26T10:41:03.241028+0000","last_clean":"2024-02-26T10:41:03.241028+0000","last_became_active":"2024-02-26T10:41:01.798614+0000","last_became_peered":"2024-02-26T10:41:01.798614+0000","last_unstale":"2024-02-26T10:41:03.241028+0000","last_undegraded":"2024-02-26T10:41:03.241028+0000","last_fullsized":"2024-02-26T10:41:03.241028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:13.201328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241213+0000","last_change":"2024-02-26T10:41:00.795196+0000","last_active":"2024-02-26T10:41:03.241213+0000","last_peered":"2024-02-26T10:41:03.241213+0000","last_clean":"2024-02-26T10:41:03.241213+0000","last_became_active":"2024-02-26T10:41:00.794870+0000","last_became_peered":"2024-02-26T10:41:00.794870+0000","last_unstale":"2024-02-26T10:41:03.241213+0000","last_undegraded":"2024-02-26T10:41:03.241213+0000","last_fullsized":"2024-02-26T10:41:03.241213+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:16.987475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780415+0000","last_change":"2024-02-26T10:40:45.621813+0000","last_active":"2024-02-26T10:41:01.780415+0000","last_peered":"2024-02-26T10:41:01.780415+0000","last_clean":"2024-02-26T10:41:01.780415+0000","last_became_active":"2024-02-26T10:40:45.621532+0000","last_became_peered":"2024-02-26T10:40:45.621532+0000","last_unstale":"2024-02-26T10:41:01.780415+0000","last_undegraded":"2024-02-26T10:41:01.780415+0000","last_fullsized":"2024-02-26T10:41:01.780415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.784527+0000","last_change":"2024-02-26T10:40:44.594679+0000","last_active":"2024-02-26T10:41:01.784527+0000","last_peered":"2024-02-26T10:41:01.784527+0000","last_clean":"2024-02-26T10:41:01.784527+0000","last_became_active":"2024-02-26T10:40:44.593817+0000","last_became_peered":"2024-02-26T10:40:44.593817+0000","last_unstale":"2024-02-26T10:41:01.784527+0000","last_undegraded":"2024-02-26T10:41:01.784527+0000","last_fullsized":"2024-02-26T10:41:01.784527+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:47.713723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799302+0000","last_change":"2024-02-26T10:41:01.786081+0000","last_active":"2024-02-26T10:41:01.799302+0000","last_peered":"2024-02-26T10:41:01.799302+0000","last_clean":"2024-02-26T10:41:01.799302+0000","last_became_active":"2024-02-26T10:41:01.779677+0000","last_became_peered":"2024-02-26T10:41:01.779677+0000","last_unstale":"2024-02-26T10:41:01.799302+0000","last_undegraded":"2024-02-26T10:41:01.799302+0000","last_fullsized":"2024-02-26T10:41:01.799302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:53:59.497223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.800337+0000","last_change":"2024-02-26T10:41:00.779600+0000","last_active":"2024-02-26T10:41:01.800337+0000","last_peered":"2024-02-26T10:41:01.800337+0000","last_clean":"2024-02-26T10:41:01.800337+0000","last_became_active":"2024-02-26T10:41:00.778132+0000","last_became_peered":"2024-02-26T10:41:00.778132+0000","last_unstale":"2024-02-26T10:41:01.800337+0000","last_undegraded":"2024-02-26T10:41:01.800337+0000","last_fullsized":"2024-02-26T10:41:01.800337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:16.987475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781144+0000","last_change":"2024-02-26T10:40:45.630558+0000","last_active":"2024-02-26T10:41:01.781144+0000","last_peered":"2024-02-26T10:41:01.781144+0000","last_clean":"2024-02-26T10:41:01.781144+0000","last_became_active":"2024-02-26T10:40:45.630319+0000","last_became_peered":"2024-02-26T10:40:45.630319+0000","last_unstale":"2024-02-26T10:41:01.781144+0000","last_undegraded":"2024-02-26T10:41:01.781144+0000","last_fullsized":"2024-02-26T10:41:01.781144+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":24,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113500+0000","last_change":"2024-02-26T10:40:44.616740+0000","last_active":"2024-02-26T10:41:03.113500+0000","last_peered":"2024-02-26T10:41:03.113500+0000","last_clean":"2024-02-26T10:41:03.113500+0000","last_became_active":"2024-02-26T10:40:44.616416+0000","last_became_peered":"2024-02-26T10:40:44.616416+0000","last_unstale":"2024-02-26T10:41:03.113500+0000","last_undegraded":"2024-02-26T10:41:03.113500+0000","last_fullsized":"2024-02-26T10:41:03.113500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:06:58.112139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.791976+0000","last_change":"2024-02-26T10:41:01.779451+0000","last_active":"2024-02-26T10:41:01.791976+0000","last_peered":"2024-02-26T10:41:01.791976+0000","last_clean":"2024-02-26T10:41:01.791976+0000","last_became_active":"2024-02-26T10:41:01.779125+0000","last_became_peered":"2024-02-26T10:41:01.779125+0000","last_unstale":"2024-02-26T10:41:01.791976+0000","last_undegraded":"2024-02-26T10:41:01.791976+0000","last_fullsized":"2024-02-26T10:41:01.791976+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:09:55.280668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.762926+0000","last_change":"2024-02-26T10:41:00.771434+0000","last_active":"2024-02-26T10:41:01.762926+0000","last_peered":"2024-02-26T10:41:01.762926+0000","last_clean":"2024-02-26T10:41:01.762926+0000","last_became_active":"2024-02-26T10:41:00.770953+0000","last_became_peered":"2024-02-26T10:41:00.770953+0000","last_unstale":"2024-02-26T10:41:01.762926+0000","last_undegraded":"2024-02-26T10:41:01.762926+0000","last_fullsized":"2024-02-26T10:41:01.762926+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:14.265905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794747+0000","last_change":"2024-02-26T10:40:45.631974+0000","last_active":"2024-02-26T10:41:02.794747+0000","last_peered":"2024-02-26T10:41:02.794747+0000","last_clean":"2024-02-26T10:41:02.794747+0000","last_became_active":"2024-02-26T10:40:45.631691+0000","last_became_peered":"2024-02-26T10:40:45.631691+0000","last_unstale":"2024-02-26T10:41:02.794747+0000","last_undegraded":"2024-02-26T10:41:02.794747+0000","last_fullsized":"2024-02-26T10:41:02.794747+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:26:57.553070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366385+0000","last_change":"2024-02-26T10:40:44.614734+0000","last_active":"2024-02-26T10:41:03.366385+0000","last_peered":"2024-02-26T10:41:03.366385+0000","last_clean":"2024-02-26T10:41:03.366385+0000","last_became_active":"2024-02-26T10:40:44.612533+0000","last_became_peered":"2024-02-26T10:40:44.612533+0000","last_unstale":"2024-02-26T10:41:03.366385+0000","last_undegraded":"2024-02-26T10:41:03.366385+0000","last_fullsized":"2024-02-26T10:41:03.366385+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:00:39.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.802963+0000","last_change":"2024-02-26T10:41:01.799273+0000","last_active":"2024-02-26T10:41:01.802963+0000","last_peered":"2024-02-26T10:41:01.802963+0000","last_clean":"2024-02-26T10:41:01.802963+0000","last_became_active":"2024-02-26T10:41:01.798562+0000","last_became_peered":"2024-02-26T10:41:01.798562+0000","last_unstale":"2024-02-26T10:41:01.802963+0000","last_undegraded":"2024-02-26T10:41:01.802963+0000","last_fullsized":"2024-02-26T10:41:01.802963+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:23:51.770356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763637+0000","last_change":"2024-02-26T10:41:00.767901+0000","last_active":"2024-02-26T10:41:01.763637+0000","last_peered":"2024-02-26T10:41:01.763637+0000","last_clean":"2024-02-26T10:41:01.763637+0000","last_became_active":"2024-02-26T10:41:00.767605+0000","last_became_peered":"2024-02-26T10:41:00.767605+0000","last_unstale":"2024-02-26T10:41:01.763637+0000","last_undegraded":"2024-02-26T10:41:01.763637+0000","last_fullsized":"2024-02-26T10:41:01.763637+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:20:06.779347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795426+0000","last_change":"2024-02-26T10:40:45.645152+0000","last_active":"2024-02-26T10:41:02.795426+0000","last_peered":"2024-02-26T10:41:02.795426+0000","last_clean":"2024-02-26T10:41:02.795426+0000","last_became_active":"2024-02-26T10:40:45.644904+0000","last_became_peered":"2024-02-26T10:40:45.644904+0000","last_unstale":"2024-02-26T10:41:02.795426+0000","last_undegraded":"2024-02-26T10:41:02.795426+0000","last_fullsized":"2024-02-26T10:41:02.795426+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:43.820459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781142+0000","last_change":"2024-02-26T10:40:44.608899+0000","last_active":"2024-02-26T10:41:01.781142+0000","last_peered":"2024-02-26T10:41:01.781142+0000","last_clean":"2024-02-26T10:41:01.781142+0000","last_became_active":"2024-02-26T10:40:44.608621+0000","last_became_peered":"2024-02-26T10:40:44.608621+0000","last_unstale":"2024-02-26T10:41:01.781142+0000","last_undegraded":"2024-02-26T10:41:01.781142+0000","last_fullsized":"2024-02-26T10:41:01.781142+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:51.049847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112730+0000","last_change":"2024-02-26T10:41:01.786178+0000","last_active":"2024-02-26T10:41:03.112730+0000","last_peered":"2024-02-26T10:41:03.112730+0000","last_clean":"2024-02-26T10:41:03.112730+0000","last_became_active":"2024-02-26T10:41:01.785908+0000","last_became_peered":"2024-02-26T10:41:01.785908+0000","last_unstale":"2024-02-26T10:41:03.112730+0000","last_undegraded":"2024-02-26T10:41:03.112730+0000","last_fullsized":"2024-02-26T10:41:03.112730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:11:58.363793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795285+0000","last_change":"2024-02-26T10:41:00.788397+0000","last_active":"2024-02-26T10:41:02.795285+0000","last_peered":"2024-02-26T10:41:02.795285+0000","last_clean":"2024-02-26T10:41:02.795285+0000","last_became_active":"2024-02-26T10:41:00.787968+0000","last_became_peered":"2024-02-26T10:41:00.787968+0000","last_unstale":"2024-02-26T10:41:02.795285+0000","last_undegraded":"2024-02-26T10:41:02.795285+0000","last_fullsized":"2024-02-26T10:41:02.795285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:17:12.185042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792608+0000","last_change":"2024-02-26T10:40:45.643761+0000","last_active":"2024-02-26T10:41:02.792608+0000","last_peered":"2024-02-26T10:41:02.792608+0000","last_clean":"2024-02-26T10:41:02.792608+0000","last_became_active":"2024-02-26T10:40:45.643356+0000","last_became_peered":"2024-02-26T10:40:45.643356+0000","last_unstale":"2024-02-26T10:41:02.792608+0000","last_undegraded":"2024-02-26T10:41:02.792608+0000","last_fullsized":"2024-02-26T10:41:02.792608+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:30.965210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113044+0000","last_change":"2024-02-26T10:40:44.621252+0000","last_active":"2024-02-26T10:41:03.113044+0000","last_peered":"2024-02-26T10:41:03.113044+0000","last_clean":"2024-02-26T10:41:03.113044+0000","last_became_active":"2024-02-26T10:40:44.620990+0000","last_became_peered":"2024-02-26T10:40:44.620990+0000","last_unstale":"2024-02-26T10:41:03.113044+0000","last_undegraded":"2024-02-26T10:41:03.113044+0000","last_fullsized":"2024-02-26T10:41:03.113044+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:32.271293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366236+0000","last_change":"2024-02-26T10:41:01.792324+0000","last_active":"2024-02-26T10:41:03.366236+0000","last_peered":"2024-02-26T10:41:03.366236+0000","last_clean":"2024-02-26T10:41:03.366236+0000","last_became_active":"2024-02-26T10:41:01.792045+0000","last_became_peered":"2024-02-26T10:41:01.792045+0000","last_unstale":"2024-02-26T10:41:03.366236+0000","last_undegraded":"2024-02-26T10:41:03.366236+0000","last_fullsized":"2024-02-26T10:41:03.366236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:59.265674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792996+0000","last_change":"2024-02-26T10:41:00.790542+0000","last_active":"2024-02-26T10:41:02.792996+0000","last_peered":"2024-02-26T10:41:02.792996+0000","last_clean":"2024-02-26T10:41:02.792996+0000","last_became_active":"2024-02-26T10:41:00.789842+0000","last_became_peered":"2024-02-26T10:41:00.789842+0000","last_unstale":"2024-02-26T10:41:02.792996+0000","last_undegraded":"2024-02-26T10:41:02.792996+0000","last_fullsized":"2024-02-26T10:41:02.792996+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:48.425059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794173+0000","last_change":"2024-02-26T10:40:45.647365+0000","last_active":"2024-02-26T10:41:02.794173+0000","last_peered":"2024-02-26T10:41:02.794173+0000","last_clean":"2024-02-26T10:41:02.794173+0000","last_became_active":"2024-02-26T10:40:45.638659+0000","last_became_peered":"2024-02-26T10:40:45.638659+0000","last_unstale":"2024-02-26T10:41:02.794173+0000","last_undegraded":"2024-02-26T10:41:02.794173+0000","last_fullsized":"2024-02-26T10:41:02.794173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:35:53.706078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365749+0000","last_change":"2024-02-26T10:40:44.615854+0000","last_active":"2024-02-26T10:41:03.365749+0000","last_peered":"2024-02-26T10:41:03.365749+0000","last_clean":"2024-02-26T10:41:03.365749+0000","last_became_active":"2024-02-26T10:40:44.615511+0000","last_became_peered":"2024-02-26T10:40:44.615511+0000","last_unstale":"2024-02-26T10:41:03.365749+0000","last_undegraded":"2024-02-26T10:41:03.365749+0000","last_fullsized":"2024-02-26T10:41:03.365749+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:04:43.874952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793203+0000","last_change":"2024-02-26T10:41:01.783755+0000","last_active":"2024-02-26T10:41:02.793203+0000","last_peered":"2024-02-26T10:41:02.793203+0000","last_clean":"2024-02-26T10:41:02.793203+0000","last_became_active":"2024-02-26T10:41:01.783486+0000","last_became_peered":"2024-02-26T10:41:01.783486+0000","last_unstale":"2024-02-26T10:41:02.793203+0000","last_undegraded":"2024-02-26T10:41:02.793203+0000","last_fullsized":"2024-02-26T10:41:02.793203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:01:09.228302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798279+0000","last_change":"2024-02-26T10:41:00.788941+0000","last_active":"2024-02-26T10:41:02.798279+0000","last_peered":"2024-02-26T10:41:02.798279+0000","last_clean":"2024-02-26T10:41:02.798279+0000","last_became_active":"2024-02-26T10:41:00.788657+0000","last_became_peered":"2024-02-26T10:41:00.788657+0000","last_unstale":"2024-02-26T10:41:02.798279+0000","last_undegraded":"2024-02-26T10:41:02.798279+0000","last_fullsized":"2024-02-26T10:41:02.798279+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:12:39.553888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367486+0000","last_change":"2024-02-26T10:40:45.645033+0000","last_active":"2024-02-26T10:41:03.367486+0000","last_peered":"2024-02-26T10:41:03.367486+0000","last_clean":"2024-02-26T10:41:03.367486+0000","last_became_active":"2024-02-26T10:40:45.644738+0000","last_became_peered":"2024-02-26T10:40:45.644738+0000","last_unstale":"2024-02-26T10:41:03.367486+0000","last_undegraded":"2024-02-26T10:41:03.367486+0000","last_fullsized":"2024-02-26T10:41:03.367486+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:53:58.631698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241843+0000","last_change":"2024-02-26T10:40:44.619564+0000","last_active":"2024-02-26T10:41:03.241843+0000","last_peered":"2024-02-26T10:41:03.241843+0000","last_clean":"2024-02-26T10:41:03.241843+0000","last_became_active":"2024-02-26T10:40:44.619054+0000","last_became_peered":"2024-02-26T10:40:44.619054+0000","last_unstale":"2024-02-26T10:41:03.241843+0000","last_undegraded":"2024-02-26T10:41:03.241843+0000","last_fullsized":"2024-02-26T10:41:03.241843+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:18:55.475057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799223+0000","last_change":"2024-02-26T10:41:01.788092+0000","last_active":"2024-02-26T10:41:01.799223+0000","last_peered":"2024-02-26T10:41:01.799223+0000","last_clean":"2024-02-26T10:41:01.799223+0000","last_became_active":"2024-02-26T10:41:01.786793+0000","last_became_peered":"2024-02-26T10:41:01.786793+0000","last_unstale":"2024-02-26T10:41:01.799223+0000","last_undegraded":"2024-02-26T10:41:01.799223+0000","last_fullsized":"2024-02-26T10:41:01.799223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:46:52.532082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"58'6","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.803437+0000","last_change":"2024-02-26T10:41:00.770606+0000","last_active":"2024-02-26T10:41:01.803437+0000","last_peered":"2024-02-26T10:41:01.803437+0000","last_clean":"2024-02-26T10:41:01.803437+0000","last_became_active":"2024-02-26T10:41:00.770049+0000","last_became_peered":"2024-02-26T10:41:00.770049+0000","last_unstale":"2024-02-26T10:41:01.803437+0000","last_undegraded":"2024-02-26T10:41:01.803437+0000","last_fullsized":"2024-02-26T10:41:01.803437+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:21.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241520+0000","last_change":"2024-02-26T10:40:45.651473+0000","last_active":"2024-02-26T10:41:03.241520+0000","last_peered":"2024-02-26T10:41:03.241520+0000","last_clean":"2024-02-26T10:41:03.241520+0000","last_became_active":"2024-02-26T10:40:45.651136+0000","last_became_peered":"2024-02-26T10:40:45.651136+0000","last_unstale":"2024-02-26T10:41:03.241520+0000","last_undegraded":"2024-02-26T10:41:03.241520+0000","last_fullsized":"2024-02-26T10:41:03.241520+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:32.396909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.798922+0000","last_change":"2024-02-26T10:40:44.613640+0000","last_active":"2024-02-26T10:41:02.798922+0000","last_peered":"2024-02-26T10:41:02.798922+0000","last_clean":"2024-02-26T10:41:02.798922+0000","last_became_active":"2024-02-26T10:40:44.613419+0000","last_became_peered":"2024-02-26T10:40:44.613419+0000","last_unstale":"2024-02-26T10:41:02.798922+0000","last_undegraded":"2024-02-26T10:41:02.798922+0000","last_fullsized":"2024-02-26T10:41:02.798922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:57.206752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113873+0000","last_change":"2024-02-26T10:41:01.796297+0000","last_active":"2024-02-26T10:41:03.113873+0000","last_peered":"2024-02-26T10:41:03.113873+0000","last_clean":"2024-02-26T10:41:03.113873+0000","last_became_active":"2024-02-26T10:41:01.795214+0000","last_became_peered":"2024-02-26T10:41:01.795214+0000","last_unstale":"2024-02-26T10:41:03.113873+0000","last_undegraded":"2024-02-26T10:41:03.113873+0000","last_fullsized":"2024-02-26T10:41:03.113873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:39:29.874066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366995+0000","last_change":"2024-02-26T10:41:00.789707+0000","last_active":"2024-02-26T10:41:03.366995+0000","last_peered":"2024-02-26T10:41:03.366995+0000","last_clean":"2024-02-26T10:41:03.366995+0000","last_became_active":"2024-02-26T10:41:00.787710+0000","last_became_peered":"2024-02-26T10:41:00.787710+0000","last_unstale":"2024-02-26T10:41:03.366995+0000","last_undegraded":"2024-02-26T10:41:03.366995+0000","last_fullsized":"2024-02-26T10:41:03.366995+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:52:45.987760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.766459+0000","last_change":"2024-02-26T10:40:45.640605+0000","last_active":"2024-02-26T10:41:01.766459+0000","last_peered":"2024-02-26T10:41:01.766459+0000","last_clean":"2024-02-26T10:41:01.766459+0000","last_became_active":"2024-02-26T10:40:45.640078+0000","last_became_peered":"2024-02-26T10:40:45.640078+0000","last_unstale":"2024-02-26T10:41:01.766459+0000","last_undegraded":"2024-02-26T10:41:01.766459+0000","last_fullsized":"2024-02-26T10:41:01.766459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:28.520753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":25,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241233+0000","last_change":"2024-02-26T10:40:44.618998+0000","last_active":"2024-02-26T10:41:03.241233+0000","last_peered":"2024-02-26T10:41:03.241233+0000","last_clean":"2024-02-26T10:41:03.241233+0000","last_became_active":"2024-02-26T10:40:44.618710+0000","last_became_peered":"2024-02-26T10:40:44.618710+0000","last_unstale":"2024-02-26T10:41:03.241233+0000","last_undegraded":"2024-02-26T10:41:03.241233+0000","last_fullsized":"2024-02-26T10:41:03.241233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:55.190540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241129+0000","last_change":"2024-02-26T10:41:01.799011+0000","last_active":"2024-02-26T10:41:03.241129+0000","last_peered":"2024-02-26T10:41:03.241129+0000","last_clean":"2024-02-26T10:41:03.241129+0000","last_became_active":"2024-02-26T10:41:01.798726+0000","last_became_peered":"2024-02-26T10:41:01.798726+0000","last_unstale":"2024-02-26T10:41:03.241129+0000","last_undegraded":"2024-02-26T10:41:03.241129+0000","last_fullsized":"2024-02-26T10:41:03.241129+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:56:24.277610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793687+0000","last_change":"2024-02-26T10:41:00.792723+0000","last_active":"2024-02-26T10:41:02.793687+0000","last_peered":"2024-02-26T10:41:02.793687+0000","last_clean":"2024-02-26T10:41:02.793687+0000","last_became_active":"2024-02-26T10:41:00.792217+0000","last_became_peered":"2024-02-26T10:41:00.792217+0000","last_unstale":"2024-02-26T10:41:02.793687+0000","last_undegraded":"2024-02-26T10:41:02.793687+0000","last_fullsized":"2024-02-26T10:41:02.793687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:19.984955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240905+0000","last_change":"2024-02-26T10:40:45.652659+0000","last_active":"2024-02-26T10:41:03.240905+0000","last_peered":"2024-02-26T10:41:03.240905+0000","last_clean":"2024-02-26T10:41:03.240905+0000","last_became_active":"2024-02-26T10:40:45.652312+0000","last_became_peered":"2024-02-26T10:40:45.652312+0000","last_unstale":"2024-02-26T10:41:03.240905+0000","last_undegraded":"2024-02-26T10:41:03.240905+0000","last_fullsized":"2024-02-26T10:41:03.240905+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:16.081443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'9","reported_seq":31,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367191+0000","last_change":"2024-02-26T10:40:44.615861+0000","last_active":"2024-02-26T10:41:03.367191+0000","last_peered":"2024-02-26T10:41:03.367191+0000","last_clean":"2024-02-26T10:41:03.367191+0000","last_became_active":"2024-02-26T10:40:44.615538+0000","last_became_peered":"2024-02-26T10:40:44.615538+0000","last_unstale":"2024-02-26T10:41:03.367191+0000","last_undegraded":"2024-02-26T10:41:03.367191+0000","last_fullsized":"2024-02-26T10:41:03.367191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:17:29.960665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3115,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.802711+0000","last_change":"2024-02-26T10:41:01.799282+0000","last_active":"2024-02-26T10:41:01.802711+0000","last_peered":"2024-02-26T10:41:01.802711+0000","last_clean":"2024-02-26T10:41:01.802711+0000","last_became_active":"2024-02-26T10:41:01.797979+0000","last_became_peered":"2024-02-26T10:41:01.797979+0000","last_unstale":"2024-02-26T10:41:01.802711+0000","last_undegraded":"2024-02-26T10:41:01.802711+0000","last_fullsized":"2024-02-26T10:41:01.802711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:27:13.718577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112873+0000","last_change":"2024-02-26T10:41:00.788343+0000","last_active":"2024-02-26T10:41:03.112873+0000","last_peered":"2024-02-26T10:41:03.112873+0000","last_clean":"2024-02-26T10:41:03.112873+0000","last_became_active":"2024-02-26T10:41:00.788053+0000","last_became_peered":"2024-02-26T10:41:00.788053+0000","last_unstale":"2024-02-26T10:41:03.112873+0000","last_undegraded":"2024-02-26T10:41:03.112873+0000","last_fullsized":"2024-02-26T10:41:03.112873+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:50.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112728+0000","last_change":"2024-02-26T10:40:45.628631+0000","last_active":"2024-02-26T10:41:03.112728+0000","last_peered":"2024-02-26T10:41:03.112728+0000","last_clean":"2024-02-26T10:41:03.112728+0000","last_became_active":"2024-02-26T10:40:45.628321+0000","last_became_peered":"2024-02-26T10:40:45.628321+0000","last_unstale":"2024-02-26T10:41:03.112728+0000","last_undegraded":"2024-02-26T10:41:03.112728+0000","last_fullsized":"2024-02-26T10:41:03.112728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":24,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795703+0000","last_change":"2024-02-26T10:40:44.617278+0000","last_active":"2024-02-26T10:41:02.795703+0000","last_peered":"2024-02-26T10:41:02.795703+0000","last_clean":"2024-02-26T10:41:02.795703+0000","last_became_active":"2024-02-26T10:40:44.610922+0000","last_became_peered":"2024-02-26T10:40:44.610922+0000","last_unstale":"2024-02-26T10:41:02.795703+0000","last_undegraded":"2024-02-26T10:41:02.795703+0000","last_fullsized":"2024-02-26T10:41:02.795703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:38:31.392364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.799559+0000","last_change":"2024-02-26T10:41:01.792387+0000","last_active":"2024-02-26T10:41:01.799559+0000","last_peered":"2024-02-26T10:41:01.799559+0000","last_clean":"2024-02-26T10:41:01.799559+0000","last_became_active":"2024-02-26T10:41:01.791270+0000","last_became_peered":"2024-02-26T10:41:01.791270+0000","last_unstale":"2024-02-26T10:41:01.799559+0000","last_undegraded":"2024-02-26T10:41:01.799559+0000","last_fullsized":"2024-02-26T10:41:01.799559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:22:51.889988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365462+0000","last_change":"2024-02-26T10:41:00.779242+0000","last_active":"2024-02-26T10:41:03.365462+0000","last_peered":"2024-02-26T10:41:03.365462+0000","last_clean":"2024-02-26T10:41:03.365462+0000","last_became_active":"2024-02-26T10:41:00.779002+0000","last_became_peered":"2024-02-26T10:41:00.779002+0000","last_unstale":"2024-02-26T10:41:03.365462+0000","last_undegraded":"2024-02-26T10:41:03.365462+0000","last_fullsized":"2024-02-26T10:41:03.365462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:33.486965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240258+0000","last_change":"2024-02-26T10:40:45.644578+0000","last_active":"2024-02-26T10:41:03.240258+0000","last_peered":"2024-02-26T10:41:03.240258+0000","last_clean":"2024-02-26T10:41:03.240258+0000","last_became_active":"2024-02-26T10:40:45.644211+0000","last_became_peered":"2024-02-26T10:40:45.644211+0000","last_unstale":"2024-02-26T10:41:03.240258+0000","last_undegraded":"2024-02-26T10:41:03.240258+0000","last_fullsized":"2024-02-26T10:41:03.240258+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:59.150565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793443+0000","last_change":"2024-02-26T10:40:44.618437+0000","last_active":"2024-02-26T10:41:02.793443+0000","last_peered":"2024-02-26T10:41:02.793443+0000","last_clean":"2024-02-26T10:41:02.793443+0000","last_became_active":"2024-02-26T10:40:44.617307+0000","last_became_peered":"2024-02-26T10:40:44.617307+0000","last_unstale":"2024-02-26T10:41:02.793443+0000","last_undegraded":"2024-02-26T10:41:02.793443+0000","last_fullsized":"2024-02-26T10:41:02.793443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:15.126480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239609+0000","last_change":"2024-02-26T10:41:01.803637+0000","last_active":"2024-02-26T10:41:03.239609+0000","last_peered":"2024-02-26T10:41:03.239609+0000","last_clean":"2024-02-26T10:41:03.239609+0000","last_became_active":"2024-02-26T10:41:01.803307+0000","last_became_peered":"2024-02-26T10:41:01.803307+0000","last_unstale":"2024-02-26T10:41:03.239609+0000","last_undegraded":"2024-02-26T10:41:03.239609+0000","last_fullsized":"2024-02-26T10:41:03.239609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:25.572381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"58'2","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.801904+0000","last_change":"2024-02-26T10:41:00.786760+0000","last_active":"2024-02-26T10:41:01.801904+0000","last_peered":"2024-02-26T10:41:01.801904+0000","last_clean":"2024-02-26T10:41:01.801904+0000","last_became_active":"2024-02-26T10:41:00.786438+0000","last_became_peered":"2024-02-26T10:41:00.786438+0000","last_unstale":"2024-02-26T10:41:01.801904+0000","last_undegraded":"2024-02-26T10:41:01.801904+0000","last_fullsized":"2024-02-26T10:41:01.801904+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:22.952631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365734+0000","last_change":"2024-02-26T10:40:45.643600+0000","last_active":"2024-02-26T10:41:03.365734+0000","last_peered":"2024-02-26T10:41:03.365734+0000","last_clean":"2024-02-26T10:41:03.365734+0000","last_became_active":"2024-02-26T10:40:45.643190+0000","last_became_peered":"2024-02-26T10:40:45.643190+0000","last_unstale":"2024-02-26T10:41:03.365734+0000","last_undegraded":"2024-02-26T10:41:03.365734+0000","last_fullsized":"2024-02-26T10:41:03.365734+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:35.142103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113071+0000","last_change":"2024-02-26T10:40:44.615356+0000","last_active":"2024-02-26T10:41:03.113071+0000","last_peered":"2024-02-26T10:41:03.113071+0000","last_clean":"2024-02-26T10:41:03.113071+0000","last_became_active":"2024-02-26T10:40:44.615072+0000","last_became_peered":"2024-02-26T10:40:44.615072+0000","last_unstale":"2024-02-26T10:41:03.113071+0000","last_undegraded":"2024-02-26T10:41:03.113071+0000","last_fullsized":"2024-02-26T10:41:03.113071+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:42:06.798865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113151+0000","last_change":"2024-02-26T10:41:01.780694+0000","last_active":"2024-02-26T10:41:03.113151+0000","last_peered":"2024-02-26T10:41:03.113151+0000","last_clean":"2024-02-26T10:41:03.113151+0000","last_became_active":"2024-02-26T10:41:01.780248+0000","last_became_peered":"2024-02-26T10:41:01.780248+0000","last_unstale":"2024-02-26T10:41:03.113151+0000","last_undegraded":"2024-02-26T10:41:03.113151+0000","last_fullsized":"2024-02-26T10:41:03.113151+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:22.919759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793161+0000","last_change":"2024-02-26T10:41:00.790030+0000","last_active":"2024-02-26T10:41:02.793161+0000","last_peered":"2024-02-26T10:41:02.793161+0000","last_clean":"2024-02-26T10:41:02.793161+0000","last_became_active":"2024-02-26T10:41:00.789473+0000","last_became_peered":"2024-02-26T10:41:00.789473+0000","last_unstale":"2024-02-26T10:41:02.793161+0000","last_undegraded":"2024-02-26T10:41:02.793161+0000","last_fullsized":"2024-02-26T10:41:02.793161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:15.501040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239826+0000","last_change":"2024-02-26T10:40:45.653780+0000","last_active":"2024-02-26T10:41:03.239826+0000","last_peered":"2024-02-26T10:41:03.239826+0000","last_clean":"2024-02-26T10:41:03.239826+0000","last_became_active":"2024-02-26T10:40:45.653336+0000","last_became_peered":"2024-02-26T10:40:45.653336+0000","last_unstale":"2024-02-26T10:41:03.239826+0000","last_undegraded":"2024-02-26T10:41:03.239826+0000","last_fullsized":"2024-02-26T10:41:03.239826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:49:13.271700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":30,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239158+0000","last_change":"2024-02-26T10:40:44.621458+0000","last_active":"2024-02-26T10:41:03.239158+0000","last_peered":"2024-02-26T10:41:03.239158+0000","last_clean":"2024-02-26T10:41:03.239158+0000","last_became_active":"2024-02-26T10:40:44.620425+0000","last_became_peered":"2024-02-26T10:40:44.620425+0000","last_unstale":"2024-02-26T10:41:03.239158+0000","last_undegraded":"2024-02-26T10:41:03.239158+0000","last_fullsized":"2024-02-26T10:41:03.239158+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:47.713723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.786181+0000","last_change":"2024-02-26T10:41:01.777865+0000","last_active":"2024-02-26T10:41:01.786181+0000","last_peered":"2024-02-26T10:41:01.786181+0000","last_clean":"2024-02-26T10:41:01.786181+0000","last_became_active":"2024-02-26T10:41:01.777419+0000","last_became_peered":"2024-02-26T10:41:01.777419+0000","last_unstale":"2024-02-26T10:41:01.786181+0000","last_undegraded":"2024-02-26T10:41:01.786181+0000","last_fullsized":"2024-02-26T10:41:01.786181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:27:16.517326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792414+0000","last_change":"2024-02-26T10:41:00.795096+0000","last_active":"2024-02-26T10:41:02.792414+0000","last_peered":"2024-02-26T10:41:02.792414+0000","last_clean":"2024-02-26T10:41:02.792414+0000","last_became_active":"2024-02-26T10:41:00.794058+0000","last_became_peered":"2024-02-26T10:41:00.794058+0000","last_unstale":"2024-02-26T10:41:02.792414+0000","last_undegraded":"2024-02-26T10:41:02.792414+0000","last_fullsized":"2024-02-26T10:41:02.792414+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:56:30.738848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241175+0000","last_change":"2024-02-26T10:40:45.654175+0000","last_active":"2024-02-26T10:41:03.241175+0000","last_peered":"2024-02-26T10:41:03.241175+0000","last_clean":"2024-02-26T10:41:03.241175+0000","last_became_active":"2024-02-26T10:40:45.653092+0000","last_became_peered":"2024-02-26T10:40:45.653092+0000","last_unstale":"2024-02-26T10:41:03.241175+0000","last_undegraded":"2024-02-26T10:41:03.241175+0000","last_fullsized":"2024-02-26T10:41:03.241175+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:48.803196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.799457+0000","last_change":"2024-02-26T10:40:44.614563+0000","last_active":"2024-02-26T10:41:02.799457+0000","last_peered":"2024-02-26T10:41:02.799457+0000","last_clean":"2024-02-26T10:41:02.799457+0000","last_became_active":"2024-02-26T10:40:44.614281+0000","last_became_peered":"2024-02-26T10:40:44.614281+0000","last_unstale":"2024-02-26T10:41:02.799457+0000","last_undegraded":"2024-02-26T10:41:02.799457+0000","last_fullsized":"2024-02-26T10:41:02.799457+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:18:58.146019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781917+0000","last_change":"2024-02-26T10:41:01.778881+0000","last_active":"2024-02-26T10:41:01.781917+0000","last_peered":"2024-02-26T10:41:01.781917+0000","last_clean":"2024-02-26T10:41:01.781917+0000","last_became_active":"2024-02-26T10:41:01.778567+0000","last_became_peered":"2024-02-26T10:41:01.778567+0000","last_unstale":"2024-02-26T10:41:01.781917+0000","last_undegraded":"2024-02-26T10:41:01.781917+0000","last_fullsized":"2024-02-26T10:41:01.781917+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:30:26.138159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240358+0000","last_change":"2024-02-26T10:41:00.791881+0000","last_active":"2024-02-26T10:41:03.240358+0000","last_peered":"2024-02-26T10:41:03.240358+0000","last_clean":"2024-02-26T10:41:03.240358+0000","last_became_active":"2024-02-26T10:41:00.790958+0000","last_became_peered":"2024-02-26T10:41:00.790958+0000","last_unstale":"2024-02-26T10:41:03.240358+0000","last_undegraded":"2024-02-26T10:41:03.240358+0000","last_fullsized":"2024-02-26T10:41:03.240358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:50.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114128+0000","last_change":"2024-02-26T10:40:45.628322+0000","last_active":"2024-02-26T10:41:03.114128+0000","last_peered":"2024-02-26T10:41:03.114128+0000","last_clean":"2024-02-26T10:41:03.114128+0000","last_became_active":"2024-02-26T10:40:45.624341+0000","last_became_peered":"2024-02-26T10:40:45.624341+0000","last_unstale":"2024-02-26T10:41:03.114128+0000","last_undegraded":"2024-02-26T10:41:03.114128+0000","last_fullsized":"2024-02-26T10:41:03.114128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.782961+0000","last_change":"2024-02-26T10:40:44.613428+0000","last_active":"2024-02-26T10:41:01.782961+0000","last_peered":"2024-02-26T10:41:01.782961+0000","last_clean":"2024-02-26T10:41:01.782961+0000","last_became_active":"2024-02-26T10:40:44.613145+0000","last_became_peered":"2024-02-26T10:40:44.613145+0000","last_unstale":"2024-02-26T10:41:01.782961+0000","last_undegraded":"2024-02-26T10:41:01.782961+0000","last_fullsized":"2024-02-26T10:41:01.782961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:44:55.190540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367545+0000","last_change":"2024-02-26T10:41:01.782129+0000","last_active":"2024-02-26T10:41:03.367545+0000","last_peered":"2024-02-26T10:41:03.367545+0000","last_clean":"2024-02-26T10:41:03.367545+0000","last_became_active":"2024-02-26T10:41:01.780673+0000","last_became_peered":"2024-02-26T10:41:01.780673+0000","last_unstale":"2024-02-26T10:41:03.367545+0000","last_undegraded":"2024-02-26T10:41:03.367545+0000","last_fullsized":"2024-02-26T10:41:03.367545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:19.088038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792713+0000","last_change":"2024-02-26T10:41:00.792372+0000","last_active":"2024-02-26T10:41:02.792713+0000","last_peered":"2024-02-26T10:41:02.792713+0000","last_clean":"2024-02-26T10:41:02.792713+0000","last_became_active":"2024-02-26T10:41:00.792068+0000","last_became_peered":"2024-02-26T10:41:00.792068+0000","last_unstale":"2024-02-26T10:41:02.792713+0000","last_undegraded":"2024-02-26T10:41:02.792713+0000","last_fullsized":"2024-02-26T10:41:02.792713+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:32:30.430501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.795588+0000","last_change":"2024-02-26T10:40:45.637902+0000","last_active":"2024-02-26T10:41:02.795588+0000","last_peered":"2024-02-26T10:41:02.795588+0000","last_clean":"2024-02-26T10:41:02.795588+0000","last_became_active":"2024-02-26T10:40:45.636813+0000","last_became_peered":"2024-02-26T10:40:45.636813+0000","last_unstale":"2024-02-26T10:41:02.795588+0000","last_undegraded":"2024-02-26T10:41:02.795588+0000","last_fullsized":"2024-02-26T10:41:02.795588+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:06.754252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113595+0000","last_change":"2024-02-26T10:40:44.616165+0000","last_active":"2024-02-26T10:41:03.113595+0000","last_peered":"2024-02-26T10:41:03.113595+0000","last_clean":"2024-02-26T10:41:03.113595+0000","last_became_active":"2024-02-26T10:40:44.614801+0000","last_became_peered":"2024-02-26T10:40:44.614801+0000","last_unstale":"2024-02-26T10:41:03.113595+0000","last_undegraded":"2024-02-26T10:41:03.113595+0000","last_fullsized":"2024-02-26T10:41:03.113595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:17.333199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241714+0000","last_change":"2024-02-26T10:41:01.791218+0000","last_active":"2024-02-26T10:41:03.241714+0000","last_peered":"2024-02-26T10:41:03.241714+0000","last_clean":"2024-02-26T10:41:03.241714+0000","last_became_active":"2024-02-26T10:41:01.790919+0000","last_became_peered":"2024-02-26T10:41:01.790919+0000","last_unstale":"2024-02-26T10:41:03.241714+0000","last_undegraded":"2024-02-26T10:41:03.241714+0000","last_fullsized":"2024-02-26T10:41:03.241714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:58.921535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.367154+0000","last_change":"2024-02-26T10:41:00.794191+0000","last_active":"2024-02-26T10:41:03.367154+0000","last_peered":"2024-02-26T10:41:03.367154+0000","last_clean":"2024-02-26T10:41:03.367154+0000","last_became_active":"2024-02-26T10:41:00.793888+0000","last_became_peered":"2024-02-26T10:41:00.793888+0000","last_unstale":"2024-02-26T10:41:03.367154+0000","last_undegraded":"2024-02-26T10:41:03.367154+0000","last_fullsized":"2024-02-26T10:41:03.367154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:56:48.425059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.241056+0000","last_change":"2024-02-26T10:40:45.655962+0000","last_active":"2024-02-26T10:41:03.241056+0000","last_peered":"2024-02-26T10:41:03.241056+0000","last_clean":"2024-02-26T10:41:03.241056+0000","last_became_active":"2024-02-26T10:40:45.655701+0000","last_became_peered":"2024-02-26T10:40:45.655701+0000","last_unstale":"2024-02-26T10:41:03.241056+0000","last_undegraded":"2024-02-26T10:41:03.241056+0000","last_fullsized":"2024-02-26T10:41:03.241056+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:34.742648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780514+0000","last_change":"2024-02-26T10:40:44.609742+0000","last_active":"2024-02-26T10:41:01.780514+0000","last_peered":"2024-02-26T10:41:01.780514+0000","last_clean":"2024-02-26T10:41:01.780514+0000","last_became_active":"2024-02-26T10:40:44.609452+0000","last_became_peered":"2024-02-26T10:40:44.609452+0000","last_unstale":"2024-02-26T10:41:01.780514+0000","last_undegraded":"2024-02-26T10:41:01.780514+0000","last_fullsized":"2024-02-26T10:41:01.780514+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:52:55.391000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792994+0000","last_change":"2024-02-26T10:41:01.798444+0000","last_active":"2024-02-26T10:41:02.792994+0000","last_peered":"2024-02-26T10:41:02.792994+0000","last_clean":"2024-02-26T10:41:02.792994+0000","last_became_active":"2024-02-26T10:41:01.797201+0000","last_became_peered":"2024-02-26T10:41:01.797201+0000","last_unstale":"2024-02-26T10:41:02.792994+0000","last_undegraded":"2024-02-26T10:41:02.792994+0000","last_fullsized":"2024-02-26T10:41:02.792994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:07.040156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112620+0000","last_change":"2024-02-26T10:41:00.788831+0000","last_active":"2024-02-26T10:41:03.112620+0000","last_peered":"2024-02-26T10:41:03.112620+0000","last_clean":"2024-02-26T10:41:03.112620+0000","last_became_active":"2024-02-26T10:41:00.787849+0000","last_became_peered":"2024-02-26T10:41:00.787849+0000","last_unstale":"2024-02-26T10:41:03.112620+0000","last_undegraded":"2024-02-26T10:41:03.112620+0000","last_fullsized":"2024-02-26T10:41:03.112620+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:07:16.987475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.112857+0000","last_change":"2024-02-26T10:40:45.631443+0000","last_active":"2024-02-26T10:41:03.112857+0000","last_peered":"2024-02-26T10:41:03.112857+0000","last_clean":"2024-02-26T10:41:03.112857+0000","last_became_active":"2024-02-26T10:40:45.631119+0000","last_became_peered":"2024-02-26T10:40:45.631119+0000","last_unstale":"2024-02-26T10:41:03.112857+0000","last_undegraded":"2024-02-26T10:41:03.112857+0000","last_fullsized":"2024-02-26T10:41:03.112857+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:33:58.211297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":24,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.796159+0000","last_change":"2024-02-26T10:40:44.616723+0000","last_active":"2024-02-26T10:41:02.796159+0000","last_peered":"2024-02-26T10:41:02.796159+0000","last_clean":"2024-02-26T10:41:02.796159+0000","last_became_active":"2024-02-26T10:40:44.616480+0000","last_became_peered":"2024-02-26T10:40:44.616480+0000","last_unstale":"2024-02-26T10:41:02.796159+0000","last_undegraded":"2024-02-26T10:41:02.796159+0000","last_fullsized":"2024-02-26T10:41:02.796159+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:42.095622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793711+0000","last_change":"2024-02-26T10:41:01.795973+0000","last_active":"2024-02-26T10:41:02.793711+0000","last_peered":"2024-02-26T10:41:02.793711+0000","last_clean":"2024-02-26T10:41:02.793711+0000","last_became_active":"2024-02-26T10:41:01.795683+0000","last_became_peered":"2024-02-26T10:41:01.795683+0000","last_unstale":"2024-02-26T10:41:02.793711+0000","last_undegraded":"2024-02-26T10:41:02.793711+0000","last_fullsized":"2024-02-26T10:41:02.793711+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:57:58.927024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.773689+0000","last_change":"2024-02-26T10:41:00.778570+0000","last_active":"2024-02-26T10:41:01.773689+0000","last_peered":"2024-02-26T10:41:01.773689+0000","last_clean":"2024-02-26T10:41:01.773689+0000","last_became_active":"2024-02-26T10:41:00.778299+0000","last_became_peered":"2024-02-26T10:41:00.778299+0000","last_unstale":"2024-02-26T10:41:01.773689+0000","last_undegraded":"2024-02-26T10:41:01.773689+0000","last_fullsized":"2024-02-26T10:41:01.773689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:42:27.400748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.779431+0000","last_change":"2024-02-26T10:40:45.639556+0000","last_active":"2024-02-26T10:41:01.779431+0000","last_peered":"2024-02-26T10:41:01.779431+0000","last_clean":"2024-02-26T10:41:01.779431+0000","last_became_active":"2024-02-26T10:40:45.639271+0000","last_became_peered":"2024-02-26T10:40:45.639271+0000","last_unstale":"2024-02-26T10:41:01.779431+0000","last_undegraded":"2024-02-26T10:41:01.779431+0000","last_fullsized":"2024-02-26T10:41:01.779431+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:40:17.975237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763918+0000","last_change":"2024-02-26T10:40:44.612287+0000","last_active":"2024-02-26T10:41:01.763918+0000","last_peered":"2024-02-26T10:41:01.763918+0000","last_clean":"2024-02-26T10:41:01.763918+0000","last_became_active":"2024-02-26T10:40:44.611972+0000","last_became_peered":"2024-02-26T10:40:44.611972+0000","last_unstale":"2024-02-26T10:41:01.763918+0000","last_undegraded":"2024-02-26T10:41:01.763918+0000","last_fullsized":"2024-02-26T10:41:01.763918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:38.390162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794866+0000","last_change":"2024-02-26T10:41:01.789881+0000","last_active":"2024-02-26T10:41:02.794866+0000","last_peered":"2024-02-26T10:41:02.794866+0000","last_clean":"2024-02-26T10:41:02.794866+0000","last_became_active":"2024-02-26T10:41:01.789504+0000","last_became_peered":"2024-02-26T10:41:01.789504+0000","last_unstale":"2024-02-26T10:41:02.794866+0000","last_undegraded":"2024-02-26T10:41:02.794866+0000","last_fullsized":"2024-02-26T10:41:02.794866+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:40:07.205812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793427+0000","last_change":"2024-02-26T10:41:00.790257+0000","last_active":"2024-02-26T10:41:02.793427+0000","last_peered":"2024-02-26T10:41:02.793427+0000","last_clean":"2024-02-26T10:41:02.793427+0000","last_became_active":"2024-02-26T10:41:00.789638+0000","last_became_peered":"2024-02-26T10:41:00.789638+0000","last_unstale":"2024-02-26T10:41:02.793427+0000","last_undegraded":"2024-02-26T10:41:02.793427+0000","last_fullsized":"2024-02-26T10:41:02.793427+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:52:45.987760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.775895+0000","last_change":"2024-02-26T10:40:45.644041+0000","last_active":"2024-02-26T10:41:01.775895+0000","last_peered":"2024-02-26T10:41:01.775895+0000","last_clean":"2024-02-26T10:41:01.775895+0000","last_became_active":"2024-02-26T10:40:45.640298+0000","last_became_peered":"2024-02-26T10:40:45.640298+0000","last_unstale":"2024-02-26T10:41:01.775895+0000","last_undegraded":"2024-02-26T10:41:01.775895+0000","last_fullsized":"2024-02-26T10:41:01.775895+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:49:13.271700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.111963+0000","last_change":"2024-02-26T10:40:44.611018+0000","last_active":"2024-02-26T10:41:03.111963+0000","last_peered":"2024-02-26T10:41:03.111963+0000","last_clean":"2024-02-26T10:41:03.111963+0000","last_became_active":"2024-02-26T10:40:44.610676+0000","last_became_peered":"2024-02-26T10:40:44.610676+0000","last_unstale":"2024-02-26T10:41:03.111963+0000","last_undegraded":"2024-02-26T10:41:03.111963+0000","last_fullsized":"2024-02-26T10:41:03.111963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:46:52.342064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240045+0000","last_change":"2024-02-26T10:41:01.799263+0000","last_active":"2024-02-26T10:41:03.240045+0000","last_peered":"2024-02-26T10:41:03.240045+0000","last_clean":"2024-02-26T10:41:03.240045+0000","last_became_active":"2024-02-26T10:41:01.798213+0000","last_became_peered":"2024-02-26T10:41:01.798213+0000","last_unstale":"2024-02-26T10:41:03.240045+0000","last_undegraded":"2024-02-26T10:41:03.240045+0000","last_fullsized":"2024-02-26T10:41:03.240045+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:39:29.874066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.770595+0000","last_change":"2024-02-26T10:41:00.775411+0000","last_active":"2024-02-26T10:41:01.770595+0000","last_peered":"2024-02-26T10:41:01.770595+0000","last_clean":"2024-02-26T10:41:01.770595+0000","last_became_active":"2024-02-26T10:41:00.774584+0000","last_became_peered":"2024-02-26T10:41:00.774584+0000","last_unstale":"2024-02-26T10:41:01.770595+0000","last_undegraded":"2024-02-26T10:41:01.770595+0000","last_fullsized":"2024-02-26T10:41:01.770595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:28:56.754194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114326+0000","last_change":"2024-02-26T10:40:45.641308+0000","last_active":"2024-02-26T10:41:03.114326+0000","last_peered":"2024-02-26T10:41:03.114326+0000","last_clean":"2024-02-26T10:41:03.114326+0000","last_became_active":"2024-02-26T10:40:45.641045+0000","last_became_peered":"2024-02-26T10:40:45.641045+0000","last_unstale":"2024-02-26T10:41:03.114326+0000","last_undegraded":"2024-02-26T10:41:03.114326+0000","last_fullsized":"2024-02-26T10:41:03.114326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.769480+0000","last_change":"2024-02-26T10:40:44.609096+0000","last_active":"2024-02-26T10:41:01.769480+0000","last_peered":"2024-02-26T10:41:01.769480+0000","last_clean":"2024-02-26T10:41:01.769480+0000","last_became_active":"2024-02-26T10:40:44.608811+0000","last_became_peered":"2024-02-26T10:40:44.608811+0000","last_unstale":"2024-02-26T10:41:01.769480+0000","last_undegraded":"2024-02-26T10:41:01.769480+0000","last_fullsized":"2024-02-26T10:41:01.769480+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:00:39.605726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240751+0000","last_change":"2024-02-26T10:41:01.797991+0000","last_active":"2024-02-26T10:41:03.240751+0000","last_peered":"2024-02-26T10:41:03.240751+0000","last_clean":"2024-02-26T10:41:03.240751+0000","last_became_active":"2024-02-26T10:41:01.797620+0000","last_became_peered":"2024-02-26T10:41:01.797620+0000","last_unstale":"2024-02-26T10:41:03.240751+0000","last_undegraded":"2024-02-26T10:41:03.240751+0000","last_fullsized":"2024-02-26T10:41:03.240751+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:11:20.045564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.780200+0000","last_change":"2024-02-26T10:41:00.773110+0000","last_active":"2024-02-26T10:41:01.780200+0000","last_peered":"2024-02-26T10:41:01.780200+0000","last_clean":"2024-02-26T10:41:01.780200+0000","last_became_active":"2024-02-26T10:41:00.772804+0000","last_became_peered":"2024-02-26T10:41:00.772804+0000","last_unstale":"2024-02-26T10:41:01.780200+0000","last_undegraded":"2024-02-26T10:41:01.780200+0000","last_fullsized":"2024-02-26T10:41:01.780200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:45.340492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114161+0000","last_change":"2024-02-26T10:40:45.642661+0000","last_active":"2024-02-26T10:41:03.114161+0000","last_peered":"2024-02-26T10:41:03.114161+0000","last_clean":"2024-02-26T10:41:03.114161+0000","last_became_active":"2024-02-26T10:40:45.640611+0000","last_became_peered":"2024-02-26T10:40:45.640611+0000","last_unstale":"2024-02-26T10:41:03.114161+0000","last_undegraded":"2024-02-26T10:41:03.114161+0000","last_fullsized":"2024-02-26T10:41:03.114161+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:14.395806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.783087+0000","last_change":"2024-02-26T10:40:44.612941+0000","last_active":"2024-02-26T10:41:01.783087+0000","last_peered":"2024-02-26T10:41:01.783087+0000","last_clean":"2024-02-26T10:41:01.783087+0000","last_became_active":"2024-02-26T10:40:44.612610+0000","last_became_peered":"2024-02-26T10:40:44.612610+0000","last_unstale":"2024-02-26T10:41:01.783087+0000","last_undegraded":"2024-02-26T10:41:01.783087+0000","last_fullsized":"2024-02-26T10:41:01.783087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:51:06.688227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.803343+0000","last_change":"2024-02-26T10:41:01.801264+0000","last_active":"2024-02-26T10:41:01.803343+0000","last_peered":"2024-02-26T10:41:01.803343+0000","last_clean":"2024-02-26T10:41:01.803343+0000","last_became_active":"2024-02-26T10:41:01.800910+0000","last_became_peered":"2024-02-26T10:41:01.800910+0000","last_unstale":"2024-02-26T10:41:01.803343+0000","last_undegraded":"2024-02-26T10:41:01.803343+0000","last_fullsized":"2024-02-26T10:41:01.803343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:49.607021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793600+0000","last_change":"2024-02-26T10:41:00.792170+0000","last_active":"2024-02-26T10:41:02.793600+0000","last_peered":"2024-02-26T10:41:02.793600+0000","last_clean":"2024-02-26T10:41:02.793600+0000","last_became_active":"2024-02-26T10:41:00.791870+0000","last_became_peered":"2024-02-26T10:41:00.791870+0000","last_unstale":"2024-02-26T10:41:02.793600+0000","last_undegraded":"2024-02-26T10:41:02.793600+0000","last_fullsized":"2024-02-26T10:41:02.793600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:12:42.026861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.765253+0000","last_change":"2024-02-26T10:40:45.640358+0000","last_active":"2024-02-26T10:41:01.765253+0000","last_peered":"2024-02-26T10:41:01.765253+0000","last_clean":"2024-02-26T10:41:01.765253+0000","last_became_active":"2024-02-26T10:40:45.639707+0000","last_became_peered":"2024-02-26T10:40:45.639707+0000","last_unstale":"2024-02-26T10:41:01.765253+0000","last_undegraded":"2024-02-26T10:41:01.765253+0000","last_fullsized":"2024-02-26T10:41:01.765253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:03.232355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792881+0000","last_change":"2024-02-26T10:40:44.619084+0000","last_active":"2024-02-26T10:41:02.792881+0000","last_peered":"2024-02-26T10:41:02.792881+0000","last_clean":"2024-02-26T10:41:02.792881+0000","last_became_active":"2024-02-26T10:40:44.618715+0000","last_became_peered":"2024-02-26T10:40:44.618715+0000","last_unstale":"2024-02-26T10:41:02.792881+0000","last_undegraded":"2024-02-26T10:41:02.792881+0000","last_fullsized":"2024-02-26T10:41:02.792881+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:58.496066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114020+0000","last_change":"2024-02-26T10:41:01.790983+0000","last_active":"2024-02-26T10:41:03.114020+0000","last_peered":"2024-02-26T10:41:03.114020+0000","last_clean":"2024-02-26T10:41:03.114020+0000","last_became_active":"2024-02-26T10:41:01.790642+0000","last_became_peered":"2024-02-26T10:41:01.790642+0000","last_unstale":"2024-02-26T10:41:03.114020+0000","last_undegraded":"2024-02-26T10:41:03.114020+0000","last_fullsized":"2024-02-26T10:41:03.114020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:10.745518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.114154+0000","last_change":"2024-02-26T10:41:00.789535+0000","last_active":"2024-02-26T10:41:03.114154+0000","last_peered":"2024-02-26T10:41:03.114154+0000","last_clean":"2024-02-26T10:41:03.114154+0000","last_became_active":"2024-02-26T10:41:00.789225+0000","last_became_peered":"2024-02-26T10:41:00.789225+0000","last_unstale":"2024-02-26T10:41:03.114154+0000","last_undegraded":"2024-02-26T10:41:03.114154+0000","last_fullsized":"2024-02-26T10:41:03.114154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:45:45.340492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.781273+0000","last_change":"2024-02-26T10:40:45.630528+0000","last_active":"2024-02-26T10:41:01.781273+0000","last_peered":"2024-02-26T10:41:01.781273+0000","last_clean":"2024-02-26T10:41:01.781273+0000","last_became_active":"2024-02-26T10:40:45.630230+0000","last_became_peered":"2024-02-26T10:40:45.630230+0000","last_unstale":"2024-02-26T10:41:01.781273+0000","last_undegraded":"2024-02-26T10:41:01.781273+0000","last_fullsized":"2024-02-26T10:41:01.781273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.113685+0000","last_change":"2024-02-26T10:40:44.619432+0000","last_active":"2024-02-26T10:41:03.113685+0000","last_peered":"2024-02-26T10:41:03.113685+0000","last_clean":"2024-02-26T10:41:03.113685+0000","last_became_active":"2024-02-26T10:40:44.619141+0000","last_became_peered":"2024-02-26T10:40:44.619141+0000","last_unstale":"2024-02-26T10:41:03.113685+0000","last_undegraded":"2024-02-26T10:41:03.113685+0000","last_fullsized":"2024-02-26T10:41:03.113685+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:05.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.790251+0000","last_change":"2024-02-26T10:41:01.786301+0000","last_active":"2024-02-26T10:41:01.790251+0000","last_peered":"2024-02-26T10:41:01.790251+0000","last_clean":"2024-02-26T10:41:01.790251+0000","last_became_active":"2024-02-26T10:41:01.785990+0000","last_became_peered":"2024-02-26T10:41:01.785990+0000","last_unstale":"2024-02-26T10:41:01.790251+0000","last_undegraded":"2024-02-26T10:41:01.790251+0000","last_fullsized":"2024-02-26T10:41:01.790251+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:21.001241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.365351+0000","last_change":"2024-02-26T10:41:00.777167+0000","last_active":"2024-02-26T10:41:03.365351+0000","last_peered":"2024-02-26T10:41:03.365351+0000","last_clean":"2024-02-26T10:41:03.365351+0000","last_became_active":"2024-02-26T10:41:00.776713+0000","last_became_peered":"2024-02-26T10:41:00.776713+0000","last_unstale":"2024-02-26T10:41:03.365351+0000","last_undegraded":"2024-02-26T10:41:03.365351+0000","last_fullsized":"2024-02-26T10:41:03.365351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:24:21.789674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794975+0000","last_change":"2024-02-26T10:40:45.634981+0000","last_active":"2024-02-26T10:41:02.794975+0000","last_peered":"2024-02-26T10:41:02.794975+0000","last_clean":"2024-02-26T10:41:02.794975+0000","last_became_active":"2024-02-26T10:40:45.634638+0000","last_became_peered":"2024-02-26T10:40:45.634638+0000","last_unstale":"2024-02-26T10:41:02.794975+0000","last_undegraded":"2024-02-26T10:41:02.794975+0000","last_fullsized":"2024-02-26T10:41:02.794975+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:01:22.095627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.762921+0000","last_change":"2024-02-26T10:40:44.599450+0000","last_active":"2024-02-26T10:41:01.762921+0000","last_peered":"2024-02-26T10:41:01.762921+0000","last_clean":"2024-02-26T10:41:01.762921+0000","last_became_active":"2024-02-26T10:40:44.598802+0000","last_became_peered":"2024-02-26T10:40:44.598802+0000","last_unstale":"2024-02-26T10:41:01.762921+0000","last_undegraded":"2024-02-26T10:41:01.762921+0000","last_fullsized":"2024-02-26T10:41:01.762921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:45:58.496066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792411+0000","last_change":"2024-02-26T10:41:01.785467+0000","last_active":"2024-02-26T10:41:02.792411+0000","last_peered":"2024-02-26T10:41:02.792411+0000","last_clean":"2024-02-26T10:41:02.792411+0000","last_became_active":"2024-02-26T10:41:01.784587+0000","last_became_peered":"2024-02-26T10:41:01.784587+0000","last_unstale":"2024-02-26T10:41:02.792411+0000","last_undegraded":"2024-02-26T10:41:02.792411+0000","last_fullsized":"2024-02-26T10:41:02.792411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:38:19.088038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.791877+0000","last_change":"2024-02-26T10:41:00.793641+0000","last_active":"2024-02-26T10:41:02.791877+0000","last_peered":"2024-02-26T10:41:02.791877+0000","last_clean":"2024-02-26T10:41:02.791877+0000","last_became_active":"2024-02-26T10:41:00.793308+0000","last_became_peered":"2024-02-26T10:41:00.793308+0000","last_unstale":"2024-02-26T10:41:02.791877+0000","last_undegraded":"2024-02-26T10:41:02.791877+0000","last_fullsized":"2024-02-26T10:41:02.791877+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:17:14.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.240555+0000","last_change":"2024-02-26T10:40:45.650467+0000","last_active":"2024-02-26T10:41:03.240555+0000","last_peered":"2024-02-26T10:41:03.240555+0000","last_clean":"2024-02-26T10:41:03.240555+0000","last_became_active":"2024-02-26T10:40:45.649189+0000","last_became_peered":"2024-02-26T10:40:45.649189+0000","last_unstale":"2024-02-26T10:41:03.240555+0000","last_undegraded":"2024-02-26T10:41:03.240555+0000","last_fullsized":"2024-02-26T10:41:03.240555+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:00:43.100167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.366185+0000","last_change":"2024-02-26T10:40:44.613063+0000","last_active":"2024-02-26T10:41:03.366185+0000","last_peered":"2024-02-26T10:41:03.366185+0000","last_clean":"2024-02-26T10:41:03.366185+0000","last_became_active":"2024-02-26T10:40:44.612792+0000","last_became_peered":"2024-02-26T10:40:44.612792+0000","last_unstale":"2024-02-26T10:41:03.366185+0000","last_undegraded":"2024-02-26T10:41:03.366185+0000","last_fullsized":"2024-02-26T10:41:03.366185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:08:38.390162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.801870+0000","last_change":"2024-02-26T10:41:01.793419+0000","last_active":"2024-02-26T10:41:01.801870+0000","last_peered":"2024-02-26T10:41:01.801870+0000","last_clean":"2024-02-26T10:41:01.801870+0000","last_became_active":"2024-02-26T10:41:01.785140+0000","last_became_peered":"2024-02-26T10:41:01.785140+0000","last_unstale":"2024-02-26T10:41:01.801870+0000","last_undegraded":"2024-02-26T10:41:01.801870+0000","last_fullsized":"2024-02-26T10:41:01.801870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:34:32.359358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763665+0000","last_change":"2024-02-26T10:41:00.790834+0000","last_active":"2024-02-26T10:41:01.763665+0000","last_peered":"2024-02-26T10:41:01.763665+0000","last_clean":"2024-02-26T10:41:01.763665+0000","last_became_active":"2024-02-26T10:41:00.790521+0000","last_became_peered":"2024-02-26T10:41:00.790521+0000","last_unstale":"2024-02-26T10:41:01.763665+0000","last_undegraded":"2024-02-26T10:41:01.763665+0000","last_fullsized":"2024-02-26T10:41:01.763665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:38.428822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.764538+0000","last_change":"2024-02-26T10:40:45.636109+0000","last_active":"2024-02-26T10:41:01.764538+0000","last_peered":"2024-02-26T10:41:01.764538+0000","last_clean":"2024-02-26T10:41:01.764538+0000","last_became_active":"2024-02-26T10:40:45.635783+0000","last_became_peered":"2024-02-26T10:40:45.635783+0000","last_unstale":"2024-02-26T10:41:01.764538+0000","last_undegraded":"2024-02-26T10:41:01.764538+0000","last_fullsized":"2024-02-26T10:41:01.764538+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:45:18.756782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763240+0000","last_change":"2024-02-26T10:40:44.595757+0000","last_active":"2024-02-26T10:41:01.763240+0000","last_peered":"2024-02-26T10:41:01.763240+0000","last_clean":"2024-02-26T10:41:01.763240+0000","last_became_active":"2024-02-26T10:40:44.595491+0000","last_became_peered":"2024-02-26T10:40:44.595491+0000","last_unstale":"2024-02-26T10:41:01.763240+0000","last_undegraded":"2024-02-26T10:41:01.763240+0000","last_fullsized":"2024-02-26T10:41:01.763240+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:33:15.126480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792698+0000","last_change":"2024-02-26T10:41:01.790344+0000","last_active":"2024-02-26T10:41:02.792698+0000","last_peered":"2024-02-26T10:41:02.792698+0000","last_clean":"2024-02-26T10:41:02.792698+0000","last_became_active":"2024-02-26T10:41:01.787549+0000","last_became_peered":"2024-02-26T10:41:01.787549+0000","last_unstale":"2024-02-26T10:41:02.792698+0000","last_undegraded":"2024-02-26T10:41:02.792698+0000","last_fullsized":"2024-02-26T10:41:02.792698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:22:57.319850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-02-26T10:41:01.763599+0000","last_change":"2024-02-26T10:41:00.777425+0000","last_active":"2024-02-26T10:41:01.763599+0000","last_peered":"2024-02-26T10:41:01.763599+0000","last_clean":"2024-02-26T10:41:01.763599+0000","last_became_active":"2024-02-26T10:41:00.777106+0000","last_became_peered":"2024-02-26T10:41:00.777106+0000","last_unstale":"2024-02-26T10:41:01.763599+0000","last_undegraded":"2024-02-26T10:41:01.763599+0000","last_fullsized":"2024-02-26T10:41:01.763599+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:33.486965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794390+0000","last_change":"2024-02-26T10:40:45.647115+0000","last_active":"2024-02-26T10:41:02.794390+0000","last_peered":"2024-02-26T10:41:02.794390+0000","last_clean":"2024-02-26T10:41:02.794390+0000","last_became_active":"2024-02-26T10:40:45.637035+0000","last_became_peered":"2024-02-26T10:40:45.637035+0000","last_unstale":"2024-02-26T10:41:02.794390+0000","last_undegraded":"2024-02-26T10:41:02.794390+0000","last_fullsized":"2024-02-26T10:41:02.794390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:26.126929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.794174+0000","last_change":"2024-02-26T10:40:44.620365+0000","last_active":"2024-02-26T10:41:02.794174+0000","last_peered":"2024-02-26T10:41:02.794174+0000","last_clean":"2024-02-26T10:41:02.794174+0000","last_became_active":"2024-02-26T10:40:44.619424+0000","last_became_peered":"2024-02-26T10:40:44.619424+0000","last_unstale":"2024-02-26T10:41:02.794174+0000","last_undegraded":"2024-02-26T10:41:02.794174+0000","last_fullsized":"2024-02-26T10:41:02.794174+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:15.076898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793576+0000","last_change":"2024-02-26T10:41:01.794921+0000","last_active":"2024-02-26T10:41:02.793576+0000","last_peered":"2024-02-26T10:41:02.793576+0000","last_clean":"2024-02-26T10:41:02.793576+0000","last_became_active":"2024-02-26T10:41:01.794650+0000","last_became_peered":"2024-02-26T10:41:01.794650+0000","last_unstale":"2024-02-26T10:41:02.793576+0000","last_undegraded":"2024-02-26T10:41:02.793576+0000","last_fullsized":"2024-02-26T10:41:02.793576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:00.736657+0000","last_clean_scrub_stamp":"2024-02-26T10:41:00.736657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:45:35.543250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792894+0000","last_change":"2024-02-26T10:41:00.786808+0000","last_active":"2024-02-26T10:41:02.792894+0000","last_peered":"2024-02-26T10:41:02.792894+0000","last_clean":"2024-02-26T10:41:02.792894+0000","last_became_active":"2024-02-26T10:41:00.784517+0000","last_became_peered":"2024-02-26T10:41:00.784517+0000","last_unstale":"2024-02-26T10:41:02.792894+0000","last_undegraded":"2024-02-26T10:41:02.792894+0000","last_fullsized":"2024-02-26T10:41:02.792894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:59.720371+0000","last_clean_scrub_stamp":"2024-02-26T10:40:59.720371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:57:33.486965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.793600+0000","last_change":"2024-02-26T10:40:45.633488+0000","last_active":"2024-02-26T10:41:02.793600+0000","last_peered":"2024-02-26T10:41:02.793600+0000","last_clean":"2024-02-26T10:41:02.793600+0000","last_became_active":"2024-02-26T10:40:45.633157+0000","last_became_peered":"2024-02-26T10:40:45.633157+0000","last_unstale":"2024-02-26T10:41:02.793600+0000","last_undegraded":"2024-02-26T10:41:02.793600+0000","last_fullsized":"2024-02-26T10:41:02.793600+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:44.571150+0000","last_clean_scrub_stamp":"2024-02-26T10:40:44.571150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:27:33.441514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:02.792164+0000","last_change":"2024-02-26T10:40:44.614389+0000","last_active":"2024-02-26T10:41:02.792164+0000","last_peered":"2024-02-26T10:41:02.792164+0000","last_clean":"2024-02-26T10:41:02.792164+0000","last_became_active":"2024-02-26T10:40:44.613960+0000","last_became_peered":"2024-02-26T10:40:44.613960+0000","last_unstale":"2024-02-26T10:41:02.792164+0000","last_undegraded":"2024-02-26T10:41:02.792164+0000","last_fullsized":"2024-02-26T10:41:02.792164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:40:43.566605+0000","last_clean_scrub_stamp":"2024-02-26T10:40:43.566605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:09:07.764711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":242,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-02-26T10:41:03.239191+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:41:03.239191+0000","last_peered":"2024-02-26T10:41:03.239191+0000","last_clean":"2024-02-26T10:41:03.239191+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:41:03.239191+0000","last_undegraded":"2024-02-26T10:41:03.239191+0000","last_fullsized":"2024-02-26T10:41:03.239191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3913,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":894522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":310627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":704512,"data_stored":304120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":306563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":303546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":712704,"data_stored":305046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":716800,"data_stored":305696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542179,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":897443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:41:06.844 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:41:06.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-02-26T10:41:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.117+0000 7fa6df200640 1 Processor -- start 2024-02-26T10:41:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.117+0000 7fa6df200640 1 -- start start 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6df200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6d813c000 0x7fa6d8072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6df200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d8153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6df200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d806e8f0 con 0x7fa6d806e460 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6df200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d806eb60 con 0x7fa6d813c000 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6df200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d81564b0 con 0x7fa6d8152ea0 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6dde00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6d813c000 0x7fa6d8072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6dde00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6d813c000 0x7fa6d8072dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54864/0 (socket says 172.21.15.67:54864) 2024-02-26T10:41:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.119+0000 7fa6dde00640 1 -- 172.21.15.67:0/3666900553 learned_addr learned my addr 172.21.15.67:0/3666900553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.120+0000 7fa6de800640 1 --2- 172.21.15.67:0/3666900553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d8153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.120+0000 7fa6dde00640 1 -- 172.21.15.67:0/3666900553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 msgr2=0x7fa6d8153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.120+0000 7fa6dde00640 1 --2- 172.21.15.67:0/3666900553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d8153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.120+0000 7fa6dde00640 1 -- 172.21.15.67:0/3666900553 >> v1:172.21.15.82:6789/0 conn(0x7fa6d806e460 legacy=0x7fa6d80c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.120+0000 7fa6dde00640 1 -- 172.21.15.67:0/3666900553 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8156790 con 0x7fa6d813c000 2024-02-26T10:41:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.122+0000 7fa6dde00640 1 --2- 172.21.15.67:0/3666900553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8072dd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009870 tx=0x7fa6cc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5428af3f270a06 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.122+0000 7fa6dca00640 1 -- 172.21.15.67:0/3666900553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0317c0 con 0x7fa6d813c000 2024-02-26T10:41:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.122+0000 7fa6dca00640 1 -- 172.21.15.67:0/3666900553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc037070 con 0x7fa6d813c000 2024-02-26T10:41:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.122+0000 7fa6dca00640 1 -- 172.21.15.67:0/3666900553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc031b80 con 0x7fa6d813c000 2024-02-26T10:41:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 -- 172.21.15.67:0/3666900553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 msgr2=0x7fa6d8072dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 --2- 172.21.15.67:0/3666900553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8072dd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009870 tx=0x7fa6cc02b860 comp rx=0 tx=0).stop 2024-02-26T10:41:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 -- 172.21.15.67:0/3666900553 shutdown_connections 2024-02-26T10:41:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 --2- 172.21.15.67:0/3666900553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d8153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 --2- 172.21.15.67:0/3666900553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8072dd0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 -- 172.21.15.67:0/3666900553 >> 172.21.15.67:0/3666900553 conn(0x7fa6d80a84a0 msgr2=0x7fa6d80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 -- 172.21.15.67:0/3666900553 shutdown_connections 2024-02-26T10:41:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.123+0000 7fa6df200640 1 -- 172.21.15.67:0/3666900553 wait complete. 2024-02-26T10:41:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.125+0000 7fa6df200640 1 Processor -- start 2024-02-26T10:41:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.125+0000 7fa6df200640 1 -- start start 2024-02-26T10:41:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6df200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d806e460 0x7fa6d80c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6df200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6df200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d81edf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6df200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8157270 con 0x7fa6d813c000 2024-02-26T10:41:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6df200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8156d50 con 0x7fa6d806e460 2024-02-26T10:41:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6df200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d813c3e0 con 0x7fa6d8152ea0 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6dd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.126+0000 7fa6dde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d806e460 0x7fa6d80c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8131c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54880/0 (socket says 172.21.15.67:54880) 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d806e460 0x7fa6d80c2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40464/0 (socket says 172.21.15.67:40464) 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 -- 172.21.15.67:0/3783582598 learned_addr learned my addr 172.21.15.67:0/3783582598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6de800640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d81edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 -- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 msgr2=0x7fa6d81edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d81edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 -- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d806e460 msgr2=0x7fa6d80c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d806e460 0x7fa6d80c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 -- 172.21.15.67:0/3783582598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80aa380 con 0x7fa6d813c000 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.127+0000 7fa6dd400640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8131c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b420 tx=0x7fa6c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.128+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011830 con 0x7fa6d813c000 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.128+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4016070 con 0x7fa6d813c000 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.128+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011c10 con 0x7fa6d813c000 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.128+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d80aa660 con 0x7fa6d813c000 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.128+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d80c2580 con 0x7fa6d813c000 2024-02-26T10:41:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.129+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d8074a30 con 0x7fa6d813c000 2024-02-26T10:41:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.130+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6c401b460 con 0x7fa6d813c000 2024-02-26T10:41:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.131+0000 7fa6d2000640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6b806b500 0x7fa6b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.131+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa6c4089f20 con 0x7fa6d813c000 2024-02-26T10:41:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.132+0000 7fa6dde00640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6b806b500 0x7fa6b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.133+0000 7fa6dde00640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6b806b500 0x7fa6b806d9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc036000 tx=0x7fa6cc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:07.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.136+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6c4020020 con 0x7fa6d813c000 2024-02-26T10:41:07.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.406+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7fa6d80a8b70 con 0x7fa6d813c000 2024-02-26T10:41:07.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.411+0000 7fa6d2000640 1 -- 172.21.15.67:0/3783582598 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v14) v1 ==== 185+0+142 (secure 0 0 0) 0x7fa6c40583e0 con 0x7fa6d813c000 2024-02-26T10:41:07.410 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-02-26T10:41:07.410 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw== 2024-02-26T10:41:07.410 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-02-26T10:41:07.410 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:41:07.410 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw pool=cephfs2_data" 2024-02-26T10:41:07.411 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.1 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6b806b500 msgr2=0x7fa6b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6b806b500 0x7fa6b806d9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc036000 tx=0x7fa6cc03d040 comp rx=0 tx=0).stop 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 msgr2=0x7fa6d8131c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8131c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b420 tx=0x7fa6c400b8e0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 shutdown_connections 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6d8152ea0 0x7fa6d81edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6d813c000 0x7fa6d8131c80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6b806b500 0x7fa6b806d9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 --2- 172.21.15.67:0/3783582598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d806e460 0x7fa6d80c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.413+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 >> 172.21.15.67:0/3783582598 conn(0x7fa6d80a84a0 msgr2=0x7fa6d8072b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:07.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.414+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 shutdown_connections 2024-02-26T10:41:07.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.414+0000 7fa6df200640 1 -- 172.21.15.67:0/3783582598 wait complete. 2024-02-26T10:41:07.433 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:07.433 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:07.493 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:07.493 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:07.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.767+0000 7f6097200640 1 Processor -- start 2024-02-26T10:41:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.768+0000 7f6097200640 1 -- start start 2024-02-26T10:41:07.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.769+0000 7f6097200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6098158d80 0x7f60980c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.769+0000 7f6097200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6098155840 con 0x7f60980c54e0 2024-02-26T10:41:07.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.769+0000 7f6097200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6098155b20 con 0x7f6098152460 2024-02-26T10:41:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.769+0000 7f6097200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6098158330 con 0x7f6098158d80 2024-02-26T10:41:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.769+0000 7f6096800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f60980c54e0 0x7f6098154d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37446/0 (socket says 172.21.15.67:37446) 2024-02-26T10:41:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.769+0000 7f6096800640 1 -- 172.21.15.67:0/2158371230 learned_addr learned my addr 172.21.15.67:0/2158371230 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6095400640 1 --2- 172.21.15.67:0/2158371230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6098158d80 0x7f60980c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620269648 0 0) 0x7f6098155840 con 0x7f60980c54e0 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c003520 con 0x7f60980c54e0 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337919985 0 0) 0x7f6098155b20 con 0x7f6098152460 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6098155840 con 0x7f6098152460 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886082395 0 0) 0x7f607c003520 con 0x7f60980c54e0 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6095400640 1 -- 172.21.15.67:0/2158371230 >> v1:172.21.15.82:6789/0 conn(0x7f6098152460 legacy=0x7f6098156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6095400640 1 -- 172.21.15.67:0/2158371230 >> v1:172.21.15.67:6789/0 conn(0x7f60980c54e0 legacy=0x7f6098154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.770+0000 7f6095400640 1 -- 172.21.15.67:0/2158371230 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6098158580 con 0x7f6098158d80 2024-02-26T10:41:07.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.772+0000 7f6095400640 1 --2- 172.21.15.67:0/2158371230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6098158d80 0x7f60980c4fa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6088009dc0 tx=0x7f608802bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e13d69d78a6aaa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.772+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088039070 con 0x7f6098158d80 2024-02-26T10:41:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608802cba0 con 0x7f6098158d80 2024-02-26T10:41:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6094a00640 1 -- 172.21.15.67:0/2158371230 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088034440 con 0x7f6098158d80 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 -- 172.21.15.67:0/2158371230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6098158d80 msgr2=0x7f60980c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 --2- 172.21.15.67:0/2158371230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6098158d80 0x7f60980c4fa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6088009dc0 tx=0x7f608802bdf0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 -- 172.21.15.67:0/2158371230 shutdown_connections 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 --2- 172.21.15.67:0/2158371230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6098158d80 0x7f60980c4fa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 -- 172.21.15.67:0/2158371230 >> 172.21.15.67:0/2158371230 conn(0x7f60980ac1b0 msgr2=0x7f60980ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 -- 172.21.15.67:0/2158371230 shutdown_connections 2024-02-26T10:41:07.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.773+0000 7f6097200640 1 -- 172.21.15.67:0/2158371230 wait complete. 2024-02-26T10:41:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.775+0000 7f6097200640 1 Processor -- start 2024-02-26T10:41:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.775+0000 7f6097200640 1 -- start start 2024-02-26T10:41:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6097200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60980c54e0 0x7f60980c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6097200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6098152460 0x7f60980c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6097200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 0x7f60980ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6097200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6098155490 con 0x7f6098158d80 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6097200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981588f0 con 0x7f6098152460 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6097200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60980ca0c0 con 0x7f60980c54e0 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6096800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 0x7f60980ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6096800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 0x7f60980ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54882/0 (socket says 172.21.15.67:54882) 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.776+0000 7f6096800640 1 -- 172.21.15.67:0/224598755 learned_addr learned my addr 172.21.15.67:0/224598755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6095e00640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60980c54e0 0x7f60980c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6095400640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6098152460 0x7f60980c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6096800640 1 -- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60980c54e0 msgr2=0x7f60980c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6096800640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60980c54e0 0x7f60980c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6096800640 1 -- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6098152460 msgr2=0x7f60980c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6096800640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6098152460 0x7f60980c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6096800640 1 -- 172.21.15.67:0/224598755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980c8ae0 con 0x7f6098158d80 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f6096800640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 0x7f60980ca9b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f609000c440 tx=0x7f609000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090017070 con 0x7f6098158d80 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090005d00 con 0x7f6098158d80 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.777+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090007310 con 0x7f6098158d80 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.778+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60980c8dc0 con 0x7f6098158d80 2024-02-26T10:41:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.778+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60980c9380 con 0x7f6098158d80 2024-02-26T10:41:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.779+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098074a30 con 0x7f6098158d80 2024-02-26T10:41:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.780+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f609001a070 con 0x7f6098158d80 2024-02-26T10:41:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.781+0000 7f608e000640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f607406b520 0x7f607406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.781+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f6090089870 con 0x7f6098158d80 2024-02-26T10:41:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.781+0000 7f6095e00640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f607406b520 0x7f607406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:07.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.782+0000 7f6095e00640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f607406b520 0x7f607406d9e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f60800011e0 tx=0x7f6080009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:07.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.785+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6090058400 con 0x7f6098158d80 2024-02-26T10:41:07.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.984+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f609806d8e0 con 0x7f6098158d80 2024-02-26T10:41:07.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.988+0000 7f608e000640 1 -- 172.21.15.67:0/224598755 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+20374 (secure 0 0 0) 0x7f6090057d00 con 0x7f6098158d80 2024-02-26T10:41:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:07.988 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":59,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:02.772898+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:07.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.990+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f607406b520 msgr2=0x7f607406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.990+0000 7f6097200640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f607406b520 0x7f607406d9e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f60800011e0 tx=0x7f6080009040 comp rx=0 tx=0).stop 2024-02-26T10:41:07.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 msgr2=0x7f60980ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 0x7f60980ca9b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f609000c440 tx=0x7f609000c900 comp rx=0 tx=0).stop 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 shutdown_connections 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6098158d80 0x7f60980ca9b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f607406b520 0x7f607406d9e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6098152460 0x7f60980c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 --2- 172.21.15.67:0/224598755 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60980c54e0 0x7f60980c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 >> 172.21.15.67:0/224598755 conn(0x7f60980ac1b0 msgr2=0x7f60980ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 shutdown_connections 2024-02-26T10:41:07.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:07.991+0000 7f6097200640 1 -- 172.21.15.67:0/224598755 wait complete. 2024-02-26T10:41:08.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:40:43.472687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:40:44.096497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:40:58.810192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:41:00.247724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:08.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:08.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.241+0000 7ff732a00640 1 Processor -- start 2024-02-26T10:41:08.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.242+0000 7ff732a00640 1 -- start start 2024-02-26T10:41:08.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.243+0000 7ff732a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff72c0af8a0 0x7ff72c0afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.243+0000 7ff732a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c06c110 con 0x7ff72c1562f0 2024-02-26T10:41:08.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.243+0000 7ff732a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c06d000 con 0x7ff72c0c5290 2024-02-26T10:41:08.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.243+0000 7ff732a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c0c72c0 con 0x7ff72c0af8a0 2024-02-26T10:41:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.244+0000 7ff730c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff72c1562f0 0x7ff72c06c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37452/0 (socket says 172.21.15.67:37452) 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.244+0000 7ff730c00640 1 -- 172.21.15.67:0/2323113264 learned_addr learned my addr 172.21.15.67:0/2323113264 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.244+0000 7ff731600640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff72c0c5290 0x7ff72c155880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:45936/0 (socket says 172.21.15.67:45936) 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.244+0000 7ff732000640 1 --2- 172.21.15.67:0/2323113264 >> v2:172.21.15.82:3300/0 conn(0x7ff72c0af8a0 0x7ff72c0afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.244+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108378356 0 0) 0x7ff72c06c110 con 0x7ff72c1562f0 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.244+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff714003520 con 0x7ff72c1562f0 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936576312 0 0) 0x7ff72c06d000 con 0x7ff72c0c5290 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff72c06c110 con 0x7ff72c0c5290 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577746303 0 0) 0x7ff714003520 con 0x7ff72c1562f0 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 >> v2:172.21.15.82:3300/0 conn(0x7ff72c0af8a0 msgr2=0x7ff72c0afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 --2- 172.21.15.67:0/2323113264 >> v2:172.21.15.82:3300/0 conn(0x7ff72c0af8a0 0x7ff72c0afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 >> v1:172.21.15.82:6789/0 conn(0x7ff72c0c5290 legacy=0x7ff72c155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0c75a0 con 0x7ff72c1562f0 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ff71c001670 con 0x7ff72c1562f0 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 >> v1:172.21.15.67:6789/0 conn(0x7ff72c1562f0 legacy=0x7ff72c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff714003b40 0x7ff714003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff714004800 0x7ff714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c72c0 con 0x7ff7140051c0 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71400c0f0 con 0x7ff714003b40 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71400c2f0 con 0x7ff714004800 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.245+0000 7ff732000640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff731600640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff714003b40 0x7ff714003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff730c00640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff714004800 0x7ff714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff732000640 1 -- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff714004800 msgr2=0x7ff714004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff732000640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff714004800 0x7ff714004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff732000640 1 -- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff714003b40 msgr2=0x7ff714003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff732000640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff714003b40 0x7ff714003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.246+0000 7ff732000640 1 -- 172.21.15.67:0/2323113264 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff714003520 con 0x7ff7140051c0 2024-02-26T10:41:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.247+0000 7ff732000640 1 -- 172.21.15.67:0/2323113264 reap_dead start 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.247+0000 7ff732000640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff724009fc0 tx=0x7ff714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e8f7265fd3f50f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.247+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724009c10 con 0x7ff7140051c0 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.247+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72402cca0 con 0x7ff7140051c0 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.247+0000 7ff72be00640 1 -- 172.21.15.67:0/2323113264 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724003bd0 con 0x7ff7140051c0 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 -- 172.21.15.67:0/2323113264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 msgr2=0x7ff71400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff724009fc0 tx=0x7ff714005830 comp rx=0 tx=0).stop 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 -- 172.21.15.67:0/2323113264 shutdown_connections 2024-02-26T10:41:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 --2- 172.21.15.67:0/2323113264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff71400b9c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 -- 172.21.15.67:0/2323113264 >> 172.21.15.67:0/2323113264 conn(0x7ff72c0ac230 msgr2=0x7ff72c0ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 -- 172.21.15.67:0/2323113264 shutdown_connections 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.248+0000 7ff732a00640 1 -- 172.21.15.67:0/2323113264 wait complete. 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.250+0000 7ff732a00640 1 Processor -- start 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.250+0000 7ff732a00640 1 -- start start 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff72c0c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff72c0c9b10 0x7ff72c1eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff72c0c8df0 0x7ff72c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c5670 con 0x7ff7140051c0 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c06c3d0 con 0x7ff72c0c8df0 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c57f0 con 0x7ff72c0c9b10 2024-02-26T10:41:08.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff731600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff72c0c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff731600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff72c0c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54904/0 (socket says 172.21.15.67:54904) 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff731600640 1 -- 172.21.15.67:0/2160788282 learned_addr learned my addr 172.21.15.67:0/2160788282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff730c00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff72c0c9b10 0x7ff72c1eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.251+0000 7ff732000640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff72c0c8df0 0x7ff72c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff731600640 1 -- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff72c0c9b10 msgr2=0x7ff72c1eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff731600640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff72c0c9b10 0x7ff72c1eb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff731600640 1 -- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff72c0c8df0 msgr2=0x7ff72c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff731600640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff72c0c8df0 0x7ff72c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff731600640 1 -- 172.21.15.67:0/2160788282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c1eb690 con 0x7ff7140051c0 2024-02-26T10:41:08.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff731600640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff72c0c2090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff720001650 tx=0x7ff720001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000e740 con 0x7ff7140051c0 2024-02-26T10:41:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720018070 con 0x7ff7140051c0 2024-02-26T10:41:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c0ca0b0 con 0x7ff7140051c0 2024-02-26T10:41:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000eb40 con 0x7ff7140051c0 2024-02-26T10:41:08.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.252+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c0c85f0 con 0x7ff7140051c0 2024-02-26T10:41:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.253+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c074a30 con 0x7ff7140051c0 2024-02-26T10:41:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.254+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff72000ece0 con 0x7ff7140051c0 2024-02-26T10:41:08.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.256+0000 7ff729600640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff71006b4d0 0x7ff71006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.257+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff72008a330 con 0x7ff7140051c0 2024-02-26T10:41:08.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.257+0000 7ff730c00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff71006b4d0 0x7ff71006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.258+0000 7ff730c00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff71006b4d0 0x7ff71006d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0aa750 tx=0x7ff71c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:08.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.261+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff72c1eb690 con 0x7ff7140051c0 2024-02-26T10:41:08.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.460+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff72c06d950 con 0x7ff7140051c0 2024-02-26T10:41:08.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.461+0000 7ff729600640 1 -- 172.21.15.67:0/2160788282 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff720058fa0 con 0x7ff7140051c0 2024-02-26T10:41:08.460 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff71006b4d0 msgr2=0x7ff71006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff71006b4d0 0x7ff71006d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0aa750 tx=0x7ff71c009040 comp rx=0 tx=0).stop 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 msgr2=0x7ff72c0c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff72c0c2090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff720001650 tx=0x7ff720001b10 comp rx=0 tx=0).stop 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 shutdown_connections 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff71006b4d0 0x7ff71006d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff72c0c8df0 0x7ff72c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff72c0c9b10 0x7ff72c1eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 --2- 172.21.15.67:0/2160788282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7140051c0 0x7ff72c0c2090 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.464+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 >> 172.21.15.67:0/2160788282 conn(0x7ff72c0ac230 msgr2=0x7ff72c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.465+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 shutdown_connections 2024-02-26T10:41:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.465+0000 7ff732a00640 1 -- 172.21.15.67:0/2160788282 wait complete. 2024-02-26T10:41:08.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:41:08.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.711+0000 7faeae800640 1 Processor -- start 2024-02-26T10:41:08.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.712+0000 7faeae800640 1 -- start start 2024-02-26T10:41:08.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.713+0000 7faeae800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faea80c25a0 0x7faea80c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.713+0000 7faeae800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea8152470 con 0x7faea8151980 2024-02-26T10:41:08.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.713+0000 7faeae800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea80c1be0 con 0x7faea80c1eb0 2024-02-26T10:41:08.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.713+0000 7faeae800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea80c1750 con 0x7faea80c25a0 2024-02-26T10:41:08.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.714+0000 7faeade00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faea8151980 0x7faea8151d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37462/0 (socket says 172.21.15.67:37462) 2024-02-26T10:41:08.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.714+0000 7faeade00640 1 -- 172.21.15.67:0/2613669995 learned_addr learned my addr 172.21.15.67:0/2613669995 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:08.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.714+0000 7faead400640 1 --2- 172.21.15.67:0/2613669995 >> v2:172.21.15.82:3300/0 conn(0x7faea80c25a0 0x7faea80c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.714+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353664194 0 0) 0x7faea8152470 con 0x7faea8151980 2024-02-26T10:41:08.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.714+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae90003520 con 0x7faea8151980 2024-02-26T10:41:08.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567630936 0 0) 0x7fae90003520 con 0x7faea8151980 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 >> v2:172.21.15.82:3300/0 conn(0x7faea80c25a0 msgr2=0x7faea80c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 --2- 172.21.15.67:0/2613669995 >> v2:172.21.15.82:3300/0 conn(0x7faea80c25a0 0x7faea80c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 >> v1:172.21.15.82:6789/0 conn(0x7faea80c1eb0 legacy=0x7faea814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80c3660 con 0x7faea8151980 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7faea00019f0 con 0x7faea8151980 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 >> v1:172.21.15.67:6789/0 conn(0x7faea8151980 legacy=0x7faea8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae90003a00 0x7fae90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae90004800 0x7fae90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90005210 0x7fae9000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c1750 con 0x7fae90004800 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9000e370 con 0x7fae90005210 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9000e570 con 0x7fae90003a00 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.715+0000 7faeaca00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae90004800 0x7fae90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeade00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90005210 0x7fae9000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faead400640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae90003a00 0x7fae90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeaca00640 1 -- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae90003a00 msgr2=0x7fae90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeaca00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae90003a00 0x7fae90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeaca00640 1 -- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90005210 msgr2=0x7fae9000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeaca00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90005210 0x7fae9000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeaca00640 1 -- 172.21.15.67:0/2613669995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae90003520 con 0x7fae90004800 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faeade00640 1 -- 172.21.15.67:0/2613669995 reap_dead start 2024-02-26T10:41:08.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.716+0000 7faead400640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae90003a00 0x7fae90003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.717+0000 7faeaca00640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae90004800 0x7fae90004c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faea80c2300 tx=0x7fae90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edf420ecb5778b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.717+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94001370 con 0x7fae90004800 2024-02-26T10:41:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.717+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae940052d0 con 0x7fae90004800 2024-02-26T10:41:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.717+0000 7faea7e00640 1 -- 172.21.15.67:0/2613669995 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94003830 con 0x7fae90004800 2024-02-26T10:41:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 -- 172.21.15.67:0/2613669995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae90004800 msgr2=0x7fae90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae90004800 0x7fae90004c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faea80c2300 tx=0x7fae90005830 comp rx=0 tx=0).stop 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 -- 172.21.15.67:0/2613669995 shutdown_connections 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 --2- 172.21.15.67:0/2613669995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae90004800 0x7fae90004c80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 -- 172.21.15.67:0/2613669995 >> 172.21.15.67:0/2613669995 conn(0x7faea80c6380 msgr2=0x7faea80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 -- 172.21.15.67:0/2613669995 shutdown_connections 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.718+0000 7faeae800640 1 -- 172.21.15.67:0/2613669995 wait complete. 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.720+0000 7faeae800640 1 Processor -- start 2024-02-26T10:41:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.720+0000 7faeae800640 1 -- start start 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeae800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90004800 0x7faea814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeae800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 0x7faea80bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeae800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faea81ee740 0x7faea8142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeae800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c13e0 con 0x7faea81eb2e0 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeae800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c1100 con 0x7fae90004800 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeae800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c2980 con 0x7faea81ee740 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeaca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 0x7faea80bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeaca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 0x7faea80bfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54918/0 (socket says 172.21.15.67:54918) 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeaca00640 1 -- 172.21.15.67:0/470196577 learned_addr learned my addr 172.21.15.67:0/470196577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:08.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faead400640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90004800 0x7faea814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.721+0000 7faeade00640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faea81ee740 0x7faea8142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeaca00640 1 -- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faea81ee740 msgr2=0x7faea8142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeaca00640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faea81ee740 0x7faea8142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeaca00640 1 -- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90004800 msgr2=0x7faea814c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeaca00640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90004800 0x7faea814c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeaca00640 1 -- 172.21.15.67:0/470196577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80c4310 con 0x7faea81eb2e0 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeade00640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faea81ee740 0x7faea8142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:08.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeaca00640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 0x7faea80bfa20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fae94001810 tx=0x7fae94001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94007a30 con 0x7faea81eb2e0 2024-02-26T10:41:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae94042070 con 0x7faea81eb2e0 2024-02-26T10:41:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94001370 con 0x7faea81eb2e0 2024-02-26T10:41:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea80c45b0 con 0x7faea81eb2e0 2024-02-26T10:41:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.722+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea81edf40 con 0x7faea81eb2e0 2024-02-26T10:41:08.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.724+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea80678e0 con 0x7faea81eb2e0 2024-02-26T10:41:08.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.725+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae9403b040 con 0x7faea81eb2e0 2024-02-26T10:41:08.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.725+0000 7faea5600640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae8806b500 0x7fae8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:08.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.725+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fae940abb00 con 0x7faea81eb2e0 2024-02-26T10:41:08.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.726+0000 7faead400640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae8806b500 0x7fae8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:08.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.727+0000 7faead400640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae8806b500 0x7fae8806d9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faea80c0420 tx=0x7fae9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:08.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.730+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae9407a610 con 0x7faea81eb2e0 2024-02-26T10:41:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.931+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faea8067fb0 con 0x7faea81eb2e0 2024-02-26T10:41:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.932+0000 7faea5600640 1 -- 172.21.15.67:0/470196577 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae940abe20 con 0x7faea81eb2e0 2024-02-26T10:41:08.932 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:08.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae8806b500 msgr2=0x7fae8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae8806b500 0x7fae8806d9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faea80c0420 tx=0x7fae9c008040 comp rx=0 tx=0).stop 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 msgr2=0x7faea80bfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 0x7faea80bfa20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fae94001810 tx=0x7fae94001120 comp rx=0 tx=0).stop 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 shutdown_connections 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faea81ee740 0x7faea8142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faea81eb2e0 0x7faea80bfa20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fae8806b500 0x7fae8806d9c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 --2- 172.21.15.67:0/470196577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae90004800 0x7faea814c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 >> 172.21.15.67:0/470196577 conn(0x7faea80c6380 msgr2=0x7faea814be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 shutdown_connections 2024-02-26T10:41:08.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:08.935+0000 7faeae800640 1 -- 172.21.15.67:0/470196577 wait complete. 2024-02-26T10:41:08.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:41:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.180+0000 7fe5c1c00640 1 Processor -- start 2024-02-26T10:41:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.180+0000 7fe5c1c00640 1 -- start start 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.181+0000 7fe5c1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe5bc0c2950 0x7fe5bc1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.181+0000 7fe5c1c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc153920 0x7fe5bc15d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5c1c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc0a6eb0 con 0x7fe5bc1503d0 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5c1c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc153d90 con 0x7fe5bc0c2950 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5c1c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0a7190 con 0x7fe5bc153920 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe5bc0c2950 0x7fe5bc1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe5bc0c2950 0x7fe5bc1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54922/0 (socket says 172.21.15.67:54922) 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 -- 172.21.15.67:0/3045651417 learned_addr learned my addr 172.21.15.67:0/3045651417 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5bb400640 1 --2- 172.21.15.67:0/3045651417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc153920 0x7fe5bc15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 -- 172.21.15.67:0/3045651417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc153920 msgr2=0x7fe5bc15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 --2- 172.21.15.67:0/3045651417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc153920 0x7fe5bc15d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 -- 172.21.15.67:0/3045651417 >> v1:172.21.15.82:6789/0 conn(0x7fe5bc1503d0 legacy=0x7fe5bc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.182+0000 7fe5b3e00640 1 -- 172.21.15.67:0/3045651417 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc0a7470 con 0x7fe5bc0c2950 2024-02-26T10:41:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.184+0000 7fe5b3e00640 1 --2- 172.21.15.67:0/3045651417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc0c2950 0x7fe5bc1533e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009870 tx=0x7fe5a402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc9f1891ef9eee0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.184+0000 7fe5ba000640 1 -- 172.21.15.67:0/3045651417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4038070 con 0x7fe5bc0c2950 2024-02-26T10:41:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.184+0000 7fe5ba000640 1 -- 172.21.15.67:0/3045651417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a4031740 con 0x7fe5bc0c2950 2024-02-26T10:41:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.184+0000 7fe5ba000640 1 -- 172.21.15.67:0/3045651417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a402f050 con 0x7fe5bc0c2950 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 -- 172.21.15.67:0/3045651417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc0c2950 msgr2=0x7fe5bc1533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/3045651417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc0c2950 0x7fe5bc1533e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009870 tx=0x7fe5a402b7e0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 -- 172.21.15.67:0/3045651417 shutdown_connections 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/3045651417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc153920 0x7fe5bc15d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/3045651417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc0c2950 0x7fe5bc1533e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 -- 172.21.15.67:0/3045651417 >> 172.21.15.67:0/3045651417 conn(0x7fe5bc0c6da0 msgr2=0x7fe5bc0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 -- 172.21.15.67:0/3045651417 shutdown_connections 2024-02-26T10:41:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.185+0000 7fe5c1c00640 1 -- 172.21.15.67:0/3045651417 wait complete. 2024-02-26T10:41:09.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.187+0000 7fe5c1c00640 1 Processor -- start 2024-02-26T10:41:09.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.187+0000 7fe5c1c00640 1 -- start start 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.187+0000 7fe5c1c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc0c2950 0x7fe5bc144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5c1c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5bc1503d0 0x7fe5bc0c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5c1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 0x7fe5bc0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5c1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc06ef00 con 0x7fe5bc153920 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5c1c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc152ec0 con 0x7fe5bc1503d0 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5c1c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc1507b0 con 0x7fe5bc0c2950 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 0x7fe5bc0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 0x7fe5bc0c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54936/0 (socket says 172.21.15.67:54936) 2024-02-26T10:41:09.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 -- 172.21.15.67:0/1412120802 learned_addr learned my addr 172.21.15.67:0/1412120802 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:09.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5b3e00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5bc1503d0 0x7fe5bc0c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5baa00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc0c2950 0x7fe5bc144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 -- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc0c2950 msgr2=0x7fe5bc144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc0c2950 0x7fe5bc144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 -- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5bc1503d0 msgr2=0x7fe5bc0c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5bc1503d0 0x7fe5bc0c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5bb400640 1 -- 172.21.15.67:0/1412120802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc0c4e50 con 0x7fe5bc153920 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.188+0000 7fe5baa00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc0c2950 0x7fe5bc144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.189+0000 7fe5bb400640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 0x7fe5bc0c5b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400cda0 tx=0x7fe5b400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.189+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4003b90 con 0x7fe5bc153920 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.189+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b4003d30 con 0x7fe5bc153920 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.189+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4004240 con 0x7fe5bc153920 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.189+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc1ec640 con 0x7fe5bc153920 2024-02-26T10:41:09.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.189+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc145510 con 0x7fe5bc153920 2024-02-26T10:41:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.190+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc074a30 con 0x7fe5bc153920 2024-02-26T10:41:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.191+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5b40043e0 con 0x7fe5bc153920 2024-02-26T10:41:09.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.192+0000 7fe5b2a00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe59806b500 0x7fe59806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.192+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fe5b408b070 con 0x7fe5bc153920 2024-02-26T10:41:09.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.193+0000 7fe5baa00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe59806b500 0x7fe59806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.194+0000 7fe5baa00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe59806b500 0x7fe59806d9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc145850 tx=0x7fe5ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:09.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.196+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5b4059b80 con 0x7fe5bc153920 2024-02-26T10:41:09.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.392+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5bc06ec50 con 0x7fe5bc153920 2024-02-26T10:41:09.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.393+0000 7fe5b2a00640 1 -- 172.21.15.67:0/1412120802 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5b4059480 con 0x7fe5bc153920 2024-02-26T10:41:09.392 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:09.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe59806b500 msgr2=0x7fe59806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe59806b500 0x7fe59806d9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc145850 tx=0x7fe5ac009040 comp rx=0 tx=0).stop 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 msgr2=0x7fe5bc0c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 0x7fe5bc0c5b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400cda0 tx=0x7fe5b400a660 comp rx=0 tx=0).stop 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 shutdown_connections 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5bc153920 0x7fe5bc0c5b50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe59806b500 0x7fe59806d9c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5bc1503d0 0x7fe5bc0c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 --2- 172.21.15.67:0/1412120802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5bc0c2950 0x7fe5bc144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 >> 172.21.15.67:0/1412120802 conn(0x7fe5bc0c6da0 msgr2=0x7fe5bc0c2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 shutdown_connections 2024-02-26T10:41:09.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.396+0000 7fe5c1c00640 1 -- 172.21.15.67:0/1412120802 wait complete. 2024-02-26T10:41:09.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:41:09.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.627+0000 7f0c87400640 1 Processor -- start 2024-02-26T10:41:09.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.628+0000 7f0c87400640 1 -- start start 2024-02-26T10:41:09.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.629+0000 7f0c87400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800a7d90 0x7f0c800c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.629+0000 7f0c87400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c801541c0 0x7f0c801562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.629+0000 7f0c87400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8006f020 con 0x7f0c800c3450 2024-02-26T10:41:09.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.629+0000 7f0c87400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c80157290 con 0x7f0c801541c0 2024-02-26T10:41:09.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.629+0000 7f0c87400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c6db0 con 0x7f0c800a7d90 2024-02-26T10:41:09.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.630+0000 7f0c84a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800a7d90 0x7f0c800c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.630+0000 7f0c7fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0c800c3450 0x7f0c80070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:54698/0 (socket says 172.21.15.67:54698) 2024-02-26T10:41:09.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.630+0000 7f0c7fe00640 1 -- 172.21.15.67:0/4104822613 learned_addr learned my addr 172.21.15.67:0/4104822613 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:09.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.630+0000 7f0c85400640 1 --2- 172.21.15.67:0/4104822613 >> v2:172.21.15.82:3300/0 conn(0x7f0c801541c0 0x7f0c801562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.631+0000 7f0c84a00640 1 -- 172.21.15.67:0/4104822613 >> v2:172.21.15.82:3300/0 conn(0x7f0c801541c0 msgr2=0x7f0c801562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:09.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.631+0000 7f0c84a00640 1 --2- 172.21.15.67:0/4104822613 >> v2:172.21.15.82:3300/0 conn(0x7f0c801541c0 0x7f0c801562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.631+0000 7f0c84a00640 1 -- 172.21.15.67:0/4104822613 >> v1:172.21.15.82:6789/0 conn(0x7f0c800c3450 legacy=0x7f0c80070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.631+0000 7f0c84a00640 1 -- 172.21.15.67:0/4104822613 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80156fb0 con 0x7f0c800a7d90 2024-02-26T10:41:09.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.631+0000 7f0c85400640 1 --2- 172.21.15.67:0/4104822613 >> v2:172.21.15.82:3300/0 conn(0x7f0c801541c0 0x7f0c801562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:41:09.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.633+0000 7f0c84a00640 1 --2- 172.21.15.67:0/4104822613 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800a7d90 0x7f0c800c7290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c740097a0 tx=0x7f0c7402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b185587c91370 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:09.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.634+0000 7f0c7f400640 1 -- 172.21.15.67:0/4104822613 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74035020 con 0x7f0c800a7d90 2024-02-26T10:41:09.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.634+0000 7f0c7f400640 1 -- 172.21.15.67:0/4104822613 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74031810 con 0x7f0c800a7d90 2024-02-26T10:41:09.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.634+0000 7f0c7f400640 1 -- 172.21.15.67:0/4104822613 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74031be0 con 0x7f0c800a7d90 2024-02-26T10:41:09.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 -- 172.21.15.67:0/4104822613 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800a7d90 msgr2=0x7f0c800c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 --2- 172.21.15.67:0/4104822613 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800a7d90 0x7f0c800c7290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c740097a0 tx=0x7f0c7402b8b0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 -- 172.21.15.67:0/4104822613 shutdown_connections 2024-02-26T10:41:09.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 --2- 172.21.15.67:0/4104822613 >> v2:172.21.15.82:3300/0 conn(0x7f0c801541c0 0x7f0c801562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 --2- 172.21.15.67:0/4104822613 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800a7d90 0x7f0c800c7290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 -- 172.21.15.67:0/4104822613 >> 172.21.15.67:0/4104822613 conn(0x7f0c800aa200 msgr2=0x7f0c800aa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:09.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 -- 172.21.15.67:0/4104822613 shutdown_connections 2024-02-26T10:41:09.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.635+0000 7f0c87400640 1 -- 172.21.15.67:0/4104822613 wait complete. 2024-02-26T10:41:09.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.637+0000 7f0c87400640 1 Processor -- start 2024-02-26T10:41:09.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.637+0000 7f0c87400640 1 -- start start 2024-02-26T10:41:09.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.638+0000 7f0c87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 0x7f0c801e0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.638+0000 7f0c87400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c801541c0 0x7f0c800c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.638+0000 7f0c87400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800c2590 0x7f0c8006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.638+0000 7f0c87400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80154670 con 0x7f0c800a7d90 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.638+0000 7f0c87400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8006f5a0 con 0x7f0c801541c0 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.638+0000 7f0c87400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c801547f0 con 0x7f0c800c2590 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 0x7f0c801e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 0x7f0c801e0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41168/0 (socket says 172.21.15.67:41168) 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 -- 172.21.15.67:0/3731606417 learned_addr learned my addr 172.21.15.67:0/3731606417 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c85400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800c2590 0x7f0c8006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 -- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800c2590 msgr2=0x7f0c8006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800c2590 0x7f0c8006cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 -- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c801541c0 msgr2=0x7f0c800c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c801541c0 0x7f0c800c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.639+0000 7f0c84a00640 1 -- 172.21.15.67:0/3731606417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800ad980 con 0x7f0c800a7d90 2024-02-26T10:41:09.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.640+0000 7f0c84a00640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 0x7f0c801e0940 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0c740098c0 tx=0x7f0c740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:09.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.640+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74041070 con 0x7f0c800a7d90 2024-02-26T10:41:09.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.640+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7403ad00 con 0x7f0c800a7d90 2024-02-26T10:41:09.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.640+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8014c3a0 con 0x7f0c800a7d90 2024-02-26T10:41:09.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.640+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74035040 con 0x7f0c800a7d90 2024-02-26T10:41:09.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.640+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c801eea10 con 0x7f0c800a7d90 2024-02-26T10:41:09.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.642+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c80074a30 con 0x7f0c800a7d90 2024-02-26T10:41:09.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.642+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c7404c060 con 0x7f0c800a7d90 2024-02-26T10:41:09.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.644+0000 7f0c7cc00640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c6406b4d0 0x7f0c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:09.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.644+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f0c740ac1c0 con 0x7f0c800a7d90 2024-02-26T10:41:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.645+0000 7f0c7fe00640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c6406b4d0 0x7f0c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:09.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.646+0000 7f0c7fe00640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c6406b4d0 0x7f0c6406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0c70000f60 tx=0x7f0c70007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.649+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c7407ac40 con 0x7f0c800a7d90 2024-02-26T10:41:09.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.853+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c800c7d10 con 0x7f0c800a7d90 2024-02-26T10:41:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.854+0000 7f0c7cc00640 1 -- 172.21.15.67:0/3731606417 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v59) v1 ==== 105+0+11 (secure 0 0 0) 0x7f0c7407a540 con 0x7f0c800a7d90 2024-02-26T10:41:09.853 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c6406b4d0 msgr2=0x7f0c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c6406b4d0 0x7f0c6406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0c70000f60 tx=0x7f0c70007a80 comp rx=0 tx=0).stop 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 msgr2=0x7f0c801e0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 0x7f0c801e0940 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0c740098c0 tx=0x7f0c740046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 shutdown_connections 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c800c2590 0x7f0c8006cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0c6406b4d0 0x7f0c6406d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c801541c0 0x7f0c800c1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 --2- 172.21.15.67:0/3731606417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c800a7d90 0x7f0c801e0940 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 >> 172.21.15.67:0/3731606417 conn(0x7f0c800aa200 msgr2=0x7f0c800a94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 shutdown_connections 2024-02-26T10:41:09.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:09.857+0000 7f0c87400640 1 -- 172.21.15.67:0/3731606417 wait complete. 2024-02-26T10:41:09.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:41:10.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.098+0000 7f151b200640 1 Processor -- start 2024-02-26T10:41:10.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.098+0000 7f151b200640 1 -- start start 2024-02-26T10:41:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f151b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15140c84c0 0x7f1514155720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f151b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15140c7150 con 0x7f151406c420 2024-02-26T10:41:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f151b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15140c7430 con 0x7f15141523c0 2024-02-26T10:41:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f151b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140c7a80 con 0x7f15140c84c0 2024-02-26T10:41:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f151a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f151406c420 0x7f151406c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36798/0 (socket says 172.21.15.67:36798) 2024-02-26T10:41:10.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f151a800640 1 -- 172.21.15.67:0/2145021557 learned_addr learned my addr 172.21.15.67:0/2145021557 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.100+0000 7f1519400640 1 --2- 172.21.15.67:0/2145021557 >> v2:172.21.15.82:3300/0 conn(0x7f15140c84c0 0x7f1514155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854919645 0 0) 0x7f15140c7150 con 0x7f151406c420 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1500003520 con 0x7f151406c420 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155737094 0 0) 0x7f1500003520 con 0x7f151406c420 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 >> v2:172.21.15.82:3300/0 conn(0x7f15140c84c0 msgr2=0x7f1514155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 --2- 172.21.15.67:0/2145021557 >> v2:172.21.15.82:3300/0 conn(0x7f15140c84c0 0x7f1514155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 >> v1:172.21.15.82:6789/0 conn(0x7f15141523c0 legacy=0x7f15140c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15140c7710 con 0x7f151406c420 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1510001670 con 0x7f151406c420 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 >> v1:172.21.15.67:6789/0 conn(0x7f151406c420 legacy=0x7f151406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500003a00 0x7f1500003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500004800 0x7f1500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000052e0 0x7f150000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150000c1d0 con 0x7f15000052e0 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150000c380 con 0x7f1500003a00 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.101+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150000c580 con 0x7f1500004800 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f151a800640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000052e0 0x7f150000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f1519e00640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500003a00 0x7f1500003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f1519400640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500004800 0x7f1500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f151a800640 1 -- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500004800 msgr2=0x7f1500004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f151a800640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1500004800 0x7f1500004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f151a800640 1 -- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500003a00 msgr2=0x7f1500003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f151a800640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500003a00 0x7f1500003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.102+0000 7f151a800640 1 -- 172.21.15.67:0/2145021557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500003520 con 0x7f15000052e0 2024-02-26T10:41:10.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.103+0000 7f151a800640 1 -- 172.21.15.67:0/2145021557 reap_dead start 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.103+0000 7f151a800640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000052e0 0x7f150000b9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f151000c0e0 tx=0x7f1500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e505361dde5ae43 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510007c20 con 0x7f15000052e0 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510030070 con 0x7f15000052e0 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f1518a00640 1 -- 172.21.15.67:0/2145021557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510005d30 con 0x7f15000052e0 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 -- 172.21.15.67:0/2145021557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000052e0 msgr2=0x7f150000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000052e0 0x7f150000b9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f151000c0e0 tx=0x7f1500005830 comp rx=0 tx=0).stop 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 -- 172.21.15.67:0/2145021557 shutdown_connections 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 --2- 172.21.15.67:0/2145021557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15000052e0 0x7f150000b9c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 -- 172.21.15.67:0/2145021557 >> 172.21.15.67:0/2145021557 conn(0x7f15140ac130 msgr2=0x7f15140ac540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 -- 172.21.15.67:0/2145021557 shutdown_connections 2024-02-26T10:41:10.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.104+0000 7f151b200640 1 -- 172.21.15.67:0/2145021557 wait complete. 2024-02-26T10:41:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.106+0000 7f151b200640 1 Processor -- start 2024-02-26T10:41:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.106+0000 7f151b200640 1 -- start start 2024-02-26T10:41:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000052e0 0x7f1514150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15140ad730 0x7f1514147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 0x7f1514147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15141558a0 con 0x7f15140afea0 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514155720 con 0x7f15140ad730 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1514155a20 con 0x7f15000052e0 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 0x7f1514147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 0x7f1514147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41196/0 (socket says 172.21.15.67:41196) 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.107+0000 7f151a800640 1 -- 172.21.15.67:0/2964855209 learned_addr learned my addr 172.21.15.67:0/2964855209 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f1519e00640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000052e0 0x7f1514150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f1519400640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15140ad730 0x7f1514147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f151a800640 1 -- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000052e0 msgr2=0x7f1514150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f151a800640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000052e0 0x7f1514150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f151a800640 1 -- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15140ad730 msgr2=0x7f1514147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f151a800640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15140ad730 0x7f1514147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f151a800640 1 -- 172.21.15.67:0/2964855209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15140adbe0 con 0x7f15140afea0 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f1519e00640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000052e0 0x7f1514150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:10.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f151a800640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 0x7f1514147d50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f151000c200 tx=0x7f151406d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:10.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510004bf0 con 0x7f15140afea0 2024-02-26T10:41:10.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510000ed0 con 0x7f15140afea0 2024-02-26T10:41:10.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.108+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510007070 con 0x7f15140afea0 2024-02-26T10:41:10.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.109+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15140ad450 con 0x7f15140afea0 2024-02-26T10:41:10.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.109+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15141513e0 con 0x7f15140afea0 2024-02-26T10:41:10.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.110+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1514074a30 con 0x7f15140afea0 2024-02-26T10:41:10.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.111+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1510007210 con 0x7f15140afea0 2024-02-26T10:41:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.112+0000 7f150a000640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14f806b500 0x7f14f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.112+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f15100ab1b0 con 0x7f15140afea0 2024-02-26T10:41:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.112+0000 7f1519e00640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14f806b500 0x7f14f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.114+0000 7f1519e00640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14f806b500 0x7f14f806d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15140c6940 tx=0x7f150c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:10.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.117+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1510048070 con 0x7f15140afea0 2024-02-26T10:41:10.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.318+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f151414a540 con 0x7f15140afea0 2024-02-26T10:41:10.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.319+0000 7f150a000640 1 -- 172.21.15.67:0/2964855209 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v59) v1 ==== 101+0+11 (secure 0 0 0) 0x7f1510079cc0 con 0x7f15140afea0 2024-02-26T10:41:10.318 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:10.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14f806b500 msgr2=0x7f14f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14f806b500 0x7f14f806d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15140c6940 tx=0x7f150c009040 comp rx=0 tx=0).stop 2024-02-26T10:41:10.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 msgr2=0x7f1514147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 0x7f1514147d50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f151000c200 tx=0x7f151406d550 comp rx=0 tx=0).stop 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 shutdown_connections 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15140afea0 0x7f1514147d50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f14f806b500 0x7f14f806d9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15140ad730 0x7f1514147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 --2- 172.21.15.67:0/2964855209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15000052e0 0x7f1514150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 >> 172.21.15.67:0/2964855209 conn(0x7f15140ac130 msgr2=0x7f1514153780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 shutdown_connections 2024-02-26T10:41:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.322+0000 7f151b200640 1 -- 172.21.15.67:0/2964855209 wait complete. 2024-02-26T10:41:10.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:41:10.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.559+0000 7f2f0da00640 1 Processor -- start 2024-02-26T10:41:10.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.560+0000 7f2f0da00640 1 -- start start 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f0da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f080c5820 0x7f2f08155a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f0da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2f0814ebd0 0x7f2f0814f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f0da00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f08155fb0 con 0x7f2f080c3730 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f0da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080c7100 con 0x7f2f0814ebd0 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f0da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f08158810 con 0x7f2f080c5820 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f07400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2f0814ebd0 0x7f2f0814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f06000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2f080c3730 0x7f2f08158ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36804/0 (socket says 172.21.15.67:36804) 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f06000640 1 -- 172.21.15.67:0/1923183780 learned_addr learned my addr 172.21.15.67:0/1923183780 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:10.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f07400640 1 --2- 172.21.15.67:0/1923183780 >> v2:172.21.15.67:3300/0 conn(0x7f2f0814ebd0 0x7f2f0814f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41202/0 (socket says 172.21.15.67:41202) 2024-02-26T10:41:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.561+0000 7f2f06a00640 1 --2- 172.21.15.67:0/1923183780 >> v2:172.21.15.82:3300/0 conn(0x7f2f080c5820 0x7f2f08155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.562+0000 7f2f07400640 1 -- 172.21.15.67:0/1923183780 >> v2:172.21.15.82:3300/0 conn(0x7f2f080c5820 msgr2=0x7f2f08155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.562+0000 7f2f07400640 1 --2- 172.21.15.67:0/1923183780 >> v2:172.21.15.82:3300/0 conn(0x7f2f080c5820 0x7f2f08155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.562+0000 7f2f07400640 1 -- 172.21.15.67:0/1923183780 >> v1:172.21.15.67:6789/0 conn(0x7f2f080c3730 legacy=0x7f2f08158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.562+0000 7f2f07400640 1 -- 172.21.15.67:0/1923183780 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080c6860 con 0x7f2f0814ebd0 2024-02-26T10:41:10.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.562+0000 7f2f05600640 1 -- 172.21.15.67:0/1923183780 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327134915 0 0) 0x7f2ef8001490 con 0x7f2f080c3730 2024-02-26T10:41:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.563+0000 7f2f07400640 1 --2- 172.21.15.67:0/1923183780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f0814ebd0 0x7f2f0814f000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2f080c85b0 tx=0x7f2f0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab5fe0b399d6a01 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:10.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.563+0000 7f2f05600640 1 -- 172.21.15.67:0/1923183780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00035020 con 0x7f2f0814ebd0 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.563+0000 7f2f05600640 1 -- 172.21.15.67:0/1923183780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00031830 con 0x7f2f0814ebd0 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.563+0000 7f2f05600640 1 -- 172.21.15.67:0/1923183780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00031c00 con 0x7f2f0814ebd0 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 -- 172.21.15.67:0/1923183780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f0814ebd0 msgr2=0x7f2f0814f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 --2- 172.21.15.67:0/1923183780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f0814ebd0 0x7f2f0814f000 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2f080c85b0 tx=0x7f2f0002b8d0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 -- 172.21.15.67:0/1923183780 shutdown_connections 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 --2- 172.21.15.67:0/1923183780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f0814ebd0 0x7f2f0814f000 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 --2- 172.21.15.67:0/1923183780 >> v2:172.21.15.82:3300/0 conn(0x7f2f080c5820 0x7f2f08155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 -- 172.21.15.67:0/1923183780 >> 172.21.15.67:0/1923183780 conn(0x7f2f0806b040 msgr2=0x7f2f0806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 -- 172.21.15.67:0/1923183780 shutdown_connections 2024-02-26T10:41:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.564+0000 7f2f0da00640 1 -- 172.21.15.67:0/1923183780 wait complete. 2024-02-26T10:41:10.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.566+0000 7f2f0da00640 1 Processor -- start 2024-02-26T10:41:10.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.566+0000 7f2f0da00640 1 -- start start 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f0da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 0x7f2f081eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f0da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f080c5820 0x7f2f080c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f0da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f0814ebd0 0x7f2f081edb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f0da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c6530 con 0x7f2f080c3730 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f0da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c6e20 con 0x7f2f080c5820 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f0da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080a7fc0 con 0x7f2f0814ebd0 2024-02-26T10:41:10.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f06a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 0x7f2f081eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f06a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 0x7f2f081eaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41218/0 (socket says 172.21.15.67:41218) 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.567+0000 7f2f06a00640 1 -- 172.21.15.67:0/367453096 learned_addr learned my addr 172.21.15.67:0/367453096 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06000640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f080c5820 0x7f2f080c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f07400640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f0814ebd0 0x7f2f081edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06a00640 1 -- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f0814ebd0 msgr2=0x7f2f081edb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06a00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f0814ebd0 0x7f2f081edb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06a00640 1 -- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f080c5820 msgr2=0x7f2f080c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06a00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f080c5820 0x7f2f080c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06a00640 1 -- 172.21.15.67:0/367453096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080ad090 con 0x7f2f080c3730 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f07400640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f0814ebd0 0x7f2f081edb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:10.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f06a00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 0x7f2f081eaff0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000d830 tx=0x7f2ef000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:10.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000b6c0 con 0x7f2f080c3730 2024-02-26T10:41:10.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef0019070 con 0x7f2f080c3730 2024-02-26T10:41:10.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000ba80 con 0x7f2f080c3730 2024-02-26T10:41:10.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.568+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f080ad370 con 0x7f2f080c3730 2024-02-26T10:41:10.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.569+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08070040 con 0x7f2f080c3730 2024-02-26T10:41:10.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.570+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f08074a30 con 0x7f2f080c3730 2024-02-26T10:41:10.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.571+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ef001e460 con 0x7f2f080c3730 2024-02-26T10:41:10.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.571+0000 7f2efea00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ee406b4d0 0x7f2ee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:10.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.571+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f2ef008ac20 con 0x7f2f080c3730 2024-02-26T10:41:10.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.572+0000 7f2f06000640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ee406b4d0 0x7f2ee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:10.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.573+0000 7f2f06000640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ee406b4d0 0x7f2ee406d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8005fd0 tx=0x7f2ef8007cf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:10.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.576+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ef0059730 con 0x7f2f080c3730 2024-02-26T10:41:10.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.788+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2f080c3010 con 0x7f2f080c3730 2024-02-26T10:41:10.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.789+0000 7f2efea00640 1 -- 172.21.15.67:0/367453096 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ef0059030 con 0x7f2f080c3730 2024-02-26T10:41:10.788 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ee406b4d0 msgr2=0x7f2ee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ee406b4d0 0x7f2ee406d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8005fd0 tx=0x7f2ef8007cf0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 msgr2=0x7f2f081eaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 0x7f2f081eaff0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000d830 tx=0x7f2ef000dcf0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 shutdown_connections 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f0814ebd0 0x7f2f081edb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ee406b4d0 0x7f2ee406d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f080c5820 0x7f2f080c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 --2- 172.21.15.67:0/367453096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f080c3730 0x7f2f081eaff0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:10.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 >> 172.21.15.67:0/367453096 conn(0x7f2f0806b040 msgr2=0x7f2f080c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:10.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 shutdown_connections 2024-02-26T10:41:10.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:10.792+0000 7f2f0da00640 1 -- 172.21.15.67:0/367453096 wait complete. 2024-02-26T10:41:10.812 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:41:10.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:10.812 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:41:10.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:41:10.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:10.916 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:41:10.961 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:10.961 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:11.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:41:11.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:41:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.288+0000 7f4a86000640 1 Processor -- start 2024-02-26T10:41:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.289+0000 7f4a86000640 1 -- start start 2024-02-26T10:41:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a86000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a8014ee60 0x7f4a801557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a86000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a800c8570 0x7f4a800c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a86000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4a800bdd90 0x7f4a8006c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a86000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8014f240 con 0x7f4a800c8570 2024-02-26T10:41:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a86000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8006c3f0 con 0x7f4a800bdd90 2024-02-26T10:41:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a86000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a80155ce0 con 0x7f4a8014ee60 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a7fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a800c8570 0x7f4a800c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.290+0000 7f4a7fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a800c8570 0x7f4a800c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41230/0 (socket says 172.21.15.67:41230) 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a7fe00640 1 -- 172.21.15.67:0/2396063082 learned_addr learned my addr 172.21.15.67:0/2396063082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a84c00640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a8014ee60 0x7f4a801557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a85600640 1 --2- 172.21.15.67:0/2396063082 >> v2:172.21.15.82:3300/0 conn(0x7f4a800bdd90 0x7f4a8006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a7fe00640 1 -- 172.21.15.67:0/2396063082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a8014ee60 msgr2=0x7f4a801557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a7fe00640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a8014ee60 0x7f4a801557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a7fe00640 1 -- 172.21.15.67:0/2396063082 >> v2:172.21.15.82:3300/0 conn(0x7f4a800bdd90 msgr2=0x7f4a8006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a7fe00640 1 --2- 172.21.15.67:0/2396063082 >> v2:172.21.15.82:3300/0 conn(0x7f4a800bdd90 0x7f4a8006c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a7fe00640 1 -- 172.21.15.67:0/2396063082 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a800c5100 con 0x7f4a800c8570 2024-02-26T10:41:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.291+0000 7f4a84c00640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a8014ee60 0x7f4a801557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.292+0000 7f4a7fe00640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a800c8570 0x7f4a800c7080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4a800c16d0 tx=0x7f4a6c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f54c470cf106a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.292+0000 7f4a7f400640 1 -- 172.21.15.67:0/2396063082 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c038070 con 0x7f4a800c8570 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.292+0000 7f4a7f400640 1 -- 172.21.15.67:0/2396063082 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c02f050 con 0x7f4a800c8570 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.292+0000 7f4a7f400640 1 -- 172.21.15.67:0/2396063082 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c031970 con 0x7f4a800c8570 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 -- 172.21.15.67:0/2396063082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a800c8570 msgr2=0x7f4a800c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a800c8570 0x7f4a800c7080 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4a800c16d0 tx=0x7f4a6c02b7f0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 -- 172.21.15.67:0/2396063082 shutdown_connections 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 --2- 172.21.15.67:0/2396063082 >> v2:172.21.15.82:3300/0 conn(0x7f4a800bdd90 0x7f4a8006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a800c8570 0x7f4a800c7080 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 --2- 172.21.15.67:0/2396063082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a8014ee60 0x7f4a801557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 -- 172.21.15.67:0/2396063082 >> 172.21.15.67:0/2396063082 conn(0x7f4a80072ea0 msgr2=0x7f4a800ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 -- 172.21.15.67:0/2396063082 shutdown_connections 2024-02-26T10:41:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.293+0000 7f4a86000640 1 -- 172.21.15.67:0/2396063082 wait complete. 2024-02-26T10:41:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.295+0000 7f4a86000640 1 Processor -- start 2024-02-26T10:41:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.295+0000 7f4a86000640 1 -- start start 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a86000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a800bdd90 0x7f4a801584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a86000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a800c8570 0x7f4a800be320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a86000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 0x7f4a800be860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a86000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800c75c0 con 0x7f4a8014ee60 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a86000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800c56c0 con 0x7f4a800bdd90 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a86000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800c7740 con 0x7f4a800c8570 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a85600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 0x7f4a800be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a85600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 0x7f4a800be860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41234/0 (socket says 172.21.15.67:41234) 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.296+0000 7f4a85600640 1 -- 172.21.15.67:0/1589469666 learned_addr learned my addr 172.21.15.67:0/1589469666 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a84c00640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a800bdd90 0x7f4a801584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a7fe00640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a800c8570 0x7f4a800be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a85600640 1 -- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a800c8570 msgr2=0x7f4a800be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a85600640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a800c8570 0x7f4a800be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a85600640 1 -- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a800bdd90 msgr2=0x7f4a801584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a85600640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a800bdd90 0x7f4a801584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a85600640 1 -- 172.21.15.67:0/1589469666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a80158e60 con 0x7f4a8014ee60 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a85600640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 0x7f4a800be860 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c4c0 tx=0x7f4a7800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78015070 con 0x7f4a8014ee60 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a78004240 con 0x7f4a8014ee60 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a800c0fb0 con 0x7f4a8014ee60 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.297+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78003610 con 0x7f4a8014ee60 2024-02-26T10:41:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.298+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a800c3070 con 0x7f4a8014ee60 2024-02-26T10:41:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.299+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a80074a30 con 0x7f4a8014ee60 2024-02-26T10:41:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.300+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a780037b0 con 0x7f4a8014ee60 2024-02-26T10:41:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.301+0000 7f4a7cc00640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6006b500 0x7f4a6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.301+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4a7808a1e0 con 0x7f4a8014ee60 2024-02-26T10:41:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.302+0000 7f4a84c00640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6006b500 0x7f4a6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:11.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.303+0000 7f4a84c00640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6006b500 0x7f4a6006d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007510 tx=0x7f4a700074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.306+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a78058cf0 con 0x7f4a8014ee60 2024-02-26T10:41:11.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.544+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a8006d9e0 con 0x7f4a8014ee60 2024-02-26T10:41:11.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.545+0000 7f4a7cc00640 1 -- 172.21.15.67:0/1589469666 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+149 (secure 0 0 0) 0x7f4a780585f0 con 0x7f4a8014ee60 2024-02-26T10:41:11.544 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:41:11.544 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:41:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.547+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6006b500 msgr2=0x7f4a6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.547+0000 7f4a86000640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6006b500 0x7f4a6006d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007510 tx=0x7f4a700074a0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 msgr2=0x7f4a800be860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 0x7f4a800be860 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c4c0 tx=0x7f4a7800c980 comp rx=0 tx=0).stop 2024-02-26T10:41:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 shutdown_connections 2024-02-26T10:41:11.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8014ee60 0x7f4a800be860 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a800c8570 0x7f4a800be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6006b500 0x7f4a6006d9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 --2- 172.21.15.67:0/1589469666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a800bdd90 0x7f4a801584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 >> 172.21.15.67:0/1589469666 conn(0x7f4a80072ea0 msgr2=0x7f4a800ac160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 shutdown_connections 2024-02-26T10:41:11.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:11.548+0000 7f4a86000640 1 -- 172.21.15.67:0/1589469666 wait complete. 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:41:11.566 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:41:11.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:11.567 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:11.621 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:41:11.622 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:41:11.623 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:11.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:11.624 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:41:11.707 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:41:11.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:11.711 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:41:11.762 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:41:11.762 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:41:11.762 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:41:11.762 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:41:11.763 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:41:11.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:11.763 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:41:11.763 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:41:11.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:41:11.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:41:11.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:41:11.763 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:41:11.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:11.942 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:41:11.991 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:41:11.992 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:41:11.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:11.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:41:12.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:41:12.084 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:41:12.137 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:41:12.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:41:12.138 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:41:12.188 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs2=/ 2024-02-26T10:41:12.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:12.188 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:41:12.323 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:41:12.323 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:41:12.360 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:41:12.361 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:41:12.361 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:41:12.361 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs2=/ 2024-02-26T10:41:12.361 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:41:12.363 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:41:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:41:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:41:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:41:12.363 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:41:12.367 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:41:12.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:12.367 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:41:12.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:12.462 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:41:12.527 INFO:teuthology.orchestra.run.smithi082.stdout:client6072 2024-02-26T10:41:12.527 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:12.527 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:12.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth add client.testuser mon 'allow rw fsname=cephfs, allow rw fsname=cephfs2' osd 'allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2' mds 'allow rw fsname=cephfs' 2024-02-26T10:41:12.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.822+0000 7f1a6de00640 1 Processor -- start 2024-02-26T10:41:12.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.823+0000 7f1a6de00640 1 -- start start 2024-02-26T10:41:12.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.824+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a681538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.824+0000 7f1a6de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a681542d0 con 0x7f1a6806ea10 2024-02-26T10:41:12.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.824+0000 7f1a6de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a68154570 con 0x7f1a68155050 2024-02-26T10:41:12.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.824+0000 7f1a6de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68154850 con 0x7f1a6814cf30 2024-02-26T10:41:12.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.824+0000 7f1a66000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1a6806ea10 0x7f1a6806ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36816/0 (socket says 172.21.15.67:36816) 2024-02-26T10:41:12.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a66000640 1 -- 172.21.15.67:0/1806239125 learned_addr learned my addr 172.21.15.67:0/1806239125 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:12.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a66a00640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a681538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369994927 0 0) 0x7f1a681542d0 con 0x7f1a6806ea10 2024-02-26T10:41:12.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a4c003520 con 0x7f1a6806ea10 2024-02-26T10:41:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 392774064 0 0) 0x7f1a4c003520 con 0x7f1a6806ea10 2024-02-26T10:41:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 msgr2=0x7f1a681538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a681538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 >> v1:172.21.15.82:6789/0 conn(0x7f1a68155050 legacy=0x7f1a68155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68154b30 con 0x7f1a6806ea10 2024-02-26T10:41:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.825+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1a54001670 con 0x7f1a6806ea10 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 >> v1:172.21.15.67:6789/0 conn(0x7f1a6806ea10 legacy=0x7f1a6806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c004d40 0x7f1a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68154850 con 0x7f1a4c0039e0 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c00bb50 con 0x7f1a4c004d40 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c00bd50 con 0x7f1a6814cf30 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66a00640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66000640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66a00640 1 -- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 msgr2=0x7f1a4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66a00640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66a00640 1 -- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c004d40 msgr2=0x7f1a4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66a00640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c004d40 0x7f1a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.826+0000 7f1a66a00640 1 -- 172.21.15.67:0/1806239125 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c003520 con 0x7f1a4c0039e0 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.827+0000 7f1a66000640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a6814cf30 0x7f1a4c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.828+0000 7f1a66a00640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009ac0 tx=0x7f1a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19261df9c685c07 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.828+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58039070 con 0x7f1a4c0039e0 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.828+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5802c6b0 con 0x7f1a4c0039e0 2024-02-26T10:41:12.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.828+0000 7f1a65600640 1 -- 172.21.15.67:0/1806239125 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58009690 con 0x7f1a4c0039e0 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a6de00640 1 -- 172.21.15.67:0/1806239125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c0039e0 msgr2=0x7f1a4c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a6de00640 1 --2- 172.21.15.67:0/1806239125 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a4c0039e0 0x7f1a4c003e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009ac0 tx=0x7f1a4c005830 comp rx=0 tx=0).stop 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a67400640 1 -- 172.21.15.67:0/1806239125 reap_dead start 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a6de00640 1 -- 172.21.15.67:0/1806239125 shutdown_connections 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a6de00640 1 -- 172.21.15.67:0/1806239125 >> 172.21.15.67:0/1806239125 conn(0x7f1a680c7160 msgr2=0x7f1a680a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a6de00640 1 -- 172.21.15.67:0/1806239125 shutdown_connections 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.829+0000 7f1a6de00640 1 -- 172.21.15.67:0/1806239125 wait complete. 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.831+0000 7f1a6de00640 1 Processor -- start 2024-02-26T10:41:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.831+0000 7f1a6de00640 1 -- start start 2024-02-26T10:41:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c0039e0 0x7f1a680c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a68072c60 0x7f1a680c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 0x7f1a681ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a6de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6806b330 con 0x7f1a68149770 2024-02-26T10:41:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a6de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68155c10 con 0x7f1a4c0039e0 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a6de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6806b4b0 con 0x7f1a68072c60 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a67400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 0x7f1a681ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a67400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 0x7f1a681ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41264/0 (socket says 172.21.15.67:41264) 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a67400640 1 -- 172.21.15.67:0/3592353745 learned_addr learned my addr 172.21.15.67:0/3592353745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.832+0000 7f1a66a00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c0039e0 0x7f1a680c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a66000640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a68072c60 0x7f1a680c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a67400640 1 -- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a68072c60 msgr2=0x7f1a680c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a67400640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a68072c60 0x7f1a680c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a67400640 1 -- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c0039e0 msgr2=0x7f1a680c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a67400640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c0039e0 0x7f1a680c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a67400640 1 -- 172.21.15.67:0/3592353745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68070ef0 con 0x7f1a68149770 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a67400640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 0x7f1a681ee710 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1a68155430 tx=0x7f1a6000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60003970 con 0x7f1a68149770 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60003b30 con 0x7f1a68149770 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a681ec0e0 con 0x7f1a68149770 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.833+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60004020 con 0x7f1a68149770 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.834+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a680c41d0 con 0x7f1a68149770 2024-02-26T10:41:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.835+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a68074a30 con 0x7f1a68149770 2024-02-26T10:41:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.836+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a600041c0 con 0x7f1a68149770 2024-02-26T10:41:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.837+0000 7f1a5f400640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a4806b500 0x7f1a4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.837+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1a600898c0 con 0x7f1a68149770 2024-02-26T10:41:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.837+0000 7f1a66a00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a4806b500 0x7f1a4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.839+0000 7f1a66a00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a4806b500 0x7f1a4806d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1a58036000 tx=0x7f1a58030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:12.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:12.841+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a60058470 con 0x7f1a68149770 2024-02-26T10:41:13.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.109+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow rw fsname=cephfs, allow rw fsname=cephfs2", "osd", "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2", "mds", "allow rw fsname=cephfs"]} v 0) v1 -- 0x7f1a6814b1b0 con 0x7f1a68149770 2024-02-26T10:41:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.114+0000 7f1a5f400640 1 -- 172.21.15.67:0/3592353745 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow rw fsname=cephfs, allow rw fsname=cephfs2", "osd", "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2", "mds", "allow rw fsname=cephfs"]}]=0 added key for client.testuser v15) v1 ==== 291+0+0 (secure 0 0 0) 0x7f1a60057d70 con 0x7f1a68149770 2024-02-26T10:41:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:added key for client.testuser 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.117+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a4806b500 msgr2=0x7f1a4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.117+0000 7f1a6de00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a4806b500 0x7f1a4806d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1a58036000 tx=0x7f1a58030000 comp rx=0 tx=0).stop 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 msgr2=0x7f1a681ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 0x7f1a681ee710 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1a68155430 tx=0x7f1a6000c960 comp rx=0 tx=0).stop 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 shutdown_connections 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a68149770 0x7f1a681ee710 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a68072c60 0x7f1a680c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a4806b500 0x7f1a4806d9c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 --2- 172.21.15.67:0/3592353745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a4c0039e0 0x7f1a680c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 >> 172.21.15.67:0/3592353745 conn(0x7f1a680c7160 msgr2=0x7f1a680a82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 shutdown_connections 2024-02-26T10:41:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.118+0000 7f1a6de00640 1 -- 172.21.15.67:0/3592353745 wait complete. 2024-02-26T10:41:13.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.testuser 2024-02-26T10:41:13.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.364+0000 7f24d3600640 1 Processor -- start 2024-02-26T10:41:13.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.364+0000 7f24d3600640 1 -- start start 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d3600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f24cc06ea30 0x7f24cc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d3600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc06b100 con 0x7f24cc0c1070 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d3600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24cc151670 con 0x7f24cc06ea30 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc06b3e0 con 0x7f24cc06ac20 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f24cc06ea30 0x7f24cc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d0c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f24cc06ea30 0x7f24cc06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41274/0 (socket says 172.21.15.67:41274) 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d0c00640 1 -- 172.21.15.67:0/1727019659 learned_addr learned my addr 172.21.15.67:0/1727019659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.366+0000 7f24d1600640 1 --2- 172.21.15.67:0/1727019659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.367+0000 7f24d0c00640 1 -- 172.21.15.67:0/1727019659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 msgr2=0x7f24cc072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.367+0000 7f24d0c00640 1 --2- 172.21.15.67:0/1727019659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.367+0000 7f24d0c00640 1 -- 172.21.15.67:0/1727019659 >> v1:172.21.15.82:6789/0 conn(0x7f24cc0c1070 legacy=0x7f24cc151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.367+0000 7f24d0c00640 1 -- 172.21.15.67:0/1727019659 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc06b6c0 con 0x7f24cc06ea30 2024-02-26T10:41:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.368+0000 7f24d0c00640 1 --2- 172.21.15.67:0/1727019659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc06ea30 0x7f24cc06dd80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f24cc1517f0 tx=0x7f24c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d541c75c507d5a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.368+0000 7f24cb400640 1 -- 172.21.15.67:0/1727019659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0034070 con 0x7f24cc06ea30 2024-02-26T10:41:13.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.368+0000 7f24cb400640 1 -- 172.21.15.67:0/1727019659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0030610 con 0x7f24cc06ea30 2024-02-26T10:41:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.368+0000 7f24cb400640 1 -- 172.21.15.67:0/1727019659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c00309e0 con 0x7f24cc06ea30 2024-02-26T10:41:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 -- 172.21.15.67:0/1727019659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc06ea30 msgr2=0x7f24cc06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 --2- 172.21.15.67:0/1727019659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc06ea30 0x7f24cc06dd80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f24cc1517f0 tx=0x7f24c002b960 comp rx=0 tx=0).stop 2024-02-26T10:41:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 -- 172.21.15.67:0/1727019659 shutdown_connections 2024-02-26T10:41:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 --2- 172.21.15.67:0/1727019659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 --2- 172.21.15.67:0/1727019659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc06ea30 0x7f24cc06dd80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 -- 172.21.15.67:0/1727019659 >> 172.21.15.67:0/1727019659 conn(0x7f24cc0a84a0 msgr2=0x7f24cc0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 -- 172.21.15.67:0/1727019659 shutdown_connections 2024-02-26T10:41:13.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.369+0000 7f24d3600640 1 -- 172.21.15.67:0/1727019659 wait complete. 2024-02-26T10:41:13.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.371+0000 7f24d3600640 1 Processor -- start 2024-02-26T10:41:13.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.371+0000 7f24d3600640 1 -- start start 2024-02-26T10:41:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc0aa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d3600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24cc06ea30 0x7f24cc14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d3600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 0x7f24cc14a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:13.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d3600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0c3100 con 0x7f24cc0c03c0 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d3600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc151290 con 0x7f24cc06ea30 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0c3280 con 0x7f24cc06ac20 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 0x7f24cc14a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 0x7f24cc14a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41282/0 (socket says 172.21.15.67:41282) 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d1600640 1 -- 172.21.15.67:0/1764151208 learned_addr learned my addr 172.21.15.67:0/1764151208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24d0c00640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc0aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.372+0000 7f24cbe00640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24cc06ea30 0x7f24cc14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d1600640 1 -- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 msgr2=0x7f24cc0aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d1600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc0aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d1600640 1 -- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24cc06ea30 msgr2=0x7f24cc14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d1600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24cc06ea30 0x7f24cc14a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d1600640 1 -- 172.21.15.67:0/1764151208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0bc340 con 0x7f24cc0c03c0 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d1600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 0x7f24cc14a650 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007d90 tx=0x7f24c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c400edd0 con 0x7f24cc0c03c0 2024-02-26T10:41:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c4003990 con 0x7f24cc0c03c0 2024-02-26T10:41:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4003d80 con 0x7f24cc0c03c0 2024-02-26T10:41:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24cc153ce0 con 0x7f24cc0c03c0 2024-02-26T10:41:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.373+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc0aaca0 con 0x7f24cc0c03c0 2024-02-26T10:41:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.374+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc074a30 con 0x7f24cc0c03c0 2024-02-26T10:41:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.376+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24c401b020 con 0x7f24cc0c03c0 2024-02-26T10:41:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.377+0000 7f24c8c00640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24ac06b500 0x7f24ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.377+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f24c408a700 con 0x7f24cc0c03c0 2024-02-26T10:41:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.378+0000 7f24d0c00640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24ac06b500 0x7f24ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.379+0000 7f24d0c00640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24ac06b500 0x7f24ac06d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24c0036000 tx=0x7f24c003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.382+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24c4059210 con 0x7f24cc0c03c0 2024-02-26T10:41:13.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.651+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.testuser"} v 0) v1 -- 0x7f24cc0c3d10 con 0x7f24cc0c03c0 2024-02-26T10:41:13.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.651+0000 7f24c8c00640 1 -- 172.21.15.67:0/1764151208 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.testuser"}]=0 v15) v1 ==== 85+0+245 (secure 0 0 0) 0x7f24c4058b10 con 0x7f24cc0c03c0 2024-02-26T10:41:13.651 INFO:teuthology.orchestra.run.smithi067.stdout:[client.testuser] 2024-02-26T10:41:13.651 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDJatxldmarBhAA/d7MI1T3eae+VhCeSsGkFQ== 2024-02-26T10:41:13.651 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow rw fsname=cephfs" 2024-02-26T10:41:13.651 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow rw fsname=cephfs, allow rw fsname=cephfs2" 2024-02-26T10:41:13.651 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-02-26T10:41:13.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24ac06b500 msgr2=0x7f24ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24ac06b500 0x7f24ac06d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f24c0036000 tx=0x7f24c003b040 comp rx=0 tx=0).stop 2024-02-26T10:41:13.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 msgr2=0x7f24cc14a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:13.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 0x7f24cc14a650 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007d90 tx=0x7f24c400e4b0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 shutdown_connections 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24cc0c03c0 0x7f24cc14a650 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f24ac06b500 0x7f24ac06d9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24cc06ea30 0x7f24cc14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 --2- 172.21.15.67:0/1764151208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24cc06ac20 0x7f24cc0aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.654+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 >> 172.21.15.67:0/1764151208 conn(0x7f24cc0a84a0 msgr2=0x7f24cc0c1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.655+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 shutdown_connections 2024-02-26T10:41:13.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:13.655+0000 7f24d3600640 1 -- 172.21.15.67:0/1764151208 wait complete. 2024-02-26T10:41:13.672 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:41:13.726 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.VQv9bI4xZe 2024-02-26T10:41:13.726 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:13.726 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/tmp/tmp.VQv9bI4xZe 2024-02-26T10:41:13.782 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:13.782 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:13.882 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:13.882 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:13.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:14.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.167+0000 7ff64ac00640 1 Processor -- start 2024-02-26T10:41:14.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.167+0000 7ff64ac00640 1 -- start start 2024-02-26T10:41:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff64ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 0x7ff644153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff64ac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff64406b000 0x7ff6441524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff64ac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff644155f10 con 0x7ff64406ac20 2024-02-26T10:41:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff64ac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64406b470 con 0x7ff64406b000 2024-02-26T10:41:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff64ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440c28d0 con 0x7ff6440c2310 2024-02-26T10:41:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff649800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff64406ac20 0x7ff644152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36828/0 (socket says 172.21.15.67:36828) 2024-02-26T10:41:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.168+0000 7ff649800640 1 -- 172.21.15.67:0/1767724682 learned_addr learned my addr 172.21.15.67:0/1767724682 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff648e00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 0x7ff644153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff64a200640 1 --2- 172.21.15.67:0/1767724682 >> v2:172.21.15.82:3300/0 conn(0x7ff64406b000 0x7ff6441524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757003777 0 0) 0x7ff644155f10 con 0x7ff64406ac20 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62c003520 con 0x7ff64406ac20 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190492399 0 0) 0x7ff62c003520 con 0x7ff64406ac20 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 msgr2=0x7ff644153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 0x7ff644153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 >> v2:172.21.15.82:3300/0 conn(0x7ff64406b000 msgr2=0x7ff6441524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 --2- 172.21.15.67:0/1767724682 >> v2:172.21.15.82:3300/0 conn(0x7ff64406b000 0x7ff6441524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644155150 con 0x7ff64406ac20 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ff638001670 con 0x7ff64406ac20 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 >> v1:172.21.15.67:6789/0 conn(0x7ff64406ac20 legacy=0x7ff644152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 0x7ff62c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff62c003f60 0x7ff62c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.169+0000 7ff643e00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff62c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64406b470 con 0x7ff62c004d40 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440c28d0 con 0x7ff62c003f60 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00baf0 con 0x7ff6440c2310 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff64a200640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff62c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff64a200640 1 -- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 msgr2=0x7ff62c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff648e00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff62c003f60 0x7ff62c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff64a200640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6440c2310 0x7ff62c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff64a200640 1 -- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff62c003f60 msgr2=0x7ff62c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff64a200640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff62c003f60 0x7ff62c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.170+0000 7ff64a200640 1 -- 172.21.15.67:0/1767724682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c003520 con 0x7ff62c004d40 2024-02-26T10:41:14.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.171+0000 7ff64a200640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff62c0051c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009fc0 tx=0x7ff62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bb96c42e60e0d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c039070 con 0x7ff62c004d40 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c02cc60 con 0x7ff62c004d40 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff643e00640 1 -- 172.21.15.67:0/1767724682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c009c10 con 0x7ff62c004d40 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64ac00640 1 -- 172.21.15.67:0/1767724682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 msgr2=0x7ff62c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64ac00640 1 --2- 172.21.15.67:0/1767724682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff62c0051c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009fc0 tx=0x7ff62c005830 comp rx=0 tx=0).stop 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64a200640 1 -- 172.21.15.67:0/1767724682 reap_dead start 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64ac00640 1 -- 172.21.15.67:0/1767724682 shutdown_connections 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64ac00640 1 -- 172.21.15.67:0/1767724682 >> 172.21.15.67:0/1767724682 conn(0x7ff6440c63f0 msgr2=0x7ff6440c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64ac00640 1 -- 172.21.15.67:0/1767724682 shutdown_connections 2024-02-26T10:41:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.172+0000 7ff64ac00640 1 -- 172.21.15.67:0/1767724682 wait complete. 2024-02-26T10:41:14.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.174+0000 7ff64ac00640 1 Processor -- start 2024-02-26T10:41:14.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.174+0000 7ff64ac00640 1 -- start start 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff64ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff6440c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff64ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff64413db30 0x7ff644146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff64ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff644146ac0 0x7ff64413c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff64ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441559f0 con 0x7ff62c004d40 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff64ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644155710 con 0x7ff644146ac0 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff64ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff644148850 con 0x7ff64413db30 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff649800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff6440c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.175+0000 7ff649800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff6440c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41302/0 (socket says 172.21.15.67:41302) 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 -- 172.21.15.67:0/3351454948 learned_addr learned my addr 172.21.15.67:0/3351454948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff648e00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff64413db30 0x7ff644146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff64a200640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff644146ac0 0x7ff64413c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 -- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff64413db30 msgr2=0x7ff644146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff64413db30 0x7ff644146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 -- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff644146ac0 msgr2=0x7ff64413c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff644146ac0 0x7ff64413c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 -- 172.21.15.67:0/3351454948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64413bdc0 con 0x7ff62c004d40 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff648e00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff64413db30 0x7ff644146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff64a200640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff644146ac0 0x7ff64413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.176+0000 7ff649800640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff6440c4fd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c770 tx=0x7ff63800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.177+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638018070 con 0x7ff62c004d40 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.177+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6380074e0 con 0x7ff62c004d40 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.177+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64413c080 con 0x7ff62c004d40 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.177+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638004a00 con 0x7ff62c004d40 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.177+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6440c4ab0 con 0x7ff62c004d40 2024-02-26T10:41:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.178+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6440c4cb0 con 0x7ff62c004d40 2024-02-26T10:41:14.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.179+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff63801b070 con 0x7ff62c004d40 2024-02-26T10:41:14.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.180+0000 7ff641600640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff62806b4d0 0x7ff62806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.180+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff638089af0 con 0x7ff62c004d40 2024-02-26T10:41:14.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.181+0000 7ff648e00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff62806b4d0 0x7ff62806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.182+0000 7ff648e00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff62806b4d0 0x7ff62806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff64413df30 tx=0x7ff634008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:14.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.184+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6440c4cb0 con 0x7ff62c004d40 2024-02-26T10:41:14.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.385+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff644074a30 con 0x7ff62c004d40 2024-02-26T10:41:14.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.390+0000 7ff641600640 1 -- 172.21.15.67:0/3351454948 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+20374 (secure 0 0 0) 0x7ff638058570 con 0x7ff62c004d40 2024-02-26T10:41:14.390 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:14.390 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":59,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:02.772898+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:14.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff62806b4d0 msgr2=0x7ff62806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff62806b4d0 0x7ff62806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff64413df30 tx=0x7ff634008040 comp rx=0 tx=0).stop 2024-02-26T10:41:14.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 msgr2=0x7ff6440c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff6440c4fd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c770 tx=0x7ff63800cc30 comp rx=0 tx=0).stop 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 shutdown_connections 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff62806b4d0 0x7ff62806d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff644146ac0 0x7ff64413c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff64413db30 0x7ff644146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 --2- 172.21.15.67:0/3351454948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff62c004d40 0x7ff6440c4fd0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 >> 172.21.15.67:0/3351454948 conn(0x7ff6440c63f0 msgr2=0x7ff64413e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 shutdown_connections 2024-02-26T10:41:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.394+0000 7ff64ac00640 1 -- 172.21.15.67:0/3351454948 wait complete. 2024-02-26T10:41:14.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:40:43.472687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:40:44.096497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:40:58.810192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:41:00.247724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:14.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:14.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.639+0000 7ff091e00640 1 Processor -- start 2024-02-26T10:41:14.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.639+0000 7ff091e00640 1 -- start start 2024-02-26T10:41:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff091e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff08c0c2950 0x7ff08c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff091e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 0x7ff08c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff091e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08c0a6eb0 con 0x7ff08c1503d0 2024-02-26T10:41:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff091e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08c153d90 con 0x7ff08c0c2950 2024-02-26T10:41:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff091e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0a7190 con 0x7ff08c153920 2024-02-26T10:41:14.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff08aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff08c1503d0 0x7ff08c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36844/0 (socket says 172.21.15.67:36844) 2024-02-26T10:41:14.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.641+0000 7ff08aa00640 1 -- 172.21.15.67:0/385590902 learned_addr learned my addr 172.21.15.67:0/385590902 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:14.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff08b400640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 0x7ff08c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708134836 0 0) 0x7ff08c0a6eb0 con 0x7ff08c1503d0 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff070003520 con 0x7ff08c1503d0 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff08a000640 1 --2- 172.21.15.67:0/385590902 >> v2:172.21.15.82:3300/0 conn(0x7ff08c0c2950 0x7ff08c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2356945156 0 0) 0x7ff070003520 con 0x7ff08c1503d0 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 msgr2=0x7ff08c15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 0x7ff08c15d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 >> v2:172.21.15.82:3300/0 conn(0x7ff08c0c2950 msgr2=0x7ff08c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 --2- 172.21.15.67:0/385590902 >> v2:172.21.15.82:3300/0 conn(0x7ff08c0c2950 0x7ff08c1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c0a7470 con 0x7ff08c1503d0 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ff07c001670 con 0x7ff08c1503d0 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff08a000640 1 --2- 172.21.15.67:0/385590902 >> v2:172.21.15.82:3300/0 conn(0x7ff08c0c2950 0x7ff08c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 >> v1:172.21.15.67:6789/0 conn(0x7ff08c1503d0 legacy=0x7ff08c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 0x7ff0700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff070003f30 0x7ff070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c153d90 con 0x7ff070004d40 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0a7190 con 0x7ff070003f30 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.642+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07000bb50 con 0x7ff08c153920 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08b400640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08aa00640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 0x7ff0700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08a000640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff070003f30 0x7ff070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08b400640 1 -- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 msgr2=0x7ff0700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08b400640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c153920 0x7ff0700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08b400640 1 -- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff070003f30 msgr2=0x7ff070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08b400640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff070003f30 0x7ff070004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.643+0000 7ff08b400640 1 -- 172.21.15.67:0/385590902 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff070003520 con 0x7ff070004d40 2024-02-26T10:41:14.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.644+0000 7ff08b400640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff084009fc0 tx=0x7ff070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13742a8e915b8e4c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:14.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084039070 con 0x7ff070004d40 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08402cc20 con 0x7ff070004d40 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff089600640 1 -- 172.21.15.67:0/385590902 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084009c10 con 0x7ff070004d40 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff091e00640 1 -- 172.21.15.67:0/385590902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 msgr2=0x7ff0700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff091e00640 1 --2- 172.21.15.67:0/385590902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff084009fc0 tx=0x7ff070005830 comp rx=0 tx=0).stop 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff08b400640 1 -- 172.21.15.67:0/385590902 reap_dead start 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff091e00640 1 -- 172.21.15.67:0/385590902 shutdown_connections 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.645+0000 7ff091e00640 1 -- 172.21.15.67:0/385590902 >> 172.21.15.67:0/385590902 conn(0x7ff08c0c6da0 msgr2=0x7ff08c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.646+0000 7ff091e00640 1 -- 172.21.15.67:0/385590902 shutdown_connections 2024-02-26T10:41:14.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.646+0000 7ff091e00640 1 -- 172.21.15.67:0/385590902 wait complete. 2024-02-26T10:41:14.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.647+0000 7ff091e00640 1 Processor -- start 2024-02-26T10:41:14.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.648+0000 7ff091e00640 1 -- start start 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.648+0000 7ff091e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff08c0bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.648+0000 7ff091e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c156d40 0x7ff08c070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.648+0000 7ff091e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c071410 0x7ff08c145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff091e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c06ef00 con 0x7ff070004d40 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff091e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c152ec0 con 0x7ff08c156d40 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff091e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c1507b0 con 0x7ff08c071410 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff08c0bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff08c0bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41326/0 (socket says 172.21.15.67:41326) 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 -- 172.21.15.67:0/2796311705 learned_addr learned my addr 172.21.15.67:0/2796311705 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 -- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c071410 msgr2=0x7ff08c145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c071410 0x7ff08c145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 -- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c156d40 msgr2=0x7ff08c070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c156d40 0x7ff08c070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 -- 172.21.15.67:0/2796311705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c154120 con 0x7ff070004d40 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.649+0000 7ff08aa00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff08c0bfb90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0010e0 tx=0x7ff07c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.650+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c019070 con 0x7ff070004d40 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.650+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c005320 con 0x7ff070004d40 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.650+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0047e0 con 0x7ff070004d40 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.650+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c072a10 con 0x7ff070004d40 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.650+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c0c00d0 con 0x7ff070004d40 2024-02-26T10:41:14.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.651+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c074a30 con 0x7ff070004d40 2024-02-26T10:41:14.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.652+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff07c01c070 con 0x7ff070004d40 2024-02-26T10:41:14.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.653+0000 7ff083400640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff06806b4d0 0x7ff06806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:14.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.653+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff07c089ed0 con 0x7ff070004d40 2024-02-26T10:41:14.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.653+0000 7ff08a000640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff06806b4d0 0x7ff06806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:14.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.655+0000 7ff08a000640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff06806b4d0 0x7ff06806d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff08c157120 tx=0x7ff07400e270 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:14.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.658+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff07c058960 con 0x7ff070004d40 2024-02-26T10:41:14.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.855+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff08c06ec50 con 0x7ff070004d40 2024-02-26T10:41:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.857+0000 7ff083400640 1 -- 172.21.15.67:0/2796311705 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff07c058260 con 0x7ff070004d40 2024-02-26T10:41:14.856 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff06806b4d0 msgr2=0x7ff06806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff06806b4d0 0x7ff06806d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff08c157120 tx=0x7ff07400e270 comp rx=0 tx=0).stop 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 msgr2=0x7ff08c0bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff08c0bfb90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0010e0 tx=0x7ff07c00ed50 comp rx=0 tx=0).stop 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 shutdown_connections 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff08c071410 0x7ff08c145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff06806b4d0 0x7ff06806d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c156d40 0x7ff08c070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 --2- 172.21.15.67:0/2796311705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff070004d40 0x7ff08c0bfb90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 >> 172.21.15.67:0/2796311705 conn(0x7ff08c0c6da0 msgr2=0x7ff08c151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.859+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 shutdown_connections 2024-02-26T10:41:14.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:14.860+0000 7ff091e00640 1 -- 172.21.15.67:0/2796311705 wait complete. 2024-02-26T10:41:14.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:41:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.107+0000 7ffb7ec00640 1 Processor -- start 2024-02-26T10:41:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.107+0000 7ffb7ec00640 1 -- start start 2024-02-26T10:41:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.109+0000 7ffb7ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb780a9430 0x7ffb780ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.109+0000 7ffb7ec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb78150780 con 0x7ffb7806e2d0 2024-02-26T10:41:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.109+0000 7ffb7ec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb780a98c0 con 0x7ffb781503a0 2024-02-26T10:41:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.109+0000 7ffb7ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780a9ba0 con 0x7ffb780a9430 2024-02-26T10:41:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.109+0000 7ffb7e200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffb7806e2d0 0x7ffb780c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36852/0 (socket says 172.21.15.67:36852) 2024-02-26T10:41:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.109+0000 7ffb7e200640 1 -- 172.21.15.67:0/242230960 learned_addr learned my addr 172.21.15.67:0/242230960 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb7ce00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb780a9430 0x7ffb780ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911369614 0 0) 0x7ffb78150780 con 0x7ffb7806e2d0 2024-02-26T10:41:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb60003520 con 0x7ffb7806e2d0 2024-02-26T10:41:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828016022 0 0) 0x7ffb60003520 con 0x7ffb7806e2d0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb780a9430 msgr2=0x7ffb780ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb780a9430 0x7ffb780ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 >> v1:172.21.15.82:6789/0 conn(0x7ffb781503a0 legacy=0x7ffb781535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780a9e80 con 0x7ffb7806e2d0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ffb700019f0 con 0x7ffb7806e2d0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 >> v1:172.21.15.67:6789/0 conn(0x7ffb7806e2d0 legacy=0x7ffb780c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb600039e0 0x7ffb60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb60004800 0x7ffb60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780a9ba0 con 0x7ffb600039e0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6000bf00 con 0x7ffb60004800 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6000c080 con 0x7ffb600051c0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.110+0000 7ffb7d800640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb600039e0 0x7ffb60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7ce00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb60004800 0x7ffb60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7e200640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7d800640 1 -- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb600051c0 msgr2=0x7ffb6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7d800640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7d800640 1 -- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb60004800 msgr2=0x7ffb60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7d800640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb60004800 0x7ffb60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7d800640 1 -- 172.21.15.67:0/242230960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60003520 con 0x7ffb600039e0 2024-02-26T10:41:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.111+0000 7ffb7e200640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.112+0000 7ffb7d800640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb600039e0 0x7ffb60003e00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001750 tx=0x7ffb60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772a064c7360ba17 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.112+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c001370 con 0x7ffb600039e0 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.112+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c0052d0 con 0x7ffb600039e0 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.112+0000 7ffb77e00640 1 -- 172.21.15.67:0/242230960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c003830 con 0x7ffb600039e0 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7ec00640 1 -- 172.21.15.67:0/242230960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb600039e0 msgr2=0x7ffb60003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/242230960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb600039e0 0x7ffb60003e00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001750 tx=0x7ffb60005830 comp rx=0 tx=0).stop 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7e200640 1 -- 172.21.15.67:0/242230960 reap_dead start 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7ec00640 1 -- 172.21.15.67:0/242230960 shutdown_connections 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7ec00640 1 -- 172.21.15.67:0/242230960 >> 172.21.15.67:0/242230960 conn(0x7ffb780c6380 msgr2=0x7ffb780c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7ec00640 1 -- 172.21.15.67:0/242230960 shutdown_connections 2024-02-26T10:41:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.113+0000 7ffb7ec00640 1 -- 172.21.15.67:0/242230960 wait complete. 2024-02-26T10:41:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.115+0000 7ffb7ec00640 1 Processor -- start 2024-02-26T10:41:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.115+0000 7ffb7ec00640 1 -- start start 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb600039e0 0x7ffb780c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 0x7ffb781e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb78155c50 0x7ffb781560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780c0f80 con 0x7ffb7814f560 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7806ddb0 con 0x7ffb600039e0 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780c3520 con 0x7ffb78155c50 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 0x7ffb781e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 0x7ffb781e6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41348/0 (socket says 172.21.15.67:41348) 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7ce00640 1 -- 172.21.15.67:0/140019292 learned_addr learned my addr 172.21.15.67:0/140019292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.116+0000 7ffb7d800640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb600039e0 0x7ffb780c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7e200640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb78155c50 0x7ffb781560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ce00640 1 -- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb78155c50 msgr2=0x7ffb781560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ce00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb78155c50 0x7ffb781560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ce00640 1 -- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb600039e0 msgr2=0x7ffb780c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ce00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb600039e0 0x7ffb780c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ce00640 1 -- 172.21.15.67:0/140019292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78156b10 con 0x7ffb7814f560 2024-02-26T10:41:15.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ce00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 0x7ffb781e6530 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffb64011630 tx=0x7ffb64011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400f920 con 0x7ffb7814f560 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6400fac0 con 0x7ffb7814f560 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400fe70 con 0x7ffb7814f560 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.117+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb78156df0 con 0x7ffb7814f560 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.118+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb78156610 con 0x7ffb7814f560 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.119+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb780678e0 con 0x7ffb7814f560 2024-02-26T10:41:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.120+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb64015460 con 0x7ffb7814f560 2024-02-26T10:41:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.121+0000 7ffb75600640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5806b4d0 0x7ffb5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.121+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ffb6408adf0 con 0x7ffb7814f560 2024-02-26T10:41:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.122+0000 7ffb7d800640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5806b4d0 0x7ffb5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.123+0000 7ffb7d800640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5806b4d0 0x7ffb5806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c033000 tx=0x7ffb6c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:15.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.126+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb64019030 con 0x7ffb7814f560 2024-02-26T10:41:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.326+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb78074a30 con 0x7ffb7814f560 2024-02-26T10:41:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.326+0000 7ffb75600640 1 -- 172.21.15.67:0/140019292 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb640599a0 con 0x7ffb7814f560 2024-02-26T10:41:15.326 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5806b4d0 msgr2=0x7ffb5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5806b4d0 0x7ffb5806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c033000 tx=0x7ffb6c009030 comp rx=0 tx=0).stop 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 msgr2=0x7ffb781e6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 0x7ffb781e6530 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffb64011630 tx=0x7ffb64011af0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 shutdown_connections 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb78155c50 0x7ffb781560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7814f560 0x7ffb781e6530 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5806b4d0 0x7ffb5806d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 --2- 172.21.15.67:0/140019292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb600039e0 0x7ffb780c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.329+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 >> 172.21.15.67:0/140019292 conn(0x7ffb780c6380 msgr2=0x7ffb781513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.330+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 shutdown_connections 2024-02-26T10:41:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.330+0000 7ffb7ec00640 1 -- 172.21.15.67:0/140019292 wait complete. 2024-02-26T10:41:15.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:41:15.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.574+0000 7fa62cc00640 1 Processor -- start 2024-02-26T10:41:15.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.574+0000 7fa62cc00640 1 -- start start 2024-02-26T10:41:15.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa62cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa628158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa62cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6280c8470 0x7fa628155680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa62cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa628155bc0 0x7fa6280c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa62cc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6280c8cd0 con 0x7fa6280c8470 2024-02-26T10:41:15.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa62cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa628154ea0 con 0x7fa628155bc0 2024-02-26T10:41:15.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa62cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa628070870 con 0x7fa6280c8090 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa61ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6280c8470 0x7fa628155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa61ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6280c8470 0x7fa628155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41364/0 (socket says 172.21.15.67:41364) 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.575+0000 7fa61ea00640 1 -- 172.21.15.67:0/522161230 learned_addr learned my addr 172.21.15.67:0/522161230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa627e00640 1 --2- 172.21.15.67:0/522161230 >> v2:172.21.15.82:3300/0 conn(0x7fa628155bc0 0x7fa6280c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa627400640 1 --2- 172.21.15.67:0/522161230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa628158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa61ea00640 1 -- 172.21.15.67:0/522161230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 msgr2=0x7fa628158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa61ea00640 1 --2- 172.21.15.67:0/522161230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa628158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa61ea00640 1 -- 172.21.15.67:0/522161230 >> v2:172.21.15.82:3300/0 conn(0x7fa628155bc0 msgr2=0x7fa6280c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:15.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa61ea00640 1 --2- 172.21.15.67:0/522161230 >> v2:172.21.15.82:3300/0 conn(0x7fa628155bc0 0x7fa6280c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.576+0000 7fa61ea00640 1 -- 172.21.15.67:0/522161230 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6280c6f20 con 0x7fa6280c8470 2024-02-26T10:41:15.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.577+0000 7fa61ea00640 1 --2- 172.21.15.67:0/522161230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280c8470 0x7fa628155680 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa628070a80 tx=0x7fa61002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332db45d2eaf93d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:15.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.577+0000 7fa626a00640 1 -- 172.21.15.67:0/522161230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610038070 con 0x7fa6280c8470 2024-02-26T10:41:15.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.577+0000 7fa626a00640 1 -- 172.21.15.67:0/522161230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa610031740 con 0x7fa6280c8470 2024-02-26T10:41:15.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.577+0000 7fa626a00640 1 -- 172.21.15.67:0/522161230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61002f050 con 0x7fa6280c8470 2024-02-26T10:41:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 -- 172.21.15.67:0/522161230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280c8470 msgr2=0x7fa628155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 --2- 172.21.15.67:0/522161230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280c8470 0x7fa628155680 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa628070a80 tx=0x7fa61002b7e0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 -- 172.21.15.67:0/522161230 shutdown_connections 2024-02-26T10:41:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 --2- 172.21.15.67:0/522161230 >> v2:172.21.15.82:3300/0 conn(0x7fa628155bc0 0x7fa6280c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 --2- 172.21.15.67:0/522161230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280c8470 0x7fa628155680 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 --2- 172.21.15.67:0/522161230 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa628158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 -- 172.21.15.67:0/522161230 >> 172.21.15.67:0/522161230 conn(0x7fa62806ac20 msgr2=0x7fa62806b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 -- 172.21.15.67:0/522161230 shutdown_connections 2024-02-26T10:41:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.578+0000 7fa62cc00640 1 -- 172.21.15.67:0/522161230 wait complete. 2024-02-26T10:41:15.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.580+0000 7fa62cc00640 1 Processor -- start 2024-02-26T10:41:15.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.580+0000 7fa62cc00640 1 -- start start 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa62cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa6281eea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa62cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280c8470 0x7fa6280aa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa62cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 0x7fa62814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa62cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa628152770 con 0x7fa6280a9540 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa62cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6280c88d0 con 0x7fa6280c8470 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa62cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281528f0 con 0x7fa6280c8090 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa627e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 0x7fa62814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa627e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 0x7fa62814b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41374/0 (socket says 172.21.15.67:41374) 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa627e00640 1 -- 172.21.15.67:0/3599825831 learned_addr learned my addr 172.21.15.67:0/3599825831 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.581+0000 7fa627400640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa6281eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa61ea00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280c8470 0x7fa6280aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa627e00640 1 -- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 msgr2=0x7fa6281eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa627e00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa6281eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa627e00640 1 -- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280c8470 msgr2=0x7fa6280aa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa627e00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280c8470 0x7fa6280aa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa627e00640 1 -- 172.21.15.67:0/3599825831 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6281eacc0 con 0x7fa6280a9540 2024-02-26T10:41:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa627e00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 0x7fa62814b950 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa620009fd0 tx=0x7fa62000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620014420 con 0x7fa6280a9540 2024-02-26T10:41:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa620019070 con 0x7fa6280a9540 2024-02-26T10:41:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620014800 con 0x7fa6280a9540 2024-02-26T10:41:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.582+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6281eafa0 con 0x7fa6280a9540 2024-02-26T10:41:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.583+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6281eb560 con 0x7fa6280a9540 2024-02-26T10:41:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.584+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa628074a30 con 0x7fa6280a9540 2024-02-26T10:41:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.585+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6200149a0 con 0x7fa6280a9540 2024-02-26T10:41:15.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.586+0000 7fa61fe00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa60406b500 0x7fa60406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.586+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa62008ad40 con 0x7fa6280a9540 2024-02-26T10:41:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.586+0000 7fa627400640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa60406b500 0x7fa60406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:15.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.587+0000 7fa627400640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa60406b500 0x7fa60406d9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa618006fd0 tx=0x7fa618008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.590+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6200598d0 con 0x7fa6280a9540 2024-02-26T10:41:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.788+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6280c9380 con 0x7fa6280a9540 2024-02-26T10:41:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.789+0000 7fa61fe00640 1 -- 172.21.15.67:0/3599825831 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6200591d0 con 0x7fa6280a9540 2024-02-26T10:41:15.788 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.791+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa60406b500 msgr2=0x7fa60406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.791+0000 7fa62cc00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa60406b500 0x7fa60406d9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa618006fd0 tx=0x7fa618008040 comp rx=0 tx=0).stop 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.791+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 msgr2=0x7fa62814b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.791+0000 7fa62cc00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 0x7fa62814b950 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa620009fd0 tx=0x7fa62000e4b0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 shutdown_connections 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6280a9540 0x7fa62814b950 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa60406b500 0x7fa60406d9c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280c8470 0x7fa6280aa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 --2- 172.21.15.67:0/3599825831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6280c8090 0x7fa6281eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 >> 172.21.15.67:0/3599825831 conn(0x7fa62806ac20 msgr2=0x7fa6280c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 shutdown_connections 2024-02-26T10:41:15.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:15.792+0000 7fa62cc00640 1 -- 172.21.15.67:0/3599825831 wait complete. 2024-02-26T10:41:15.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:41:16.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.038+0000 7f4dff800640 1 Processor -- start 2024-02-26T10:41:16.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.038+0000 7f4dff800640 1 -- start start 2024-02-26T10:41:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.039+0000 7f4dff800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 0x7f4df806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.039+0000 7f4dff800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df80a6eb0 con 0x7f4df8153af0 2024-02-26T10:41:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.039+0000 7f4dff800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df80a7190 con 0x7f4df806e390 2024-02-26T10:41:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.039+0000 7f4dff800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80a7470 con 0x7f4df81503d0 2024-02-26T10:41:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.040+0000 7f4dfd800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4df8153af0 0x7f4df815d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36862/0 (socket says 172.21.15.67:36862) 2024-02-26T10:41:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.040+0000 7f4dfd800640 1 -- 172.21.15.67:0/1769115761 learned_addr learned my addr 172.21.15.67:0/1769115761 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.040+0000 7f4dfce00640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 0x7f4df806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.040+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433946084 0 0) 0x7f4df80a6eb0 con 0x7f4df8153af0 2024-02-26T10:41:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.040+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de0003520 con 0x7f4df8153af0 2024-02-26T10:41:16.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4140314579 0 0) 0x7f4de0003520 con 0x7f4df8153af0 2024-02-26T10:41:16.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 msgr2=0x7f4df806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:16.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 0x7f4df806de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 >> v1:172.21.15.82:6789/0 conn(0x7f4df806e390 legacy=0x7f4df81533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80a7750 con 0x7f4df8153af0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f4df00019f0 con 0x7f4df8153af0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 >> v1:172.21.15.67:6789/0 conn(0x7f4df8153af0 legacy=0x7f4df815d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4de00039e0 0x7f4de0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 0x7f4de0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4de0004d40 0x7f4de00051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80a7470 con 0x7f4de0004d40 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de000bbd0 con 0x7f4de00039e0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de000bdd0 con 0x7f4df81503d0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4dfd800640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4de0004d40 0x7f4de00051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.041+0000 7f4df7e00640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 0x7f4de0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfce00640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4de00039e0 0x7f4de0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfd800640 1 -- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 msgr2=0x7f4de0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfd800640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4df81503d0 0x7f4de0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfd800640 1 -- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4de00039e0 msgr2=0x7f4de0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfd800640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4de00039e0 0x7f4de0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfd800640 1 -- 172.21.15.67:0/1769115761 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0003520 con 0x7f4de0004d40 2024-02-26T10:41:16.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.042+0000 7f4dfce00640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4de00039e0 0x7f4de0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.043+0000 7f4dfd800640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4de0004d40 0x7f4de00051a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4df806d060 tx=0x7f4de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b003e701e0598e51 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.043+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000a390 con 0x7f4de0004d40 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.043+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0034650 con 0x7f4de0004d40 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.043+0000 7f4df7400640 1 -- 172.21.15.67:0/1769115761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0034a40 con 0x7f4de0004d40 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dff800640 1 -- 172.21.15.67:0/1769115761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4de0004d40 msgr2=0x7f4de00051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dff800640 1 --2- 172.21.15.67:0/1769115761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4de0004d40 0x7f4de00051a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4df806d060 tx=0x7f4de0005830 comp rx=0 tx=0).stop 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dfd800640 1 -- 172.21.15.67:0/1769115761 reap_dead start 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dff800640 1 -- 172.21.15.67:0/1769115761 shutdown_connections 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dff800640 1 -- 172.21.15.67:0/1769115761 >> 172.21.15.67:0/1769115761 conn(0x7f4df80c6da0 msgr2=0x7f4df80c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dff800640 1 -- 172.21.15.67:0/1769115761 shutdown_connections 2024-02-26T10:41:16.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.044+0000 7f4dff800640 1 -- 172.21.15.67:0/1769115761 wait complete. 2024-02-26T10:41:16.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.046+0000 7f4dff800640 1 Processor -- start 2024-02-26T10:41:16.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.046+0000 7f4dff800640 1 -- start start 2024-02-26T10:41:16.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.046+0000 7f4dff800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4de0004d40 0x7f4df814af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dff800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4df80ab900 0x7f4df80ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dff800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 0x7f4df81e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dff800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81507b0 con 0x7f4df81e5550 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dff800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df806ef00 con 0x7f4df80ab900 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dff800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80c0ef0 con 0x7f4de0004d40 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 0x7f4df81e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 0x7f4df81e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41402/0 (socket says 172.21.15.67:41402) 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 -- 172.21.15.67:0/2843054533 learned_addr learned my addr 172.21.15.67:0/2843054533 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 -- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4de0004d40 msgr2=0x7f4df814af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4de0004d40 0x7f4df814af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 -- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4df80ab900 msgr2=0x7f4df80ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4df7e00640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4df80ab900 0x7f4df80ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4df80ab900 0x7f4df80ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.047+0000 7f4dfd800640 1 -- 172.21.15.67:0/2843054533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df8156d90 con 0x7f4df81e5550 2024-02-26T10:41:16.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.048+0000 7f4dfd800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 0x7f4df81e59d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4df002cb30 tx=0x7f4df80a7080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.048+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df00017b0 con 0x7f4df81e5550 2024-02-26T10:41:16.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.048+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0044070 con 0x7f4df81e5550 2024-02-26T10:41:16.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.048+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0034800 con 0x7f4df81e5550 2024-02-26T10:41:16.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.048+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df80a99b0 con 0x7f4df81e5550 2024-02-26T10:41:16.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.048+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df80a9110 con 0x7f4df81e5550 2024-02-26T10:41:16.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.049+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df8074a30 con 0x7f4df81e5550 2024-02-26T10:41:16.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.050+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4df00349a0 con 0x7f4df81e5550 2024-02-26T10:41:16.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.052+0000 7f4df4c00640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4dd806b500 0x7f4dd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.052+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4df00ac950 con 0x7f4df81e5550 2024-02-26T10:41:16.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.052+0000 7f4dfce00640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4dd806b500 0x7f4dd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.053+0000 7f4dfce00640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4dd806b500 0x7f4dd806d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009a00 tx=0x7f4dec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.056+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4df007b580 con 0x7f4df81e5550 2024-02-26T10:41:16.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.257+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4df806ec50 con 0x7f4df81e5550 2024-02-26T10:41:16.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.258+0000 7f4df4c00640 1 -- 172.21.15.67:0/2843054533 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v59) v1 ==== 105+0+11 (secure 0 0 0) 0x7f4df007ae80 con 0x7f4df81e5550 2024-02-26T10:41:16.257 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:16.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.260+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4dd806b500 msgr2=0x7f4dd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.260+0000 7f4dff800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4dd806b500 0x7f4dd806d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009a00 tx=0x7f4dec008040 comp rx=0 tx=0).stop 2024-02-26T10:41:16.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.260+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 msgr2=0x7f4df81e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.260+0000 7f4dff800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 0x7f4df81e59d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4df002cb30 tx=0x7f4df80a7080 comp rx=0 tx=0).stop 2024-02-26T10:41:16.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 shutdown_connections 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4df81e5550 0x7f4df81e59d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4dd806b500 0x7f4dd806d9c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4df80ab900 0x7f4df80ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 --2- 172.21.15.67:0/2843054533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4de0004d40 0x7f4df814af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 >> 172.21.15.67:0/2843054533 conn(0x7f4df80c6da0 msgr2=0x7f4df806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 shutdown_connections 2024-02-26T10:41:16.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.261+0000 7f4dff800640 1 -- 172.21.15.67:0/2843054533 wait complete. 2024-02-26T10:41:16.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:41:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.508+0000 7f0051400640 1 Processor -- start 2024-02-26T10:41:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.508+0000 7f0051400640 1 -- start start 2024-02-26T10:41:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.509+0000 7f0051400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c1591b0 0x7f004c158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.509+0000 7f0051400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f004c0c0ef0 0x7f004c070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.509+0000 7f0051400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f004c071410 0x7f004c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.509+0000 7f0051400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c073010 con 0x7f004c071410 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.509+0000 7f0051400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c14f490 con 0x7f004c0c0ef0 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.509+0000 7f0051400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c072ce0 con 0x7f004c1591b0 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f004c071410 0x7f004c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f004c071410 0x7f004c06de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41410/0 (socket says 172.21.15.67:41410) 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 -- 172.21.15.67:0/897112708 learned_addr learned my addr 172.21.15.67:0/897112708 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004a800640 1 --2- 172.21.15.67:0/897112708 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c1591b0 0x7f004c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f0049e00640 1 --2- 172.21.15.67:0/897112708 >> v2:172.21.15.82:3300/0 conn(0x7f004c0c0ef0 0x7f004c070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 -- 172.21.15.67:0/897112708 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c1591b0 msgr2=0x7f004c158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 --2- 172.21.15.67:0/897112708 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c1591b0 0x7f004c158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 -- 172.21.15.67:0/897112708 >> v2:172.21.15.82:3300/0 conn(0x7f004c0c0ef0 msgr2=0x7f004c070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 --2- 172.21.15.67:0/897112708 >> v2:172.21.15.82:3300/0 conn(0x7f004c0c0ef0 0x7f004c070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.510+0000 7f004b200640 1 -- 172.21.15.67:0/897112708 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c155bf0 con 0x7f004c071410 2024-02-26T10:41:16.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.511+0000 7f004b200640 1 --2- 172.21.15.67:0/897112708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c071410 0x7f004c06de50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f004c06f280 tx=0x7f004402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7acc2b37f95f293 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.512+0000 7f0049400640 1 -- 172.21.15.67:0/897112708 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004402e070 con 0x7f004c071410 2024-02-26T10:41:16.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.512+0000 7f0049400640 1 -- 172.21.15.67:0/897112708 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044037070 con 0x7f004c071410 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.512+0000 7f0049400640 1 -- 172.21.15.67:0/897112708 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044031a00 con 0x7f004c071410 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 -- 172.21.15.67:0/897112708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c071410 msgr2=0x7f004c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 --2- 172.21.15.67:0/897112708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c071410 0x7f004c06de50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f004c06f280 tx=0x7f004402b880 comp rx=0 tx=0).stop 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 -- 172.21.15.67:0/897112708 shutdown_connections 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 --2- 172.21.15.67:0/897112708 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c071410 0x7f004c06de50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 --2- 172.21.15.67:0/897112708 >> v2:172.21.15.82:3300/0 conn(0x7f004c0c0ef0 0x7f004c070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 --2- 172.21.15.67:0/897112708 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c1591b0 0x7f004c158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 -- 172.21.15.67:0/897112708 >> 172.21.15.67:0/897112708 conn(0x7f004c0c6da0 msgr2=0x7f004c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 -- 172.21.15.67:0/897112708 shutdown_connections 2024-02-26T10:41:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.513+0000 7f0051400640 1 -- 172.21.15.67:0/897112708 wait complete. 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.515+0000 7f0051400640 1 Processor -- start 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.515+0000 7f0051400640 1 -- start start 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0051400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c071410 0x7f004c14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0051400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f004c0c0ef0 0x7f004c149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0051400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 0x7f004c1471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0051400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0a8d20 con 0x7f004c1591b0 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0051400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0a8ba0 con 0x7f004c0c0ef0 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0051400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c06e390 con 0x7f004c071410 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f004b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 0x7f004c1471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f004b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 0x7f004c1471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41412/0 (socket says 172.21.15.67:41412) 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f004b200640 1 -- 172.21.15.67:0/2614659153 learned_addr learned my addr 172.21.15.67:0/2614659153 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:16.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f004a800640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c071410 0x7f004c14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.516+0000 7f0049e00640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f004c0c0ef0 0x7f004c149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004b200640 1 -- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c071410 msgr2=0x7f004c14f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004b200640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c071410 0x7f004c14f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004b200640 1 -- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f004c0c0ef0 msgr2=0x7f004c149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004b200640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f004c0c0ef0 0x7f004c149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004b200640 1 -- 172.21.15.67:0/2614659153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c148280 con 0x7f004c1591b0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004a800640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c071410 0x7f004c14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f004b200640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 0x7f004c1471b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00440090b0 tx=0x7f00440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044032670 con 0x7f004c1591b0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044032830 con 0x7f004c1591b0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c147da0 con 0x7f004c1591b0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.517+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044032c20 con 0x7f004c1591b0 2024-02-26T10:41:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.518+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c1486a0 con 0x7f004c1591b0 2024-02-26T10:41:16.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.518+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c0c2ce0 con 0x7f004c1591b0 2024-02-26T10:41:16.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.520+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f004404c060 con 0x7f004c1591b0 2024-02-26T10:41:16.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.520+0000 7f0042a00640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f002c06b500 0x7f002c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.521+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f00440ac130 con 0x7f004c1591b0 2024-02-26T10:41:16.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.521+0000 7f004a800640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f002c06b500 0x7f002c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.522+0000 7f004a800640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f002c06b500 0x7f002c06d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f003c006fd0 tx=0x7f003c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.525+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0044040070 con 0x7f004c1591b0 2024-02-26T10:41:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.728+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f004c074a30 con 0x7f004c1591b0 2024-02-26T10:41:16.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.729+0000 7f0042a00640 1 -- 172.21.15.67:0/2614659153 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v59) v1 ==== 101+0+11 (secure 0 0 0) 0x7f0044039310 con 0x7f004c1591b0 2024-02-26T10:41:16.729 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f002c06b500 msgr2=0x7f002c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f002c06b500 0x7f002c06d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f003c006fd0 tx=0x7f003c008040 comp rx=0 tx=0).stop 2024-02-26T10:41:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 msgr2=0x7f004c1471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 0x7f004c1471b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f00440090b0 tx=0x7f00440046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 shutdown_connections 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f004c1591b0 0x7f004c1471b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f002c06b500 0x7f002c06d9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f004c0c0ef0 0x7f004c149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 --2- 172.21.15.67:0/2614659153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f004c071410 0x7f004c14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 >> 172.21.15.67:0/2614659153 conn(0x7f004c0c6da0 msgr2=0x7f004c0bf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 shutdown_connections 2024-02-26T10:41:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.732+0000 7f0051400640 1 -- 172.21.15.67:0/2614659153 wait complete. 2024-02-26T10:41:16.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:41:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.971+0000 7f96be400640 1 Processor -- start 2024-02-26T10:41:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.971+0000 7f96be400640 1 -- start start 2024-02-26T10:41:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96be400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96b806b000 0x7f96b80c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96be400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b806b430 con 0x7f96b806ac20 2024-02-26T10:41:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96be400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b8159800 con 0x7f96b80c12e0 2024-02-26T10:41:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96be400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b815a090 con 0x7f96b806b000 2024-02-26T10:41:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96bda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96b806b000 0x7f96b80c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96bd000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f96b806ac20 0x7f96b806e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36872/0 (socket says 172.21.15.67:36872) 2024-02-26T10:41:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.973+0000 7f96bd000640 1 -- 172.21.15.67:0/3771061237 learned_addr learned my addr 172.21.15.67:0/3771061237 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.974+0000 7f96b7400640 1 -- 172.21.15.67:0/3771061237 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182576267 0 0) 0x7f96b806b430 con 0x7f96b806ac20 2024-02-26T10:41:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.974+0000 7f96bda00640 1 -- 172.21.15.67:0/3771061237 >> v1:172.21.15.82:6789/0 conn(0x7f96b80c12e0 legacy=0x7f96b81590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.974+0000 7f96bda00640 1 -- 172.21.15.67:0/3771061237 >> v1:172.21.15.67:6789/0 conn(0x7f96b806ac20 legacy=0x7f96b806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.974+0000 7f96bda00640 1 -- 172.21.15.67:0/3771061237 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8159ad0 con 0x7f96b806b000 2024-02-26T10:41:16.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.975+0000 7f96bda00640 1 --2- 172.21.15.67:0/3771061237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b80c19b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dac4258f9e3622 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.975+0000 7f96b7400640 1 -- 172.21.15.67:0/3771061237 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00315a0 con 0x7f96b806b000 2024-02-26T10:41:16.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.975+0000 7f96b7400640 1 -- 172.21.15.67:0/3771061237 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0037070 con 0x7f96b806b000 2024-02-26T10:41:16.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.975+0000 7f96b7400640 1 -- 172.21.15.67:0/3771061237 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0031960 con 0x7f96b806b000 2024-02-26T10:41:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 -- 172.21.15.67:0/3771061237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 msgr2=0x7f96b80c19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 --2- 172.21.15.67:0/3771061237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b80c19b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002b840 comp rx=0 tx=0).stop 2024-02-26T10:41:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 -- 172.21.15.67:0/3771061237 shutdown_connections 2024-02-26T10:41:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 --2- 172.21.15.67:0/3771061237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b80c19b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 -- 172.21.15.67:0/3771061237 >> 172.21.15.67:0/3771061237 conn(0x7f96b80c63f0 msgr2=0x7f96b80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 -- 172.21.15.67:0/3771061237 shutdown_connections 2024-02-26T10:41:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.976+0000 7f96be400640 1 -- 172.21.15.67:0/3771061237 wait complete. 2024-02-26T10:41:16.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.978+0000 7f96be400640 1 Processor -- start 2024-02-26T10:41:16.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.978+0000 7f96be400640 1 -- start start 2024-02-26T10:41:16.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96be400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96b806ac20 0x7f96b8071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96be400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96be400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96b80c12e0 0x7f96b814e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96be400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b806df10 con 0x7f96b806b000 2024-02-26T10:41:16.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96be400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b806dc30 con 0x7f96b806ac20 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96be400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b81534d0 con 0x7f96b80c12e0 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b814dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41422/0 (socket says 172.21.15.67:41422) 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96b7e00640 1 -- 172.21.15.67:0/3723241855 learned_addr learned my addr 172.21.15.67:0/3723241855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96bd000640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96b806ac20 0x7f96b8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96bda00640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96b80c12e0 0x7f96b814e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96b7e00640 1 -- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96b80c12e0 msgr2=0x7f96b814e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.979+0000 7f96b7e00640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96b80c12e0 0x7f96b814e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b7e00640 1 -- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96b806ac20 msgr2=0x7f96b8071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b7e00640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96b806ac20 0x7f96b8071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b7e00640 1 -- 172.21.15.67:0/3723241855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b815d5f0 con 0x7f96b806b000 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96bd000640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96b806ac20 0x7f96b8071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96bda00640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96b80c12e0 0x7f96b814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b7e00640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b814dfe0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c3c0 tx=0x7f96a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x7f96b806b000 2024-02-26T10:41:16.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a00052f0 con 0x7f96b806b000 2024-02-26T10:41:16.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0003830 con 0x7f96b806b000 2024-02-26T10:41:16.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b81565b0 con 0x7f96b806b000 2024-02-26T10:41:16.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.980+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b81f23d0 con 0x7f96b806b000 2024-02-26T10:41:16.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.981+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b80c2760 con 0x7f96b806b000 2024-02-26T10:41:16.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.982+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96a001a070 con 0x7f96b806b000 2024-02-26T10:41:16.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.983+0000 7f96b4c00640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f969406b4d0 0x7f969406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:16.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.983+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f96a0089850 con 0x7f96b806b000 2024-02-26T10:41:16.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.984+0000 7f96bd000640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f969406b4d0 0x7f969406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:16.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.985+0000 7f96bd000640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f969406b4d0 0x7f969406d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96a80012e0 tx=0x7f96a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:16.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:16.988+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96a0058370 con 0x7f96b806b000 2024-02-26T10:41:17.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.200+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96b8074a30 con 0x7f96b806b000 2024-02-26T10:41:17.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.201+0000 7f96b4c00640 1 -- 172.21.15.67:0/3723241855 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7f96a0057c70 con 0x7f96b806b000 2024-02-26T10:41:17.200 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f969406b4d0 msgr2=0x7f969406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f969406b4d0 0x7f969406d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96a80012e0 tx=0x7f96a8009040 comp rx=0 tx=0).stop 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 msgr2=0x7f96b814dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b814dfe0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f96a000c3c0 tx=0x7f96a000c880 comp rx=0 tx=0).stop 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 shutdown_connections 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96b80c12e0 0x7f96b814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96b806b000 0x7f96b814dfe0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f969406b4d0 0x7f969406d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.203+0000 7f96be400640 1 --2- 172.21.15.67:0/3723241855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96b806ac20 0x7f96b8071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.204+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 >> 172.21.15.67:0/3723241855 conn(0x7f96b80c63f0 msgr2=0x7f96b814f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.204+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 shutdown_connections 2024-02-26T10:41:17.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.204+0000 7f96be400640 1 -- 172.21.15.67:0/3723241855 wait complete. 2024-02-26T10:41:17.220 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T10:41:17.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:17.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T10:41:17.318 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:41:17.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:17.319 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T10:41:17.373 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:17.373 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:17.423 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:41:17.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:41:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.649+0000 7f2cf2800640 1 Processor -- start 2024-02-26T10:41:17.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.649+0000 7f2cf2800640 1 -- start start 2024-02-26T10:41:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.650+0000 7f2cf2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cec152380 0x7f2cec0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.650+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 0x7f2cec155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec152760 con 0x7f2cec06d290 2024-02-26T10:41:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec155cd0 con 0x7f2cec152380 2024-02-26T10:41:17.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c7550 con 0x7f2cec0c7020 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf0a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 0x7f2cec155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf0a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 0x7f2cec155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:51088/0 (socket says 172.21.15.67:51088) 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf0a00640 1 -- 172.21.15.67:0/2865065543 learned_addr learned my addr 172.21.15.67:0/2865065543 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.651+0000 7f2cf1400640 1 --2- 172.21.15.67:0/2865065543 >> v2:172.21.15.82:3300/0 conn(0x7f2cec152380 0x7f2cec0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.652+0000 7f2cf0a00640 1 -- 172.21.15.67:0/2865065543 >> v2:172.21.15.82:3300/0 conn(0x7f2cec152380 msgr2=0x7f2cec0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.652+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2865065543 >> v2:172.21.15.82:3300/0 conn(0x7f2cec152380 0x7f2cec0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.652+0000 7f2cf0a00640 1 -- 172.21.15.67:0/2865065543 >> v1:172.21.15.82:6789/0 conn(0x7f2cec06d290 legacy=0x7f2cec0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.652+0000 7f2cf0a00640 1 -- 172.21.15.67:0/2865065543 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0c7830 con 0x7f2cec0c7020 2024-02-26T10:41:17.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.654+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2865065543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 0x7f2cec155790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009870 tx=0x7f2cd802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21d45b00f1f435a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:17.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.654+0000 7f2cebe00640 1 -- 172.21.15.67:0/2865065543 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8035020 con 0x7f2cec0c7020 2024-02-26T10:41:17.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.654+0000 7f2cebe00640 1 -- 172.21.15.67:0/2865065543 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd8031640 con 0x7f2cec0c7020 2024-02-26T10:41:17.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.654+0000 7f2cebe00640 1 -- 172.21.15.67:0/2865065543 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802d040 con 0x7f2cec0c7020 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 -- 172.21.15.67:0/2865065543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 msgr2=0x7f2cec155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2865065543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 0x7f2cec155790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009870 tx=0x7f2cd802b8f0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 -- 172.21.15.67:0/2865065543 shutdown_connections 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2865065543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec0c7020 0x7f2cec155790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2865065543 >> v2:172.21.15.82:3300/0 conn(0x7f2cec152380 0x7f2cec0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 -- 172.21.15.67:0/2865065543 >> 172.21.15.67:0/2865065543 conn(0x7f2cec0ac1a0 msgr2=0x7f2cec0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:17.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 -- 172.21.15.67:0/2865065543 shutdown_connections 2024-02-26T10:41:17.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.655+0000 7f2cf2800640 1 -- 172.21.15.67:0/2865065543 wait complete. 2024-02-26T10:41:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.657+0000 7f2cf2800640 1 Processor -- start 2024-02-26T10:41:17.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.657+0000 7f2cf2800640 1 -- start start 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 0x7f2cec1eb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cec0c7020 0x7f2cec145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec152380 0x7f2cec145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c56a0 con 0x7f2cec06d290 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec06cd70 con 0x7f2cec0c7020 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c5820 con 0x7f2cec152380 2024-02-26T10:41:17.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 0x7f2cec1eb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 0x7f2cec1eb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41426/0 (socket says 172.21.15.67:41426) 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.659+0000 7f2cf1400640 1 -- 172.21.15.67:0/2517601616 learned_addr learned my addr 172.21.15.67:0/2517601616 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cec0c7020 0x7f2cec145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1e00640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec152380 0x7f2cec145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1400640 1 -- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec152380 msgr2=0x7f2cec145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1400640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec152380 0x7f2cec145fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1400640 1 -- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cec0c7020 msgr2=0x7f2cec145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1400640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cec0c7020 0x7f2cec145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1400640 1 -- 172.21.15.67:0/2517601616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec14beb0 con 0x7f2cec06d290 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1e00640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec152380 0x7f2cec145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cec0c7020 0x7f2cec145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf1400640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 0x7f2cec1eb7e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000e9b0 tx=0x7f2ce000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0017070 con 0x7f2cec06d290 2024-02-26T10:41:17.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce000c9a0 con 0x7f2cec06d290 2024-02-26T10:41:17.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000cd90 con 0x7f2cec06d290 2024-02-26T10:41:17.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec150360 con 0x7f2cec06d290 2024-02-26T10:41:17.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.660+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec14fcb0 con 0x7f2cec06d290 2024-02-26T10:41:17.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.662+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec06bf80 con 0x7f2cec06d290 2024-02-26T10:41:17.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.662+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ce001a070 con 0x7f2cec06d290 2024-02-26T10:41:17.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.664+0000 7f2ce9600640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:17.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.664+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f2ce008a530 con 0x7f2cec06d290 2024-02-26T10:41:17.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.664+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:17.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.666+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8036000 tx=0x7f2cd8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:17.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.668+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cec06bf80 con 0x7f2cec06d290 2024-02-26T10:41:17.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.913+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2cec06ac20 con 0x7f2cec06d290 2024-02-26T10:41:17.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.913+0000 7f2ce9600640 1 -- 172.21.15.67:0/2517601616 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+149 (secure 0 0 0) 0x7f2cec06ac20 con 0x7f2cec06d290 2024-02-26T10:41:17.913 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:41:17.913 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.915+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ccc06b4d0 msgr2=0x7f2ccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.915+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8036000 tx=0x7f2cd8008030 comp rx=0 tx=0).stop 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.915+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 msgr2=0x7f2cec1eb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.915+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 0x7f2cec1eb7e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000e9b0 tx=0x7f2ce000ee70 comp rx=0 tx=0).stop 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 shutdown_connections 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cec152380 0x7f2cec145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ccc06b4d0 0x7f2ccc06d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cec0c7020 0x7f2cec145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 --2- 172.21.15.67:0/2517601616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06d290 0x7f2cec1eb7e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:17.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 >> 172.21.15.67:0/2517601616 conn(0x7f2cec0ac1a0 msgr2=0x7f2cec0c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:17.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 shutdown_connections 2024-02-26T10:41:17.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:17.916+0000 7f2cf2800640 1 -- 172.21.15.67:0/2517601616 wait complete. 2024-02-26T10:41:17.933 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-02-26T10:41:17.933 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:41:17.933 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:41:17.933 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-02-26T10:41:17.933 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:41:17.934 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-02-26T10:41:17.934 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:41:17.934 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.VQv9bI4xZe 2024-02-26T10:41:17.934 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:41:17.987 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.moyLs9iYm3 2024-02-26T10:41:17.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /tmp/tmp.VQv9bI4xZe /tmp/tmp.moyLs9iYm3 2024-02-26T10:41:18.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.moyLs9iYm3 2024-02-26T10:41:18.248 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.moyLs9iYm3 is 245B 2024-02-26T10:41:18.313 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.moyLs9iYm3 2024-02-26T10:41:18.368 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQDJatxldmarBhAA/d7MI1T3eae+VhCeSsGkFQ== caps mds = "allow rw fsname=cephfs" caps mon = "allow rw fsname=cephfs, allow rw fsname=cephfs2" caps osd = "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-02-26T10:41:18.369 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:41:18.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:18.369 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:41:18.423 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:41:18.423 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:41:18.423 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:41:18.423 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:18.423 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42372sec preferred_lft 42372sec 2024-02-26T10:41:18.424 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::288e:93ff:fe9f:f95d/64 scope link 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::3cfc:46ff:fe71:84e3/64 scope link 2024-02-26T10:41:18.425 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:41:18.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:18.426 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:41:18.508 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:41:18.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:18.512 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:41:18.569 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:41:18.569 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:41:18.570 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:41:18.570 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:41:18.570 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:41:18.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:18.570 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:41:18.570 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:41:18.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:41:18.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:41:18.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:41:18.570 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:41:18.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:18.761 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:41:18.813 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:41:18.814 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:41:18.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:18.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:41:18.914 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:41:18.914 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:41:18.969 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:41:18.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:41:18.969 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:41:19.022 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:19.022 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/tmp/tmp.VQv9bI4xZe of=/dev/stdout 2024-02-26T10:41:19.074 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs2=/ 2024-02-26T10:41:19.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:19.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs2=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQDJatxldmarBhAA/d7MI1T3eae+VhCeSsGkFQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:41:19.222 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,secret=AQDJatxldmarBhAA/d7MI1T3eae+VhCeSsGkFQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:41:19.223 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:41:19.223 INFO:teuthology.orchestra.run.smithi067.stdout:keyring.get_secret failed 2024-02-26T10:41:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:mount.ceph: kernel BUG! 2024-02-26T10:41:19.263 INFO:teuthology.orchestra.run.smithi067.stderr:mount error 13 = Permission denied 2024-02-26T10:41:19.264 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,secret=AQDJatxldmarBhAA/d7MI1T3eae+VhCeSsGkFQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:41:19.264 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:41:19.264 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:41:19.264 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: testuser@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs2=/ 2024-02-26T10:41:19.264 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:41:19.269 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-26T10:41:19.270 INFO:tasks.cephfs.kernel_mount:mount command failed 2024-02-26T10:41:19.270 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-26T10:41:19.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:41:19.383 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:41:19.503 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-26T10:41:19.504 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:19.504 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:19.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:41:19.564 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:19.564 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:19.623 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:19.623 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:19.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:19.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.906+0000 7fafc9400640 1 Processor -- start 2024-02-26T10:41:19.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.907+0000 7fafc9400640 1 -- start start 2024-02-26T10:41:19.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafc4152380 0x7fafc40c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:19.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fafc40c7020 0x7fafc4155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:19.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc9400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc4152760 con 0x7fafc406d290 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc9400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc4155cd0 con 0x7fafc4152380 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc9400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc40c7550 con 0x7fafc40c7020 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafc4152380 0x7fafc40c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.908+0000 7fafc3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafc4152380 0x7fafc40c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34168/0 (socket says 172.21.15.67:34168) 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.909+0000 7fafc3e00640 1 -- 172.21.15.67:0/1991931418 learned_addr learned my addr 172.21.15.67:0/1991931418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.909+0000 7fafc3e00640 1 -- 172.21.15.67:0/1991931418 >> v2:172.21.15.82:3300/0 conn(0x7fafc40c7020 msgr2=0x7fafc4155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:19.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.909+0000 7fafc3e00640 1 --2- 172.21.15.67:0/1991931418 >> v2:172.21.15.82:3300/0 conn(0x7fafc40c7020 0x7fafc4155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:19.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.909+0000 7fafc3e00640 1 -- 172.21.15.67:0/1991931418 >> v1:172.21.15.82:6789/0 conn(0x7fafc406d290 legacy=0x7fafc40c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:19.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.909+0000 7fafc3e00640 1 -- 172.21.15.67:0/1991931418 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40c7830 con 0x7fafc4152380 2024-02-26T10:41:19.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.910+0000 7fafc3e00640 1 --2- 172.21.15.67:0/1991931418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc4152380 0x7fafc40c8510 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fafc40acf40 tx=0x7fafb402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c11b537acdb6f3a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:19.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.910+0000 7fafc3400640 1 -- 172.21.15.67:0/1991931418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb402e070 con 0x7fafc4152380 2024-02-26T10:41:19.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.910+0000 7fafc3400640 1 -- 172.21.15.67:0/1991931418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb4037070 con 0x7fafc4152380 2024-02-26T10:41:19.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.910+0000 7fafc3400640 1 -- 172.21.15.67:0/1991931418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4031a00 con 0x7fafc4152380 2024-02-26T10:41:19.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 -- 172.21.15.67:0/1991931418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc4152380 msgr2=0x7fafc40c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:19.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 --2- 172.21.15.67:0/1991931418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc4152380 0x7fafc40c8510 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fafc40acf40 tx=0x7fafb402b880 comp rx=0 tx=0).stop 2024-02-26T10:41:19.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 -- 172.21.15.67:0/1991931418 shutdown_connections 2024-02-26T10:41:19.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 --2- 172.21.15.67:0/1991931418 >> v2:172.21.15.82:3300/0 conn(0x7fafc40c7020 0x7fafc4155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:19.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 --2- 172.21.15.67:0/1991931418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc4152380 0x7fafc40c8510 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:19.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 -- 172.21.15.67:0/1991931418 >> 172.21.15.67:0/1991931418 conn(0x7fafc40ac1a0 msgr2=0x7fafc40ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:19.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 -- 172.21.15.67:0/1991931418 shutdown_connections 2024-02-26T10:41:19.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.911+0000 7fafc9400640 1 -- 172.21.15.67:0/1991931418 wait complete. 2024-02-26T10:41:19.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.913+0000 7fafc9400640 1 Processor -- start 2024-02-26T10:41:19.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.913+0000 7fafc9400640 1 -- start start 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafc9400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fafc406d290 0x7fafc40c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafc9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 0x7fafc4149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafc9400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafc4152380 0x7fafc41eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafc9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40c56a0 con 0x7fafc40c7020 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafc9400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc406cd70 con 0x7fafc4152380 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafc9400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40c5820 con 0x7fafc406d290 2024-02-26T10:41:19.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 0x7fafc4149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 0x7fafc4149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34178/0 (socket says 172.21.15.67:34178) 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.914+0000 7fafbbe00640 1 -- 172.21.15.67:0/761642610 learned_addr learned my addr 172.21.15.67:0/761642610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafc3e00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fafc406d290 0x7fafc40c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafc8a00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafc4152380 0x7fafc41eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafbbe00640 1 -- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fafc406d290 msgr2=0x7fafc40c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafbbe00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fafc406d290 0x7fafc40c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafbbe00640 1 -- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafc4152380 msgr2=0x7fafc41eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafbbe00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafc4152380 0x7fafc41eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafbbe00640 1 -- 172.21.15.67:0/761642610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc41eea70 con 0x7fafc40c7020 2024-02-26T10:41:19.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.915+0000 7fafbbe00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 0x7fafc4149350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fafac00b4d0 tx=0x7fafac00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:19.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.916+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac014070 con 0x7fafc40c7020 2024-02-26T10:41:19.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.916+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac010870 con 0x7fafc40c7020 2024-02-26T10:41:19.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.916+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac010c60 con 0x7fafc40c7020 2024-02-26T10:41:19.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.916+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc41eaad0 con 0x7fafc40c7020 2024-02-26T10:41:19.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.916+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc4148fe0 con 0x7fafc40c7020 2024-02-26T10:41:19.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.917+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc4074a30 con 0x7fafc40c7020 2024-02-26T10:41:19.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.918+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafac017070 con 0x7fafc40c7020 2024-02-26T10:41:19.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.919+0000 7fafc0c00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fafa006b4d0 0x7fafa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:19.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.919+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fafac08a280 con 0x7fafc40c7020 2024-02-26T10:41:19.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.919+0000 7fafc3e00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fafa006b4d0 0x7fafa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:19.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.921+0000 7fafc3e00640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fafa006b4d0 0x7fafa006d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fafb4036000 tx=0x7fafb403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:19.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:19.923+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafac058e30 con 0x7fafc40c7020 2024-02-26T10:41:20.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.126+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafc40c7c50 con 0x7fafc40c7020 2024-02-26T10:41:20.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.130+0000 7fafc0c00640 1 -- 172.21.15.67:0/761642610 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+20374 (secure 0 0 0) 0x7fafac058730 con 0x7fafc40c7020 2024-02-26T10:41:20.129 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:20.129 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":59,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:02.772898+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:20.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fafa006b4d0 msgr2=0x7fafa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fafa006b4d0 0x7fafa006d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fafb4036000 tx=0x7fafb403d040 comp rx=0 tx=0).stop 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 msgr2=0x7fafc4149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 0x7fafc4149350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fafac00b4d0 tx=0x7fafac00b990 comp rx=0 tx=0).stop 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 shutdown_connections 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fafa006b4d0 0x7fafa006d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafc4152380 0x7fafc41eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafc40c7020 0x7fafc4149350 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 --2- 172.21.15.67:0/761642610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fafc406d290 0x7fafc40c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 >> 172.21.15.67:0/761642610 conn(0x7fafc40ac1a0 msgr2=0x7fafc41507e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 shutdown_connections 2024-02-26T10:41:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.133+0000 7fafc9400640 1 -- 172.21.15.67:0/761642610 wait complete. 2024-02-26T10:41:20.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:40:43.472687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:40:44.096497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:40:58.810192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:41:00.247724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:20.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:20.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.383+0000 7fd1aa800640 1 Processor -- start 2024-02-26T10:41:20.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.383+0000 7fd1aa800640 1 -- start start 2024-02-26T10:41:20.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1aa800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 0x7fd1a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1aa800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd1a406ce30 0x7fd1a40af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1aa800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a406c110 con 0x7fd1a40c5290 2024-02-26T10:41:20.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1aa800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a40c72c0 con 0x7fd1a406ce30 2024-02-26T10:41:20.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1aa800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c75a0 con 0x7fd1a41562f0 2024-02-26T10:41:20.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1a9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd1a40c5290 0x7fd1a4155880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47216/0 (socket says 172.21.15.67:47216) 2024-02-26T10:41:20.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1a9400640 1 -- 172.21.15.67:0/2215755048 learned_addr learned my addr 172.21.15.67:0/2215755048 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:20.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.385+0000 7fd1a8a00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 0x7fd1a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a9e00640 1 --2- 172.21.15.67:0/2215755048 >> v2:172.21.15.82:3300/0 conn(0x7fd1a406ce30 0x7fd1a40af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409013084 0 0) 0x7fd1a406c110 con 0x7fd1a40c5290 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd18c003520 con 0x7fd1a40c5290 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688029637 0 0) 0x7fd18c003520 con 0x7fd1a40c5290 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 msgr2=0x7fd1a406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 0x7fd1a406c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 >> v2:172.21.15.82:3300/0 conn(0x7fd1a406ce30 msgr2=0x7fd1a40af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 --2- 172.21.15.67:0/2215755048 >> v2:172.21.15.82:3300/0 conn(0x7fd1a406ce30 0x7fd1a40af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a40c7880 con 0x7fd1a40c5290 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fd198001670 con 0x7fd1a40c5290 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 >> v1:172.21.15.67:6789/0 conn(0x7fd1a40c5290 legacy=0x7fd1a4155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 0x7fd18c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.386+0000 7fd1a3e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd18c003f30 0x7fd18c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a3e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd18c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c72c0 con 0x7fd18c004d40 2024-02-26T10:41:20.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c75a0 con 0x7fd18c003f30 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c00ba10 con 0x7fd1a41562f0 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd18c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9400640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 0x7fd18c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9e00640 1 -- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 msgr2=0x7fd18c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a41562f0 0x7fd18c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9e00640 1 -- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd18c003f30 msgr2=0x7fd18c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd18c003f30 0x7fd18c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.387+0000 7fd1a9e00640 1 -- 172.21.15.67:0/2215755048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c003520 con 0x7fd18c004d40 2024-02-26T10:41:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1a9e00640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd18c0051c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009f90 tx=0x7fd18c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d1ce011746812 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c039070 con 0x7fd18c004d40 2024-02-26T10:41:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c02cbd0 con 0x7fd18c004d40 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1a3e00640 1 -- 172.21.15.67:0/2215755048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c009c10 con 0x7fd18c004d40 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1aa800640 1 -- 172.21.15.67:0/2215755048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 msgr2=0x7fd18c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1aa800640 1 --2- 172.21.15.67:0/2215755048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd18c0051c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009f90 tx=0x7fd18c005830 comp rx=0 tx=0).stop 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1a9e00640 1 -- 172.21.15.67:0/2215755048 reap_dead start 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1aa800640 1 -- 172.21.15.67:0/2215755048 shutdown_connections 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.389+0000 7fd1aa800640 1 -- 172.21.15.67:0/2215755048 >> 172.21.15.67:0/2215755048 conn(0x7fd1a40ac230 msgr2=0x7fd1a40ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.390+0000 7fd1aa800640 1 -- 172.21.15.67:0/2215755048 shutdown_connections 2024-02-26T10:41:20.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.390+0000 7fd1aa800640 1 -- 172.21.15.67:0/2215755048 wait complete. 2024-02-26T10:41:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.391+0000 7fd1aa800640 1 Processor -- start 2024-02-26T10:41:20.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 -- start start 2024-02-26T10:41:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd1a40c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a40c9530 0x7fd1a41edf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a41ee4d0 0x7fd1a414b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c57f0 con 0x7fd18c004d40 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40c5670 con 0x7fd1a41ee4d0 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.392+0000 7fd1aa800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40bddc0 con 0x7fd1a40c9530 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd1a40c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd1a40c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34204/0 (socket says 172.21.15.67:34204) 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 -- 172.21.15.67:0/796365979 learned_addr learned my addr 172.21.15.67:0/796365979 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9e00640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a41ee4d0 0x7fd1a414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a8a00640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a40c9530 0x7fd1a41edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 -- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a40c9530 msgr2=0x7fd1a41edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a40c9530 0x7fd1a41edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 -- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a41ee4d0 msgr2=0x7fd1a414b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a41ee4d0 0x7fd1a414b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.393+0000 7fd1a9400640 1 -- 172.21.15.67:0/796365979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a40c99b0 con 0x7fd18c004d40 2024-02-26T10:41:20.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.394+0000 7fd1a9400640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd1a40c8ff0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd19800e850 tx=0x7fd19800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.394+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800c870 con 0x7fd18c004d40 2024-02-26T10:41:20.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.394+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198018070 con 0x7fd18c004d40 2024-02-26T10:41:20.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.394+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a41eed10 con 0x7fd18c004d40 2024-02-26T10:41:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.394+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800cc50 con 0x7fd18c004d40 2024-02-26T10:41:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.394+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a414d220 con 0x7fd18c004d40 2024-02-26T10:41:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.395+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a4074a30 con 0x7fd18c004d40 2024-02-26T10:41:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.396+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd198013420 con 0x7fd18c004d40 2024-02-26T10:41:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.397+0000 7fd1a1600640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b500 0x7fd18406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.397+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd19808a0a0 con 0x7fd18c004d40 2024-02-26T10:41:20.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.398+0000 7fd1a8a00640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b500 0x7fd18406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.399+0000 7fd1a8a00640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b500 0x7fd18406d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd190004770 tx=0x7fd190008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.403+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd198058bb0 con 0x7fd18c004d40 2024-02-26T10:41:20.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.601+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1a40c4ad0 con 0x7fd18c004d40 2024-02-26T10:41:20.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.602+0000 7fd1a1600640 1 -- 172.21.15.67:0/796365979 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1980584b0 con 0x7fd18c004d40 2024-02-26T10:41:20.601 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b500 msgr2=0x7fd18406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b500 0x7fd18406d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd190004770 tx=0x7fd190008040 comp rx=0 tx=0).stop 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 msgr2=0x7fd1a40c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd1a40c8ff0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd19800e850 tx=0x7fd19800ed10 comp rx=0 tx=0).stop 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 shutdown_connections 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd18406b500 0x7fd18406d9c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a41ee4d0 0x7fd1a414b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1a40c9530 0x7fd1a41edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 --2- 172.21.15.67:0/796365979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd18c004d40 0x7fd1a40c8ff0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.604+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 >> 172.21.15.67:0/796365979 conn(0x7fd1a40ac230 msgr2=0x7fd1a40ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:20.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.605+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 shutdown_connections 2024-02-26T10:41:20.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.605+0000 7fd1aa800640 1 -- 172.21.15.67:0/796365979 wait complete. 2024-02-26T10:41:20.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:41:20.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.851+0000 7f7866000640 1 Processor -- start 2024-02-26T10:41:20.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.851+0000 7f7866000640 1 -- start start 2024-02-26T10:41:20.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7866000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78600c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7866000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786006e9e0 con 0x7f78600a9430 2024-02-26T10:41:20.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7866000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7860150780 con 0x7f78601503a0 2024-02-26T10:41:20.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7866000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78600ab8d0 con 0x7f78600ab480 2024-02-26T10:41:20.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7857e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f78600a9430 0x7f786006e2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47218/0 (socket says 172.21.15.67:47218) 2024-02-26T10:41:20.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7857e00640 1 -- 172.21.15.67:0/3246848445 learned_addr learned my addr 172.21.15.67:0/3246848445 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.853+0000 7f7865600640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78600c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019013638 0 0) 0x7f786006e9e0 con 0x7f78600a9430 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7844003520 con 0x7f78600a9430 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4162249459 0 0) 0x7f7844003520 con 0x7f78600a9430 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 msgr2=0x7f78600c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78600c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 >> v1:172.21.15.82:6789/0 conn(0x7f78601503a0 legacy=0x7f78601535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600a98c0 con 0x7f78600a9430 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f7848001670 con 0x7f78600a9430 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 >> v1:172.21.15.67:6789/0 conn(0x7f78600a9430 legacy=0x7f786006e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7844003f20 0x7f7844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7844004d40 0x7f78440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600ab8d0 con 0x7f7844003f20 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784400ba70 con 0x7f7844004d40 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784400bc70 con 0x7f78600ab480 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.854+0000 7f7857e00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7844003f20 0x7f7844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7865600640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7844004d40 0x7f78440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7864c00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7857e00640 1 -- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 msgr2=0x7f78440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7857e00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7857e00640 1 -- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7844004d40 msgr2=0x7f78440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:20.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7857e00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7844004d40 0x7f78440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7857e00640 1 -- 172.21.15.67:0/3246848445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7844003520 con 0x7f7844003f20 2024-02-26T10:41:20.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.855+0000 7f7864c00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78600ab480 0x7f78440039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.856+0000 7f7857e00640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7844003f20 0x7f7844004800 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f784800c190 tx=0x7f7844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d2de6b6ec8633b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.856+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848007c60 con 0x7f7844003f20 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.856+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7848030070 con 0x7f7844003f20 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.856+0000 7f785fe00640 1 -- 172.21.15.67:0/3246848445 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848005d10 con 0x7f7844003f20 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7866000640 1 -- 172.21.15.67:0/3246848445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7844003f20 msgr2=0x7f7844004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7866000640 1 --2- 172.21.15.67:0/3246848445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7844003f20 0x7f7844004800 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f784800c190 tx=0x7f7844005830 comp rx=0 tx=0).stop 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7865600640 1 -- 172.21.15.67:0/3246848445 reap_dead start 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7866000640 1 -- 172.21.15.67:0/3246848445 shutdown_connections 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7866000640 1 -- 172.21.15.67:0/3246848445 >> 172.21.15.67:0/3246848445 conn(0x7f78600c6380 msgr2=0x7f78600c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7866000640 1 -- 172.21.15.67:0/3246848445 shutdown_connections 2024-02-26T10:41:20.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.857+0000 7f7866000640 1 -- 172.21.15.67:0/3246848445 wait complete. 2024-02-26T10:41:20.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.859+0000 7f7866000640 1 Processor -- start 2024-02-26T10:41:20.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.859+0000 7f7866000640 1 -- start start 2024-02-26T10:41:20.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7866000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7844003f20 0x7f78600c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7866000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 0x7f7860151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7866000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7860144dd0 0x7f7860145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7866000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786006dad0 con 0x7f7860152720 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7866000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600a9e80 con 0x7f7860144dd0 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7866000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786006ddb0 con 0x7f7844003f20 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 0x7f7860151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 0x7f7860151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34220/0 (socket says 172.21.15.67:34220) 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7857e00640 1 -- 172.21.15.67:0/291739210 learned_addr learned my addr 172.21.15.67:0/291739210 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7864c00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7844003f20 0x7f78600c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7865600640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7860144dd0 0x7f7860145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.860+0000 7f7857e00640 1 -- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7844003f20 msgr2=0x7f78600c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7857e00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7844003f20 0x7f78600c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7857e00640 1 -- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7860144dd0 msgr2=0x7f7860145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7857e00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7860144dd0 0x7f7860145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7857e00640 1 -- 172.21.15.67:0/291739210 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78601e8030 con 0x7f7860152720 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7864c00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7844003f20 0x7f78600c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:20.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7857e00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 0x7f7860151680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f784800c280 tx=0x7f78600a9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848045070 con 0x7f7860152720 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7848004c10 con 0x7f7860152720 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78601457a0 con 0x7f7860152720 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848007c60 con 0x7f7860152720 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.861+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78601e8590 con 0x7f7860152720 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.863+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78600678e0 con 0x7f7860152720 2024-02-26T10:41:20.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.863+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7848007070 con 0x7f7860152720 2024-02-26T10:41:20.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.865+0000 7f785d600640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f783c06b4d0 0x7f783c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:20.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.865+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f78480ab550 con 0x7f7860152720 2024-02-26T10:41:20.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.865+0000 7f7864c00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f783c06b4d0 0x7f783c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:20.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.867+0000 7f7864c00640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f783c06b4d0 0x7f783c06d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f78600aa790 tx=0x7f7850009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:20.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:20.869+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78601e8030 con 0x7f7860152720 2024-02-26T10:41:21.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.066+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78600c1650 con 0x7f7860152720 2024-02-26T10:41:21.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.067+0000 7f785d600640 1 -- 172.21.15.67:0/291739210 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f784807a1e0 con 0x7f7860152720 2024-02-26T10:41:21.066 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:21.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f783c06b4d0 msgr2=0x7f783c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f783c06b4d0 0x7f783c06d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f78600aa790 tx=0x7f7850009040 comp rx=0 tx=0).stop 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 msgr2=0x7f7860151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 0x7f7860151680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f784800c280 tx=0x7f78600a9a90 comp rx=0 tx=0).stop 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 shutdown_connections 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f783c06b4d0 0x7f783c06d990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7860144dd0 0x7f7860145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7860152720 0x7f7860151680 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 --2- 172.21.15.67:0/291739210 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7844003f20 0x7f78600c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 >> 172.21.15.67:0/291739210 conn(0x7f78600c6380 msgr2=0x7f78600c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 shutdown_connections 2024-02-26T10:41:21.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.070+0000 7f7866000640 1 -- 172.21.15.67:0/291739210 wait complete. 2024-02-26T10:41:21.090 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:41:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.317+0000 7fc523e00640 1 Processor -- start 2024-02-26T10:41:21.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.317+0000 7fc523e00640 1 -- start start 2024-02-26T10:41:21.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc523e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5240c5290 0x7fc5241562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc523e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc524155880 0x7fc52406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc523e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52406c110 con 0x7fc52406ce30 2024-02-26T10:41:21.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc523e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240c72c0 con 0x7fc5240c5290 2024-02-26T10:41:21.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc523e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240c75a0 con 0x7fc524155880 2024-02-26T10:41:21.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc522a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5240c5290 0x7fc5241562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc522a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc5240c5290 0x7fc5241562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34236/0 (socket says 172.21.15.67:34236) 2024-02-26T10:41:21.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.319+0000 7fc522a00640 1 -- 172.21.15.67:0/4123930856 learned_addr learned my addr 172.21.15.67:0/4123930856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.320+0000 7fc522000640 1 --2- 172.21.15.67:0/4123930856 >> v2:172.21.15.82:3300/0 conn(0x7fc524155880 0x7fc52406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.320+0000 7fc522a00640 1 -- 172.21.15.67:0/4123930856 >> v2:172.21.15.82:3300/0 conn(0x7fc524155880 msgr2=0x7fc52406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.320+0000 7fc522a00640 1 --2- 172.21.15.67:0/4123930856 >> v2:172.21.15.82:3300/0 conn(0x7fc524155880 0x7fc52406c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.320+0000 7fc522a00640 1 -- 172.21.15.67:0/4123930856 >> v1:172.21.15.82:6789/0 conn(0x7fc52406ce30 legacy=0x7fc5240af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.320+0000 7fc522a00640 1 -- 172.21.15.67:0/4123930856 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240c7880 con 0x7fc5240c5290 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.320+0000 7fc522000640 1 --2- 172.21.15.67:0/4123930856 >> v2:172.21.15.82:3300/0 conn(0x7fc524155880 0x7fc52406c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.321+0000 7fc522a00640 1 --2- 172.21.15.67:0/4123930856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c5290 0x7fc5241562f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc5240cb770 tx=0x7fc51402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253261d788cd0b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.321+0000 7fc521600640 1 -- 172.21.15.67:0/4123930856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51402e070 con 0x7fc5240c5290 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc521600640 1 -- 172.21.15.67:0/4123930856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc514037070 con 0x7fc5240c5290 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc521600640 1 -- 172.21.15.67:0/4123930856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514031a00 con 0x7fc5240c5290 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 -- 172.21.15.67:0/4123930856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c5290 msgr2=0x7fc5241562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 --2- 172.21.15.67:0/4123930856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c5290 0x7fc5241562f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc5240cb770 tx=0x7fc51402b880 comp rx=0 tx=0).stop 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 -- 172.21.15.67:0/4123930856 shutdown_connections 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 --2- 172.21.15.67:0/4123930856 >> v2:172.21.15.82:3300/0 conn(0x7fc524155880 0x7fc52406c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 --2- 172.21.15.67:0/4123930856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c5290 0x7fc5241562f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 -- 172.21.15.67:0/4123930856 >> 172.21.15.67:0/4123930856 conn(0x7fc5240ac230 msgr2=0x7fc5240ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.322+0000 7fc523e00640 1 -- 172.21.15.67:0/4123930856 shutdown_connections 2024-02-26T10:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.323+0000 7fc523e00640 1 -- 172.21.15.67:0/4123930856 wait complete. 2024-02-26T10:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.324+0000 7fc523e00640 1 Processor -- start 2024-02-26T10:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.324+0000 7fc523e00640 1 -- start start 2024-02-26T10:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.325+0000 7fc523e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5240c5290 0x7fc5240c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.325+0000 7fc523e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524155880 0x7fc5240c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.325+0000 7fc523e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 0x7fc5240c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.325+0000 7fc523e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240affb0 con 0x7fc5240c2eb0 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.325+0000 7fc523e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc524155cf0 con 0x7fc524155880 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.325+0000 7fc523e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240b0130 con 0x7fc5240c5290 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 0x7fc5240c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 0x7fc5240c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34252/0 (socket says 172.21.15.67:34252) 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 -- 172.21.15.67:0/2294015100 learned_addr learned my addr 172.21.15.67:0/2294015100 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc522a00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5240c5290 0x7fc5240c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc522000640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524155880 0x7fc5240c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 -- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5240c5290 msgr2=0x7fc5240c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5240c5290 0x7fc5240c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 -- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524155880 msgr2=0x7fc5240c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524155880 0x7fc5240c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.326+0000 7fc523400640 1 -- 172.21.15.67:0/2294015100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52414f4d0 con 0x7fc5240c2eb0 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.327+0000 7fc523400640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 0x7fc5240c90e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00be60 tx=0x7fc51c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.327+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c019070 con 0x7fc5240c2eb0 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.327+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c014440 con 0x7fc5240c2eb0 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.327+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c014870 con 0x7fc5240c2eb0 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.327+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5240c23d0 con 0x7fc5240c2eb0 2024-02-26T10:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.327+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5240c4410 con 0x7fc5240c2eb0 2024-02-26T10:41:21.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.328+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc524074a30 con 0x7fc5240c2eb0 2024-02-26T10:41:21.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.329+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51c01c070 con 0x7fc5240c2eb0 2024-02-26T10:41:21.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.330+0000 7fc51aa00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc50006b4d0 0x7fc50006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.330+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc51c08a4d0 con 0x7fc5240c2eb0 2024-02-26T10:41:21.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.331+0000 7fc522a00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc50006b4d0 0x7fc50006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.332+0000 7fc522a00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc50006b4d0 0x7fc50006d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc514036000 tx=0x7fc51403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:21.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.334+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc51c058fe0 con 0x7fc5240c2eb0 2024-02-26T10:41:21.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.532+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc52406d950 con 0x7fc5240c2eb0 2024-02-26T10:41:21.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.533+0000 7fc51aa00640 1 -- 172.21.15.67:0/2294015100 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc51c0588e0 con 0x7fc5240c2eb0 2024-02-26T10:41:21.532 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:21.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc50006b4d0 msgr2=0x7fc50006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc50006b4d0 0x7fc50006d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc514036000 tx=0x7fc51403d040 comp rx=0 tx=0).stop 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 msgr2=0x7fc5240c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 0x7fc5240c90e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00be60 tx=0x7fc51c00bf50 comp rx=0 tx=0).stop 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 shutdown_connections 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5240c2eb0 0x7fc5240c90e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc50006b4d0 0x7fc50006d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc524155880 0x7fc5240c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 --2- 172.21.15.67:0/2294015100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5240c5290 0x7fc5240c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.535+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 >> 172.21.15.67:0/2294015100 conn(0x7fc5240ac230 msgr2=0x7fc5241544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.536+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 shutdown_connections 2024-02-26T10:41:21.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.536+0000 7fc523e00640 1 -- 172.21.15.67:0/2294015100 wait complete. 2024-02-26T10:41:21.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:41:21.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.783+0000 7fc0c5e00640 1 Processor -- start 2024-02-26T10:41:21.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.783+0000 7fc0c5e00640 1 -- start start 2024-02-26T10:41:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.785+0000 7fc0c5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c0154cf0 0x7fc0c00c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.785+0000 7fc0c5e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c00c3840 con 0x7fc0c0071fd0 2024-02-26T10:41:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.785+0000 7fc0c5e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c006dc80 con 0x7fc0c01523b0 2024-02-26T10:41:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.785+0000 7fc0c5e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0152790 con 0x7fc0c0154cf0 2024-02-26T10:41:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.786+0000 7fc0c5400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc0c0071fd0 0x7fc0c0072440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47222/0 (socket says 172.21.15.67:47222) 2024-02-26T10:41:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.786+0000 7fc0c5400640 1 -- 172.21.15.67:0/3534996583 learned_addr learned my addr 172.21.15.67:0/3534996583 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.786+0000 7fc0bfe00640 1 --2- 172.21.15.67:0/3534996583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c0154cf0 0x7fc0c00c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.786+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057947967 0 0) 0x7fc0c00c3840 con 0x7fc0c0071fd0 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.786+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a8003520 con 0x7fc0c0071fd0 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.787+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006790380 0 0) 0x7fc0c006dc80 con 0x7fc0c01523b0 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.787+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c00c3840 con 0x7fc0c01523b0 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.787+0000 7fc0bfe00640 1 -- 172.21.15.67:0/3534996583 >> v1:172.21.15.82:6789/0 conn(0x7fc0c01523b0 legacy=0x7fc0c00c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.787+0000 7fc0bfe00640 1 -- 172.21.15.67:0/3534996583 >> v1:172.21.15.67:6789/0 conn(0x7fc0c0071fd0 legacy=0x7fc0c0072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.787+0000 7fc0bfe00640 1 -- 172.21.15.67:0/3534996583 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c006df60 con 0x7fc0c0154cf0 2024-02-26T10:41:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.788+0000 7fc0bfe00640 1 --2- 172.21.15.67:0/3534996583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c0154cf0 0x7fc0c00c3300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0097d0 tx=0x7fc0ac02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c91aec5eebe89c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.789+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac031770 con 0x7fc0c0154cf0 2024-02-26T10:41:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.789+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac037070 con 0x7fc0c0154cf0 2024-02-26T10:41:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.789+0000 7fc0bf400640 1 -- 172.21.15.67:0/3534996583 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac031b10 con 0x7fc0c0154cf0 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.789+0000 7fc0c5e00640 1 -- 172.21.15.67:0/3534996583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c0154cf0 msgr2=0x7fc0c00c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.790+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/3534996583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c0154cf0 0x7fc0c00c3300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0097d0 tx=0x7fc0ac02b810 comp rx=0 tx=0).stop 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.790+0000 7fc0c5e00640 1 -- 172.21.15.67:0/3534996583 shutdown_connections 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.790+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/3534996583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c0154cf0 0x7fc0c00c3300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.790+0000 7fc0c5e00640 1 -- 172.21.15.67:0/3534996583 >> 172.21.15.67:0/3534996583 conn(0x7fc0c00c6380 msgr2=0x7fc0c00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.790+0000 7fc0c5e00640 1 -- 172.21.15.67:0/3534996583 shutdown_connections 2024-02-26T10:41:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.790+0000 7fc0c5e00640 1 -- 172.21.15.67:0/3534996583 wait complete. 2024-02-26T10:41:21.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.792+0000 7fc0c5e00640 1 Processor -- start 2024-02-26T10:41:21.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.792+0000 7fc0c5e00640 1 -- start start 2024-02-26T10:41:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 0x7fc0c00aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c01523b0 0x7fc0c01ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c0154cf0 0x7fc0c01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006e520 con 0x7fc0c0071fd0 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006da30 con 0x7fc0c0154cf0 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006e800 con 0x7fc0c01523b0 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 0x7fc0c00aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 0x7fc0c00aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34264/0 (socket says 172.21.15.67:34264) 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 -- 172.21.15.67:0/1694881191 learned_addr learned my addr 172.21.15.67:0/1694881191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0bfe00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c01523b0 0x7fc0c01ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5400640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c0154cf0 0x7fc0c01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 -- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c01523b0 msgr2=0x7fc0c01ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c01523b0 0x7fc0c01ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 -- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c0154cf0 msgr2=0x7fc0c01574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c0154cf0 0x7fc0c01574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c4a00640 1 -- 172.21.15.67:0/1694881191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0156d20 con 0x7fc0c0071fd0 2024-02-26T10:41:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0bfe00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c01523b0 0x7fc0c01ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.793+0000 7fc0c5400640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c0154cf0 0x7fc0c01574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.794+0000 7fc0c4a00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 0x7fc0c00aa0c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c4a0 tx=0x7fc0b400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.794+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4017070 con 0x7fc0c0071fd0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.794+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b4005480 con 0x7fc0c0071fd0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.794+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c00a9560 con 0x7fc0c0071fd0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.794+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4003830 con 0x7fc0c0071fd0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.794+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c01eed10 con 0x7fc0c0071fd0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.795+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c0074a30 con 0x7fc0c0071fd0 2024-02-26T10:41:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.796+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0b401a070 con 0x7fc0c0071fd0 2024-02-26T10:41:21.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.798+0000 7fc0bcc00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0a006b4d0 0x7fc0a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:21.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.798+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc0b4089930 con 0x7fc0c0071fd0 2024-02-26T10:41:21.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.798+0000 7fc0bfe00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0a006b4d0 0x7fc0a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:21.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.799+0000 7fc0bfe00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0a006b4d0 0x7fc0a006d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac036000 tx=0x7fc0ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:21.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:21.802+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0c0156d20 con 0x7fc0c0071fd0 2024-02-26T10:41:22.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.013+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0c00bf840 con 0x7fc0c0071fd0 2024-02-26T10:41:22.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.014+0000 7fc0bcc00640 1 -- 172.21.15.67:0/1694881191 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v60) v1 ==== 105+0+11 (secure 0 0 0) 0x7fc0b4058440 con 0x7fc0c0071fd0 2024-02-26T10:41:22.013 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:22.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0a006b4d0 msgr2=0x7fc0a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0a006b4d0 0x7fc0a006d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac036000 tx=0x7fc0ac008030 comp rx=0 tx=0).stop 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 msgr2=0x7fc0c00aa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 0x7fc0c00aa0c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c4a0 tx=0x7fc0b400c960 comp rx=0 tx=0).stop 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 shutdown_connections 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc0a006b4d0 0x7fc0a006d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0c0154cf0 0x7fc0c01574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0c01523b0 0x7fc0c01ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 --2- 172.21.15.67:0/1694881191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0c0071fd0 0x7fc0c00aa0c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.016+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 >> 172.21.15.67:0/1694881191 conn(0x7fc0c00c6380 msgr2=0x7fc0c014fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.017+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 shutdown_connections 2024-02-26T10:41:22.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.017+0000 7fc0c5e00640 1 -- 172.21.15.67:0/1694881191 wait complete. 2024-02-26T10:41:22.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:41:22.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.258+0000 7f9d12400640 1 Processor -- start 2024-02-26T10:41:22.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.258+0000 7f9d12400640 1 -- start start 2024-02-26T10:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.260+0000 7f9d12400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9d0c0c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.260+0000 7f9d12400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0c3840 con 0x7f9d0c071fd0 2024-02-26T10:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.260+0000 7f9d12400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c06dc80 con 0x7f9d0c154cf0 2024-02-26T10:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.260+0000 7f9d12400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c152790 con 0x7f9d0c1523b0 2024-02-26T10:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.260+0000 7f9d11a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9d0c071fd0 0x7f9d0c072440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47228/0 (socket says 172.21.15.67:47228) 2024-02-26T10:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.260+0000 7f9d11a00640 1 -- 172.21.15.67:0/2420438318 learned_addr learned my addr 172.21.15.67:0/2420438318 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d11000640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9d0c0c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171259182 0 0) 0x7f9d0c0c3840 con 0x7f9d0c071fd0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf0003520 con 0x7f9d0c071fd0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317864352 0 0) 0x7f9cf0003520 con 0x7f9d0c071fd0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 msgr2=0x7f9d0c0c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9d0c0c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 >> v1:172.21.15.82:6789/0 conn(0x7f9d0c154cf0 legacy=0x7f9d0c0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c06df60 con 0x7f9d0c071fd0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f9d04001670 con 0x7f9d0c071fd0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 >> v1:172.21.15.67:6789/0 conn(0x7f9d0c071fd0 legacy=0x7f9d0c072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9cf00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0003fb0 0x7f9cf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0004d40 0x7f9cf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c152790 con 0x7f9cf0003fb0 2024-02-26T10:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000bbb0 con 0x7f9cf0004d40 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.261+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000bdb0 con 0x7f9d0c1523b0 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0003fb0 0x7f9cf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d11000640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9cf00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d11a00640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0004d40 0x7f9cf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d0be00640 1 -- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 msgr2=0x7f9cf00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9cf00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d0be00640 1 -- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0004d40 msgr2=0x7f9cf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0004d40 0x7f9cf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d0be00640 1 -- 172.21.15.67:0/2420438318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0003520 con 0x7f9cf0003fb0 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d11a00640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0004d40 0x7f9cf00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.262+0000 7f9d11000640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c1523b0 0x7f9cf00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.263+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0003fb0 0x7f9cf0004800 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4001750 tx=0x7f9cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393d2ae358a18e46 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4001370 con 0x7f9cf0003fb0 2024-02-26T10:41:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf40052d0 con 0x7f9cf0003fb0 2024-02-26T10:41:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d0b400640 1 -- 172.21.15.67:0/2420438318 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4003830 con 0x7f9cf0003fb0 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d12400640 1 -- 172.21.15.67:0/2420438318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0003fb0 msgr2=0x7f9cf0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d12400640 1 --2- 172.21.15.67:0/2420438318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0003fb0 0x7f9cf0004800 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4001750 tx=0x7f9cf0005830 comp rx=0 tx=0).stop 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d11a00640 1 -- 172.21.15.67:0/2420438318 reap_dead start 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d12400640 1 -- 172.21.15.67:0/2420438318 shutdown_connections 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d12400640 1 -- 172.21.15.67:0/2420438318 >> 172.21.15.67:0/2420438318 conn(0x7f9d0c0c6380 msgr2=0x7f9d0c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d12400640 1 -- 172.21.15.67:0/2420438318 shutdown_connections 2024-02-26T10:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.264+0000 7f9d12400640 1 -- 172.21.15.67:0/2420438318 wait complete. 2024-02-26T10:41:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.266+0000 7f9d12400640 1 Processor -- start 2024-02-26T10:41:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.266+0000 7f9d12400640 1 -- start start 2024-02-26T10:41:22.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d12400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0003fb0 0x7f9d0c0aa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d12400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c0abcf0 0x7f9d0c1ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d12400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 0x7f9d0c14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d12400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c06e520 con 0x7f9d0c1ede70 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d12400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c06da30 con 0x7f9cf0003fb0 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d12400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c06e800 con 0x7f9d0c0abcf0 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.267+0000 7f9d11a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 0x7f9d0c14c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 0x7f9d0c14c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34280/0 (socket says 172.21.15.67:34280) 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 -- 172.21.15.67:0/1741210063 learned_addr learned my addr 172.21.15.67:0/1741210063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11000640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0003fb0 0x7f9d0c0aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d0be00640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c0abcf0 0x7f9d0c1ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 -- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c0abcf0 msgr2=0x7f9d0c1ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c0abcf0 0x7f9d0c1ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 -- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0003fb0 msgr2=0x7f9d0c0aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0003fb0 0x7f9d0c0aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 -- 172.21.15.67:0/1741210063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c1ee860 con 0x7f9d0c1ede70 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.268+0000 7f9d11a00640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 0x7f9d0c14c970 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0c71d0 tx=0x7f9d0400d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.269+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d040074e0 con 0x7f9d0c1ede70 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.269+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040047e0 con 0x7f9d0c1ede70 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.269+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004bb0 con 0x7f9d0c1ede70 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.269+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c0a9d30 con 0x7f9d0c1ede70 2024-02-26T10:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.269+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0aa2f0 con 0x7f9d0c1ede70 2024-02-26T10:41:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.270+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c0678e0 con 0x7f9d0c1ede70 2024-02-26T10:41:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.271+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d04014430 con 0x7f9d0c1ede70 2024-02-26T10:41:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.272+0000 7f9d08c00640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ce806b500 0x7f9ce806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.272+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f9d04089ea0 con 0x7f9d0c1ede70 2024-02-26T10:41:22.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.273+0000 7f9d11000640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ce806b500 0x7f9ce806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.274+0000 7f9d11000640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ce806b500 0x7f9ce806d9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc003be0 tx=0x7f9cfc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:22.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.276+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d04058a20 con 0x7f9d0c1ede70 2024-02-26T10:41:22.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.480+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f9d0c074a30 con 0x7f9d0c1ede70 2024-02-26T10:41:22.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.481+0000 7f9d08c00640 1 -- 172.21.15.67:0/1741210063 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v60) v1 ==== 101+0+11 (secure 0 0 0) 0x7f9d04058320 con 0x7f9d0c1ede70 2024-02-26T10:41:22.480 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.483+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ce806b500 msgr2=0x7f9ce806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.483+0000 7f9d12400640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ce806b500 0x7f9ce806d9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc003be0 tx=0x7f9cfc00b040 comp rx=0 tx=0).stop 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.483+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 msgr2=0x7f9d0c14c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.483+0000 7f9d12400640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 0x7f9d0c14c970 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0c71d0 tx=0x7f9d0400d900 comp rx=0 tx=0).stop 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 shutdown_connections 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ede70 0x7f9d0c14c970 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d0c0abcf0 0x7f9d0c1ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ce806b500 0x7f9ce806d9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 --2- 172.21.15.67:0/1741210063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf0003fb0 0x7f9d0c0aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 >> 172.21.15.67:0/1741210063 conn(0x7f9d0c0c6380 msgr2=0x7f9d0c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:22.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 shutdown_connections 2024-02-26T10:41:22.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.484+0000 7f9d12400640 1 -- 172.21.15.67:0/1741210063 wait complete. 2024-02-26T10:41:22.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:41:22.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.732+0000 7faa10a00640 1 Processor -- start 2024-02-26T10:41:22.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.732+0000 7faa10a00640 1 -- start start 2024-02-26T10:41:22.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa10a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa0c06ac20 0x7faa0c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa10a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faa0c06db00 0x7faa0c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa10a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0c06b460 con 0x7faa0c06b000 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa10a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c0a9a80 con 0x7faa0c06ac20 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa10a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c0a9d60 con 0x7faa0c06db00 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa0b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa0c06ac20 0x7faa0c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa0aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faa0c06b000 0x7faa0c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47234/0 (socket says 172.21.15.67:47234) 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.734+0000 7faa0aa00640 1 -- 172.21.15.67:0/2624280669 learned_addr learned my addr 172.21.15.67:0/2624280669 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.735+0000 7faa0be00640 1 --2- 172.21.15.67:0/2624280669 >> v2:172.21.15.82:3300/0 conn(0x7faa0c06db00 0x7faa0c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.735+0000 7faa0b400640 1 -- 172.21.15.67:0/2624280669 >> v2:172.21.15.82:3300/0 conn(0x7faa0c06db00 msgr2=0x7faa0c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.735+0000 7faa0b400640 1 --2- 172.21.15.67:0/2624280669 >> v2:172.21.15.82:3300/0 conn(0x7faa0c06db00 0x7faa0c0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.735+0000 7faa0b400640 1 -- 172.21.15.67:0/2624280669 >> v1:172.21.15.67:6789/0 conn(0x7faa0c06b000 legacy=0x7faa0c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.735+0000 7faa0b400640 1 -- 172.21.15.67:0/2624280669 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c06df90 con 0x7faa0c06ac20 2024-02-26T10:41:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.736+0000 7faa0b400640 1 --2- 172.21.15.67:0/2624280669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0a94a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7faa0c157220 tx=0x7fa9fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaa3f5e85a7c6d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.736+0000 7faa0a000640 1 -- 172.21.15.67:0/2624280669 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035020 con 0x7faa0c06ac20 2024-02-26T10:41:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.736+0000 7faa0a000640 1 -- 172.21.15.67:0/2624280669 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc031830 con 0x7faa0c06ac20 2024-02-26T10:41:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.736+0000 7faa0a000640 1 -- 172.21.15.67:0/2624280669 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc031c00 con 0x7faa0c06ac20 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 -- 172.21.15.67:0/2624280669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 msgr2=0x7faa0c0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 --2- 172.21.15.67:0/2624280669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0a94a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7faa0c157220 tx=0x7fa9fc02b8d0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 -- 172.21.15.67:0/2624280669 shutdown_connections 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 --2- 172.21.15.67:0/2624280669 >> v2:172.21.15.82:3300/0 conn(0x7faa0c06db00 0x7faa0c0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 --2- 172.21.15.67:0/2624280669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0a94a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 -- 172.21.15.67:0/2624280669 >> 172.21.15.67:0/2624280669 conn(0x7faa0c0c63f0 msgr2=0x7faa0c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 -- 172.21.15.67:0/2624280669 shutdown_connections 2024-02-26T10:41:22.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.737+0000 7faa10a00640 1 -- 172.21.15.67:0/2624280669 wait complete. 2024-02-26T10:41:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.739+0000 7faa10a00640 1 Processor -- start 2024-02-26T10:41:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.739+0000 7faa10a00640 1 -- start start 2024-02-26T10:41:22.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa10a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa10a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faa0c06b000 0x7faa0c1eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa10a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c06db00 0x7faa0c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa10a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0ac490 con 0x7faa0c06ac20 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa10a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0aba30 con 0x7faa0c06db00 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa10a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c150410 con 0x7faa0c06b000 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa0b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa0b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0bfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34288/0 (socket says 172.21.15.67:34288) 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa0b400640 1 -- 172.21.15.67:0/3776452966 learned_addr learned my addr 172.21.15.67:0/3776452966 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa0be00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c06db00 0x7faa0c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.740+0000 7faa0aa00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faa0c06b000 0x7faa0c1eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0b400640 1 -- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faa0c06b000 msgr2=0x7faa0c1eb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0b400640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faa0c06b000 0x7faa0c1eb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0b400640 1 -- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c06db00 msgr2=0x7faa0c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0b400640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c06db00 0x7faa0c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0b400640 1 -- 172.21.15.67:0/3776452966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c1ea950 con 0x7faa0c06ac20 2024-02-26T10:41:22.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0aa00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faa0c06b000 0x7faa0c1eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:22.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa0b400640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0bfa90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009890 tx=0x7fa9fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc041070 con 0x7faa0c06ac20 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9fc03aa30 con 0x7faa0c06ac20 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035040 con 0x7faa0c06ac20 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c070ea0 con 0x7faa0c06ac20 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.741+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c071460 con 0x7faa0c06ac20 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.743+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c074a30 con 0x7faa0c06ac20 2024-02-26T10:41:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.744+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9fc038020 con 0x7faa0c06ac20 2024-02-26T10:41:22.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.744+0000 7faa03400640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:22.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.744+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6623+0+0 (secure 0 0 0) 0x7fa9fc0ac100 con 0x7faa0c06ac20 2024-02-26T10:41:22.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.745+0000 7faa0aa00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:22.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.746+0000 7faa0aa00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faa0c157240 tx=0x7fa9f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:22.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.749+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9fc02f2e0 con 0x7faa0c06ac20 2024-02-26T10:41:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.970+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa0c06b9b0 con 0x7faa0c06ac20 2024-02-26T10:41:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.971+0000 7faa03400640 1 -- 172.21.15.67:0/3776452966 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa9fc07ab50 con 0x7faa0c06ac20 2024-02-26T10:41:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.973+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa9ec06b4d0 msgr2=0x7fa9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.973+0000 7faa10a00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faa0c157240 tx=0x7fa9f8009040 comp rx=0 tx=0).stop 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.973+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 msgr2=0x7faa0c0bfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.973+0000 7faa10a00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0bfa90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009890 tx=0x7fa9fc0046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 shutdown_connections 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa9ec06b4d0 0x7fa9ec06d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c06db00 0x7faa0c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faa0c06b000 0x7faa0c1eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 --2- 172.21.15.67:0/3776452966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c06ac20 0x7faa0c0bfa90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 >> 172.21.15.67:0/3776452966 conn(0x7faa0c0c63f0 msgr2=0x7faa0c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 shutdown_connections 2024-02-26T10:41:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:22.974+0000 7faa10a00640 1 -- 172.21.15.67:0/3776452966 wait complete. 2024-02-26T10:41:22.992 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:41:22.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:22.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:41:23.104 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:41:23.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:41:23.201 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:41:23.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.488+0000 7fd3abe00640 1 Processor -- start 2024-02-26T10:41:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.488+0000 7fd3abe00640 1 -- start start 2024-02-26T10:41:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.489+0000 7fd3abe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3ac156ad0 0x7fd3ac070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.489+0000 7fd3abe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ac0c8d40 con 0x7fd3ac153b70 2024-02-26T10:41:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.489+0000 7fd3abe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ac152dd0 con 0x7fd3ac0c79d0 2024-02-26T10:41:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.489+0000 7fd3abe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c4ae0 con 0x7fd3ac156ad0 2024-02-26T10:41:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.490+0000 7fd3ab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd3ac153b70 0x7fd3ac0c7db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47240/0 (socket says 172.21.15.67:47240) 2024-02-26T10:41:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.490+0000 7fd3ab400640 1 -- 172.21.15.67:0/166225331 learned_addr learned my addr 172.21.15.67:0/166225331 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.490+0000 7fd3aa000640 1 --2- 172.21.15.67:0/166225331 >> v2:172.21.15.82:3300/0 conn(0x7fd3ac156ad0 0x7fd3ac070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484920838 0 0) 0x7fd3ac0c8d40 con 0x7fd3ac153b70 2024-02-26T10:41:23.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd390003520 con 0x7fd3ac153b70 2024-02-26T10:41:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057359651 0 0) 0x7fd3ac152dd0 con 0x7fd3ac0c79d0 2024-02-26T10:41:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ac0c8d40 con 0x7fd3ac0c79d0 2024-02-26T10:41:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098967217 0 0) 0x7fd390003520 con 0x7fd3ac153b70 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 >> v2:172.21.15.82:3300/0 conn(0x7fd3ac156ad0 msgr2=0x7fd3ac070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 --2- 172.21.15.67:0/166225331 >> v2:172.21.15.82:3300/0 conn(0x7fd3ac156ad0 0x7fd3ac070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 >> v1:172.21.15.82:6789/0 conn(0x7fd3ac0c79d0 legacy=0x7fd3ac1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac153090 con 0x7fd3ac153b70 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fd3a4001670 con 0x7fd3ac153b70 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 >> v1:172.21.15.67:6789/0 conn(0x7fd3ac153b70 legacy=0x7fd3ac0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd390003b40 0x7fd390003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390004800 0x7fd390004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd390005230 0x7fd39000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.491+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c4ae0 con 0x7fd390003b40 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39000c190 con 0x7fd390004800 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39000c390 con 0x7fd390005230 2024-02-26T10:41:23.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd390003b40 0x7fd390003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aa000640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390004800 0x7fd390004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aaa00640 1 -- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd390005230 msgr2=0x7fd39000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd390005230 0x7fd39000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aaa00640 1 -- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390004800 msgr2=0x7fd390004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390004800 0x7fd390004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3aaa00640 1 -- 172.21.15.67:0/166225331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390003520 con 0x7fd390003b40 2024-02-26T10:41:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.492+0000 7fd3ab400640 1 -- 172.21.15.67:0/166225331 reap_dead start 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.493+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd390003b40 0x7fd390003f40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd3940018d0 tx=0x7fd390005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a7de66d557c43f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.493+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394037070 con 0x7fd390003b40 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.493+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3940052d0 con 0x7fd390003b40 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3a9600640 1 -- 172.21.15.67:0/166225331 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3940014f0 con 0x7fd390003b40 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 -- 172.21.15.67:0/166225331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd390003b40 msgr2=0x7fd390003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd390003b40 0x7fd390003f40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd3940018d0 tx=0x7fd390005830 comp rx=0 tx=0).stop 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 -- 172.21.15.67:0/166225331 shutdown_connections 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 --2- 172.21.15.67:0/166225331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd390003b40 0x7fd390003f40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 -- 172.21.15.67:0/166225331 >> 172.21.15.67:0/166225331 conn(0x7fd3ac06ac20 msgr2=0x7fd3ac06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 -- 172.21.15.67:0/166225331 shutdown_connections 2024-02-26T10:41:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.494+0000 7fd3abe00640 1 -- 172.21.15.67:0/166225331 wait complete. 2024-02-26T10:41:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.496+0000 7fd3abe00640 1 Processor -- start 2024-02-26T10:41:23.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.496+0000 7fd3abe00640 1 -- start start 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.497+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390003b40 0x7fd3ac14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.497+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 0x7fd3ac14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.497+0000 7fd3abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3ac0bde50 0x7fd3ac14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.497+0000 7fd3abe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c46a0 con 0x7fd3ac0c2120 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.497+0000 7fd3abe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c4520 con 0x7fd390003b40 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.497+0000 7fd3abe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c4820 con 0x7fd3ac0bde50 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 0x7fd3ac14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 0x7fd3ac14c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34298/0 (socket says 172.21.15.67:34298) 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 -- 172.21.15.67:0/2070951327 learned_addr learned my addr 172.21.15.67:0/2070951327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390003b40 0x7fd3ac14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3ab400640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3ac0bde50 0x7fd3ac14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 -- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3ac0bde50 msgr2=0x7fd3ac14cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3ac0bde50 0x7fd3ac14cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 -- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390003b40 msgr2=0x7fd3ac14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390003b40 0x7fd3ac14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.498+0000 7fd3aa000640 1 -- 172.21.15.67:0/2070951327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0be3f0 con 0x7fd3ac0c2120 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.499+0000 7fd3aa000640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 0x7fd3ac14c700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00f460 tx=0x7fd39c00f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.499+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0147e0 con 0x7fd3ac0c2120 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.499+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c01b070 con 0x7fd3ac0c2120 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.499+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac0be6d0 con 0x7fd3ac0c2120 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.499+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c014bc0 con 0x7fd3ac0c2120 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.499+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac0c0390 con 0x7fd3ac0c2120 2024-02-26T10:41:23.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.500+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac074a30 con 0x7fd3ac0c2120 2024-02-26T10:41:23.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.501+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd39c005d10 con 0x7fd3ac0c2120 2024-02-26T10:41:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.502+0000 7fd3a3400640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd38806b4d0 0x7fd38806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.502+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7fd39c08b400 con 0x7fd3ac0c2120 2024-02-26T10:41:23.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.503+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd38806b4d0 0x7fd38806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.504+0000 7fd3aaa00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd38806b4d0 0x7fd38806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd394034000 tx=0x7fd394031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.507+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd39c025020 con 0x7fd3ac0c2120 2024-02-26T10:41:23.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.746+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ac0c3bf0 con 0x7fd3ac0c2120 2024-02-26T10:41:23.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.747+0000 7fd3a3400640 1 -- 172.21.15.67:0/2070951327 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+6182 (secure 0 0 0) 0x7fd39c059e50 con 0x7fd3ac0c2120 2024-02-26T10:41:23.747 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:23.747 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-02-26T10:41:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.750+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd38806b4d0 msgr2=0x7fd38806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.750+0000 7fd3abe00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd38806b4d0 0x7fd38806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd394034000 tx=0x7fd394031000 comp rx=0 tx=0).stop 2024-02-26T10:41:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.750+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 msgr2=0x7fd3ac14c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.750+0000 7fd3abe00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 0x7fd3ac14c700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00f460 tx=0x7fd39c00f920 comp rx=0 tx=0).stop 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 shutdown_connections 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3ac0bde50 0x7fd3ac14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ac0c2120 0x7fd3ac14c700 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd38806b4d0 0x7fd38806d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 --2- 172.21.15.67:0/2070951327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390003b40 0x7fd3ac14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 >> 172.21.15.67:0/2070951327 conn(0x7fd3ac06ac20 msgr2=0x7fd3ac0c4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 shutdown_connections 2024-02-26T10:41:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.751+0000 7fd3abe00640 1 -- 172.21.15.67:0/2070951327 wait complete. 2024-02-26T10:41:23.768 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:23.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.994+0000 7fc742e00640 1 Processor -- start 2024-02-26T10:41:23.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.994+0000 7fc742e00640 1 -- start start 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc73c0a94a0 0x7fc73c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc73c06db00 0x7fc73c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c06df70 con 0x7fc73c06ac20 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c0a9a80 con 0x7fc73c06db00 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c0a9d60 con 0x7fc73c0a94a0 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc73c06db00 0x7fc73c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc73c06db00 0x7fc73c0abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34300/0 (socket says 172.21.15.67:34300) 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc742400640 1 -- 172.21.15.67:0/4278142865 learned_addr learned my addr 172.21.15.67:0/4278142865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:23.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc741a00640 1 --1- 172.21.15.67:0/4278142865 >> v1:172.21.15.67:6789/0 conn(0x7fc73c06ac20 0x7fc73c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47244/0 (socket says 172.21.15.67:47244) 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.996+0000 7fc741000640 1 --2- 172.21.15.67:0/4278142865 >> v2:172.21.15.82:3300/0 conn(0x7fc73c0a94a0 0x7fc73c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.997+0000 7fc742400640 1 -- 172.21.15.67:0/4278142865 >> v2:172.21.15.82:3300/0 conn(0x7fc73c0a94a0 msgr2=0x7fc73c06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.997+0000 7fc742400640 1 --2- 172.21.15.67:0/4278142865 >> v2:172.21.15.82:3300/0 conn(0x7fc73c0a94a0 0x7fc73c06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.997+0000 7fc742400640 1 -- 172.21.15.67:0/4278142865 >> v1:172.21.15.67:6789/0 conn(0x7fc73c06ac20 legacy=0x7fc73c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.997+0000 7fc742400640 1 -- 172.21.15.67:0/4278142865 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c0ab750 con 0x7fc73c06db00 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.997+0000 7fc73be00640 1 -- 172.21.15.67:0/4278142865 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588991228 0 0) 0x7fc73c06df70 con 0x7fc73c06ac20 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.998+0000 7fc742400640 1 --2- 172.21.15.67:0/4278142865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06db00 0x7fc73c0abf50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0c2dd0 tx=0x7fc73402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720a436237745d8c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.998+0000 7fc73be00640 1 -- 172.21.15.67:0/4278142865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73402cb20 con 0x7fc73c06db00 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.998+0000 7fc73be00640 1 -- 172.21.15.67:0/4278142865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73402ccc0 con 0x7fc73c06db00 2024-02-26T10:41:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.998+0000 7fc73be00640 1 -- 172.21.15.67:0/4278142865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734034610 con 0x7fc73c06db00 2024-02-26T10:41:23.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 -- 172.21.15.67:0/4278142865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06db00 msgr2=0x7fc73c0abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:23.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 --2- 172.21.15.67:0/4278142865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06db00 0x7fc73c0abf50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0c2dd0 tx=0x7fc73402bf80 comp rx=0 tx=0).stop 2024-02-26T10:41:23.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 -- 172.21.15.67:0/4278142865 shutdown_connections 2024-02-26T10:41:23.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 --2- 172.21.15.67:0/4278142865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06db00 0x7fc73c0abf50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 --2- 172.21.15.67:0/4278142865 >> v2:172.21.15.82:3300/0 conn(0x7fc73c0a94a0 0x7fc73c06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:23.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 -- 172.21.15.67:0/4278142865 >> 172.21.15.67:0/4278142865 conn(0x7fc73c0c63f0 msgr2=0x7fc73c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:23.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 -- 172.21.15.67:0/4278142865 shutdown_connections 2024-02-26T10:41:23.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:23.999+0000 7fc742e00640 1 -- 172.21.15.67:0/4278142865 wait complete. 2024-02-26T10:41:23.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.001+0000 7fc742e00640 1 Processor -- start 2024-02-26T10:41:24.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.001+0000 7fc742e00640 1 -- start start 2024-02-26T10:41:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc742e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 0x7fc73c0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc742e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc73c06db00 0x7fc73c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc742e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc73c0a94a0 0x7fc73c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc742e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c150410 con 0x7fc73c06ac20 2024-02-26T10:41:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc742e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0ac490 con 0x7fc73c06db00 2024-02-26T10:41:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc742e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c150590 con 0x7fc73c0a94a0 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc741a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 0x7fc73c0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc741a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 0x7fc73c0bfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34314/0 (socket says 172.21.15.67:34314) 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc741a00640 1 -- 172.21.15.67:0/3994029603 learned_addr learned my addr 172.21.15.67:0/3994029603 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc741000640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc73c06db00 0x7fc73c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc741a00640 1 -- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc73c0a94a0 msgr2=0x7fc73c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.002+0000 7fc741a00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc73c0a94a0 0x7fc73c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc741a00640 1 -- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc73c06db00 msgr2=0x7fc73c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc741a00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc73c06db00 0x7fc73c1559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc741a00640 1 -- 172.21.15.67:0/3994029603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c1ee430 con 0x7fc73c06ac20 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc741a00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 0x7fc73c0bfa90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c4d0 tx=0x7fc73000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730018070 con 0x7fc73c06ac20 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730007500 con 0x7fc73c06ac20 2024-02-26T10:41:24.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73c0c6040 con 0x7fc73c06ac20 2024-02-26T10:41:24.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730004a20 con 0x7fc73c06ac20 2024-02-26T10:41:24.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.003+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c1515e0 con 0x7fc73c06ac20 2024-02-26T10:41:24.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.005+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73c074a30 con 0x7fc73c06ac20 2024-02-26T10:41:24.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.005+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc73001b070 con 0x7fc73c06ac20 2024-02-26T10:41:24.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.006+0000 7fc739600640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.007+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7fc7300898a0 con 0x7fc73c06ac20 2024-02-26T10:41:24.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.007+0000 7fc741000640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.008+0000 7fc741000640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc71c06b4d0 0x7fc71c06d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc728006fd0 tx=0x7fc728008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.011+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc73c1ee430 con 0x7fc73c06ac20 2024-02-26T10:41:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.210+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc73c0c0df0 con 0x7fc73c06ac20 2024-02-26T10:41:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.213+0000 7fc739600640 1 -- 172.21.15.67:0/3994029603 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+20472 (secure 0 0 0) 0x7fc730058380 con 0x7fc73c06ac20 2024-02-26T10:41:24.213 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:24.213 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":61,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:22.912213+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc71c06b4d0 msgr2=0x7fc71c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc71c06b4d0 0x7fc71c06d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc728006fd0 tx=0x7fc728008040 comp rx=0 tx=0).stop 2024-02-26T10:41:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 msgr2=0x7fc73c0bfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 0x7fc73c0bfa90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c4d0 tx=0x7fc73000c990 comp rx=0 tx=0).stop 2024-02-26T10:41:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 shutdown_connections 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc73c0a94a0 0x7fc73c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc73c06db00 0x7fc73c1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 --2- 172.21.15.67:0/3994029603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc73c06ac20 0x7fc73c0bfa90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 >> 172.21.15.67:0/3994029603 conn(0x7fc73c0c63f0 msgr2=0x7fc73c0c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 shutdown_connections 2024-02-26T10:41:24.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.216+0000 7fc742e00640 1 -- 172.21.15.67:0/3994029603 wait complete. 2024-02-26T10:41:24.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:40:43.472687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:40:44.096497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:40:58.810192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:41:00.247724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:24.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.460+0000 7f351f800640 1 Processor -- start 2024-02-26T10:41:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.460+0000 7f351f800640 1 -- start start 2024-02-26T10:41:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.461+0000 7f351f800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35180c58d0 0x7f35180c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f351f800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518152f40 con 0x7f35180c4c80 2024-02-26T10:41:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f351f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518153220 con 0x7f35180c5cb0 2024-02-26T10:41:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f351f800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3518153d00 con 0x7f35180c58d0 2024-02-26T10:41:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f351d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f35180c4c80 0x7f35180c8310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47250/0 (socket says 172.21.15.67:47250) 2024-02-26T10:41:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f351d800640 1 -- 172.21.15.67:0/3766892193 learned_addr learned my addr 172.21.15.67:0/3766892193 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932792794 0 0) 0x7f3518152f40 con 0x7f35180c4c80 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34fc003520 con 0x7f35180c4c80 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.462+0000 7f351ce00640 1 --2- 172.21.15.67:0/3766892193 >> v2:172.21.15.82:3300/0 conn(0x7f35180c58d0 0x7f35180c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4265848248 0 0) 0x7f34fc003520 con 0x7f35180c4c80 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 >> v2:172.21.15.82:3300/0 conn(0x7f35180c58d0 msgr2=0x7f35180c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 --2- 172.21.15.67:0/3766892193 >> v2:172.21.15.82:3300/0 conn(0x7f35180c58d0 0x7f35180c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 >> v1:172.21.15.82:6789/0 conn(0x7f35180c5cb0 legacy=0x7f35180c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518153500 con 0x7f35180c4c80 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f3510001670 con 0x7f35180c4c80 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 >> v1:172.21.15.67:6789/0 conn(0x7f35180c4c80 legacy=0x7f35180c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34fc0039b0 0x7f34fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f34fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34fc0052e0 0x7f34fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518153d00 con 0x7f34fc004800 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc00c1b0 con 0x7f34fc0052e0 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc00c3b0 con 0x7f34fc0039b0 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f351ce00640 1 --2- 172.21.15.67:0/3766892193 >> v2:172.21.15.82:3300/0 conn(0x7f35180c58d0 0x7f35180c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.463+0000 7f3517e00640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f34fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f3517e00640 1 -- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34fc0039b0 msgr2=0x7f34fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f3517e00640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34fc0039b0 0x7f34fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f351d800640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34fc0052e0 0x7f34fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f3517e00640 1 -- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34fc0052e0 msgr2=0x7f34fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f3517e00640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34fc0052e0 0x7f34fc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f3517e00640 1 -- 172.21.15.67:0/3766892193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc003520 con 0x7f34fc004800 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.464+0000 7f351d800640 1 -- 172.21.15.67:0/3766892193 reap_dead start 2024-02-26T10:41:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.465+0000 7f3517e00640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f34fc004c80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3500001720 tx=0x7f34fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732ea70caaf96d81 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.465+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500001370 con 0x7f34fc004800 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.465+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35000052d0 con 0x7f34fc004800 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.465+0000 7f3517400640 1 -- 172.21.15.67:0/3766892193 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500003830 con 0x7f34fc004800 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 -- 172.21.15.67:0/3766892193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 msgr2=0x7f34fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f34fc004c80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3500001720 tx=0x7f34fc005830 comp rx=0 tx=0).stop 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 -- 172.21.15.67:0/3766892193 shutdown_connections 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 --2- 172.21.15.67:0/3766892193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f34fc004c80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 -- 172.21.15.67:0/3766892193 >> 172.21.15.67:0/3766892193 conn(0x7f351806b040 msgr2=0x7f351806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 -- 172.21.15.67:0/3766892193 shutdown_connections 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.466+0000 7f351f800640 1 -- 172.21.15.67:0/3766892193 wait complete. 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.468+0000 7f351f800640 1 Processor -- start 2024-02-26T10:41:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.468+0000 7f351f800640 1 -- start start 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f35181568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35181498f0 0x7f35181edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35181ee130 0x7f35181ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518071090 con 0x7f34fc004800 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518070f10 con 0x7f35181ee130 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3518071210 con 0x7f35181498f0 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f35181568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f35181568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34340/0 (socket says 172.21.15.67:34340) 2024-02-26T10:41:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f351ce00640 1 -- 172.21.15.67:0/211433558 learned_addr learned my addr 172.21.15.67:0/211433558 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.469+0000 7f3517e00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35181498f0 0x7f35181edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351d800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35181ee130 0x7f35181ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351ce00640 1 -- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35181498f0 msgr2=0x7f35181edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351ce00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35181498f0 0x7f35181edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351ce00640 1 -- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35181ee130 msgr2=0x7f35181ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351ce00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35181ee130 0x7f35181ea680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351ce00640 1 -- 172.21.15.67:0/211433558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35181ee730 con 0x7f34fc004800 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351d800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35181ee130 0x7f35181ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351ce00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f35181568a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3508007f70 tx=0x7f350800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350801a070 con 0x7f34fc004800 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508003b90 con 0x7f34fc004800 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508004030 con 0x7f34fc004800 2024-02-26T10:41:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35180ad2a0 con 0x7f34fc004800 2024-02-26T10:41:24.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.470+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35180abbf0 con 0x7f34fc004800 2024-02-26T10:41:24.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.472+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3518074a30 con 0x7f34fc004800 2024-02-26T10:41:24.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.473+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f350801d070 con 0x7f34fc004800 2024-02-26T10:41:24.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.474+0000 7f3514c00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f34f406b4d0 0x7f34f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.474+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f350808a9f0 con 0x7f34fc004800 2024-02-26T10:41:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.474+0000 7f3517e00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f34f406b4d0 0x7f34f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.476+0000 7f3517e00640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f34f406b4d0 0x7f34f406d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3500033000 tx=0x7f3500009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.478+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3508010070 con 0x7f34fc004800 2024-02-26T10:41:24.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.673+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f351806b760 con 0x7f34fc004800 2024-02-26T10:41:24.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.674+0000 7f3514c00640 1 -- 172.21.15.67:0/211433558 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35080594d0 con 0x7f34fc004800 2024-02-26T10:41:24.673 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:24.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.676+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f34f406b4d0 msgr2=0x7f34f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.676+0000 7f351f800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f34f406b4d0 0x7f34f406d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3500033000 tx=0x7f3500009030 comp rx=0 tx=0).stop 2024-02-26T10:41:24.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.676+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 msgr2=0x7f35181568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.676+0000 7f351f800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f35181568a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3508007f70 tx=0x7f350800d560 comp rx=0 tx=0).stop 2024-02-26T10:41:24.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 shutdown_connections 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f34f406b4d0 0x7f34f406d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35181ee130 0x7f35181ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35181498f0 0x7f35181edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 --2- 172.21.15.67:0/211433558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc004800 0x7f35181568a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 >> 172.21.15.67:0/211433558 conn(0x7f351806b040 msgr2=0x7f35180c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 shutdown_connections 2024-02-26T10:41:24.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.677+0000 7f351f800640 1 -- 172.21.15.67:0/211433558 wait complete. 2024-02-26T10:41:24.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:41:24.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.907+0000 7f1d46400640 1 Processor -- start 2024-02-26T10:41:24.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.907+0000 7f1d46400640 1 -- start start 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d46400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d401503a0 0x7f1d400ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d46400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400a9430 0x7f1d401535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d46400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d40153ae0 con 0x7f1d4006e2d0 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d46400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d40150780 con 0x7f1d401503a0 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d46400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400a98c0 con 0x7f1d400a9430 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d45000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d401503a0 0x7f1d400ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d45000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d401503a0 0x7f1d400ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34352/0 (socket says 172.21.15.67:34352) 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d45000640 1 -- 172.21.15.67:0/1874788283 learned_addr learned my addr 172.21.15.67:0/1874788283 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.909+0000 7f1d45a00640 1 --1- 172.21.15.67:0/1874788283 >> v1:172.21.15.67:6789/0 conn(0x7f1d4006e2d0 0x7f1d400c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47266/0 (socket says 172.21.15.67:47266) 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.910+0000 7f1d3fe00640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400a9430 0x7f1d401535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.910+0000 7f1d45000640 1 -- 172.21.15.67:0/1874788283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400a9430 msgr2=0x7f1d401535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:24.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.910+0000 7f1d45000640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400a9430 0x7f1d401535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.910+0000 7f1d45000640 1 -- 172.21.15.67:0/1874788283 >> v1:172.21.15.67:6789/0 conn(0x7f1d4006e2d0 legacy=0x7f1d400c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.910+0000 7f1d45000640 1 -- 172.21.15.67:0/1874788283 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400a9ba0 con 0x7f1d401503a0 2024-02-26T10:41:24.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.910+0000 7f1d3fe00640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400a9430 0x7f1d401535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:24.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.911+0000 7f1d45000640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d400ab480 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009870 tx=0x7f1d3402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4e4651469fde06 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.911+0000 7f1d3f400640 1 -- 172.21.15.67:0/1874788283 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34034070 con 0x7f1d401503a0 2024-02-26T10:41:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.911+0000 7f1d3f400640 1 -- 172.21.15.67:0/1874788283 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34030840 con 0x7f1d401503a0 2024-02-26T10:41:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.911+0000 7f1d3f400640 1 -- 172.21.15.67:0/1874788283 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34030c10 con 0x7f1d401503a0 2024-02-26T10:41:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 -- 172.21.15.67:0/1874788283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 msgr2=0x7f1d400ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d400ab480 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009870 tx=0x7f1d3402b980 comp rx=0 tx=0).stop 2024-02-26T10:41:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 -- 172.21.15.67:0/1874788283 shutdown_connections 2024-02-26T10:41:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400a9430 0x7f1d401535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 --2- 172.21.15.67:0/1874788283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d400ab480 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 -- 172.21.15.67:0/1874788283 >> 172.21.15.67:0/1874788283 conn(0x7f1d400c6380 msgr2=0x7f1d400c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:24.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 -- 172.21.15.67:0/1874788283 shutdown_connections 2024-02-26T10:41:24.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.912+0000 7f1d46400640 1 -- 172.21.15.67:0/1874788283 wait complete. 2024-02-26T10:41:24.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.914+0000 7f1d46400640 1 Processor -- start 2024-02-26T10:41:24.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.914+0000 7f1d46400640 1 -- start start 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d46400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4006e2d0 0x7f1d40154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d46400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d40156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d46400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400bfc80 0x7f1d400c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d46400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4006ddb0 con 0x7f1d401503a0 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d46400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4006dad0 con 0x7f1d4006e2d0 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d46400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400c0f80 con 0x7f1d400bfc80 2024-02-26T10:41:24.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d40156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d40156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34360/0 (socket says 172.21.15.67:34360) 2024-02-26T10:41:24.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.915+0000 7f1d3fe00640 1 -- 172.21.15.67:0/2817562102 learned_addr learned my addr 172.21.15.67:0/2817562102 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d45a00640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400bfc80 0x7f1d400c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d45000640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4006e2d0 0x7f1d40154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d3fe00640 1 -- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400bfc80 msgr2=0x7f1d400c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d3fe00640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400bfc80 0x7f1d400c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d3fe00640 1 -- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4006e2d0 msgr2=0x7f1d40154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d3fe00640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4006e2d0 0x7f1d40154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:24.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d3fe00640 1 -- 172.21.15.67:0/2817562102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d40154e10 con 0x7f1d401503a0 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d45000640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4006e2d0 0x7f1d40154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.916+0000 7f1d3fe00640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d40156930 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c820 tx=0x7f1d2c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.917+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c015070 con 0x7f1d401503a0 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.917+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c007a30 con 0x7f1d401503a0 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.917+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c007e00 con 0x7f1d401503a0 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.917+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d401550f0 con 0x7f1d401503a0 2024-02-26T10:41:24.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.917+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d400c0130 con 0x7f1d401503a0 2024-02-26T10:41:24.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.918+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d400678e0 con 0x7f1d401503a0 2024-02-26T10:41:24.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.919+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d2c018070 con 0x7f1d401503a0 2024-02-26T10:41:24.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.920+0000 7f1d3cc00640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d2006b4d0 0x7f1d2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:24.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.920+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f1d2c08a610 con 0x7f1d401503a0 2024-02-26T10:41:24.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.920+0000 7f1d45000640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d2006b4d0 0x7f1d2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:24.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.922+0000 7f1d45000640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d2006b4d0 0x7f1d2006d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036000 tx=0x7f1d3403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:24.925+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d2c059060 con 0x7f1d401503a0 2024-02-26T10:41:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.125+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d40074a30 con 0x7f1d401503a0 2024-02-26T10:41:25.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.126+0000 7f1d3cc00640 1 -- 172.21.15.67:0/2817562102 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d2c058960 con 0x7f1d401503a0 2024-02-26T10:41:25.125 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.128+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d2006b4d0 msgr2=0x7f1d2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.128+0000 7f1d46400640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d2006b4d0 0x7f1d2006d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036000 tx=0x7f1d3403b040 comp rx=0 tx=0).stop 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.128+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 msgr2=0x7f1d40156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.128+0000 7f1d46400640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d40156930 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c820 tx=0x7f1d2c00cc20 comp rx=0 tx=0).stop 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 shutdown_connections 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1d400bfc80 0x7f1d400c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d401503a0 0x7f1d40156930 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1d2006b4d0 0x7f1d2006d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 --2- 172.21.15.67:0/2817562102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4006e2d0 0x7f1d40154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 >> 172.21.15.67:0/2817562102 conn(0x7f1d400c6380 msgr2=0x7f1d400c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:25.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 shutdown_connections 2024-02-26T10:41:25.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.129+0000 7f1d46400640 1 -- 172.21.15.67:0/2817562102 wait complete. 2024-02-26T10:41:25.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:41:25.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.371+0000 7f496e800640 1 Processor -- start 2024-02-26T10:41:25.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.371+0000 7f496e800640 1 -- start start 2024-02-26T10:41:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 0x7f4968151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49680c1be0 con 0x7f49680c25a0 2024-02-26T10:41:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496e800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49680c3660 con 0x7f49680c1eb0 2024-02-26T10:41:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49680c1750 con 0x7f4968151980 2024-02-26T10:41:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f49680c25a0 0x7f49680c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47278/0 (socket says 172.21.15.67:47278) 2024-02-26T10:41:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496d400640 1 -- 172.21.15.67:0/2751900269 learned_addr learned my addr 172.21.15.67:0/2751900269 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f496de00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 0x7f4968151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930455790 0 0) 0x7f49680c1be0 con 0x7f49680c25a0 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.373+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4950003520 con 0x7f49680c25a0 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351181093 0 0) 0x7f4950003520 con 0x7f49680c25a0 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 msgr2=0x7f4968151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 0x7f4968151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 >> v1:172.21.15.82:6789/0 conn(0x7f49680c1eb0 legacy=0x7f496814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680c0e20 con 0x7f49680c25a0 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f495c001690 con 0x7f49680c25a0 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 >> v1:172.21.15.67:6789/0 conn(0x7f49680c25a0 legacy=0x7f49680c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49500039e0 0x7f4950003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 0x7f4950004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4950004d40 0x7f49500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680c1750 con 0x7f4950004d40 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495000dd00 con 0x7f49500039e0 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495000df00 con 0x7f4968151980 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.374+0000 7f496de00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4950004d40 0x7f49500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496ca00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 0x7f4950004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496d400640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49500039e0 0x7f4950003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496de00640 1 -- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 msgr2=0x7f4950004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496de00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4968151980 0x7f4950004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496de00640 1 -- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49500039e0 msgr2=0x7f4950003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496de00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49500039e0 0x7f4950003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496de00640 1 -- 172.21.15.67:0/2751900269 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950003520 con 0x7f4950004d40 2024-02-26T10:41:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.375+0000 7f496d400640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49500039e0 0x7f4950003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.376+0000 7f496de00640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4950004d40 0x7f49500051c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4960009fc0 tx=0x7f4950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab186814f297c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.376+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960039070 con 0x7f4950004d40 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.376+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496002cc20 con 0x7f4950004d40 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.376+0000 7f4967e00640 1 -- 172.21.15.67:0/2751900269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960009c10 con 0x7f4950004d40 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496e800640 1 -- 172.21.15.67:0/2751900269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4950004d40 msgr2=0x7f49500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496e800640 1 --2- 172.21.15.67:0/2751900269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4950004d40 0x7f49500051c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4960009fc0 tx=0x7f4950005830 comp rx=0 tx=0).stop 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496de00640 1 -- 172.21.15.67:0/2751900269 reap_dead start 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496e800640 1 -- 172.21.15.67:0/2751900269 shutdown_connections 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496e800640 1 -- 172.21.15.67:0/2751900269 >> 172.21.15.67:0/2751900269 conn(0x7f49680c6380 msgr2=0x7f49680c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496e800640 1 -- 172.21.15.67:0/2751900269 shutdown_connections 2024-02-26T10:41:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.377+0000 7f496e800640 1 -- 172.21.15.67:0/2751900269 wait complete. 2024-02-26T10:41:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.378+0000 7f496e800640 1 Processor -- start 2024-02-26T10:41:25.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.379+0000 7f496e800640 1 -- start start 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.379+0000 7f496e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4950004d40 0x7f49681ee540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.379+0000 7f496e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f496814b590 0x7f49680bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.379+0000 7f496e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 0x7f49681ea490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680c2980 con 0x7f496814e210 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680c13e0 con 0x7f4950004d40 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49680c2b00 con 0x7f496814b590 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 0x7f49681ea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 0x7f49681ea490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34376/0 (socket says 172.21.15.67:34376) 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 -- 172.21.15.67:0/4069767055 learned_addr learned my addr 172.21.15.67:0/4069767055 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496ca00640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f496814b590 0x7f49680bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496d400640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4950004d40 0x7f49681ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 -- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f496814b590 msgr2=0x7f49680bfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f496814b590 0x7f49680bfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 -- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4950004d40 msgr2=0x7f49681ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4950004d40 0x7f49681ee540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.380+0000 7f496de00640 1 -- 172.21.15.67:0/4069767055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680ac310 con 0x7f496814e210 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.381+0000 7f496de00640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 0x7f49681ea490 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f496002c690 tx=0x7f4960009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.381+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960039070 con 0x7f496814e210 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.381+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960044070 con 0x7f496814e210 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.381+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960009c10 con 0x7f496814e210 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.381+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49680ac5f0 con 0x7f496814e210 2024-02-26T10:41:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.381+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49681edd40 con 0x7f496814e210 2024-02-26T10:41:25.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.382+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49680678e0 con 0x7f496814e210 2024-02-26T10:41:25.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.383+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f496003e810 con 0x7f496814e210 2024-02-26T10:41:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.384+0000 7f4965600640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494806b500 0x7f494806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.384+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f49600ac860 con 0x7f496814e210 2024-02-26T10:41:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.385+0000 7f496d400640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494806b500 0x7f494806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.386+0000 7f496d400640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494806b500 0x7f494806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f495c0011e0 tx=0x7f495c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:25.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.389+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f496007b2b0 con 0x7f496814e210 2024-02-26T10:41:25.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.590+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49680a96b0 con 0x7f496814e210 2024-02-26T10:41:25.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.591+0000 7f4965600640 1 -- 172.21.15.67:0/4069767055 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f496007abb0 con 0x7f496814e210 2024-02-26T10:41:25.590 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494806b500 msgr2=0x7f494806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494806b500 0x7f494806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f495c0011e0 tx=0x7f495c009040 comp rx=0 tx=0).stop 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 msgr2=0x7f49681ea490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 0x7f49681ea490 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f496002c690 tx=0x7f4960009750 comp rx=0 tx=0).stop 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 shutdown_connections 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f496814e210 0x7f49681ea490 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f496814b590 0x7f49680bfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494806b500 0x7f494806d9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 --2- 172.21.15.67:0/4069767055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4950004d40 0x7f49681ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 >> 172.21.15.67:0/4069767055 conn(0x7f49680c6380 msgr2=0x7f496806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 shutdown_connections 2024-02-26T10:41:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.594+0000 7f496e800640 1 -- 172.21.15.67:0/4069767055 wait complete. 2024-02-26T10:41:25.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:41:25.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.836+0000 7f207ca00640 1 Processor -- start 2024-02-26T10:41:25.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.837+0000 7f207ca00640 1 -- start start 2024-02-26T10:41:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.838+0000 7f207ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f20781562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.838+0000 7f207ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207806c110 con 0x7f2078155880 2024-02-26T10:41:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.838+0000 7f207ca00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207806d000 con 0x7f20780af8a0 2024-02-26T10:41:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.838+0000 7f207ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780c72c0 con 0x7f20780c5290 2024-02-26T10:41:25.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2078155880 0x7f207806c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47288/0 (socket says 172.21.15.67:47288) 2024-02-26T10:41:25.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076a00640 1 -- 172.21.15.67:0/2443026673 learned_addr learned my addr 172.21.15.67:0/2443026673 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2077400640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f20781562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721906781 0 0) 0x7f207806c110 con 0x7f2078155880 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205c003520 con 0x7f2078155880 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637684262 0 0) 0x7f205c003520 con 0x7f2078155880 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 msgr2=0x7f20781562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f20781562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 >> v1:172.21.15.82:6789/0 conn(0x7f20780af8a0 legacy=0x7f207814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.839+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780c75a0 con 0x7f2078155880 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f2060001670 con 0x7f2078155880 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 >> v1:172.21.15.67:6789/0 conn(0x7f2078155880 legacy=0x7f207806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f205c0039e0 0x7f205c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f205c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c004d40 0x7f205c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780c72c0 con 0x7f205c0039e0 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00baf0 con 0x7f205c004d40 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00bcf0 con 0x7f20780c5290 2024-02-26T10:41:25.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f205c0039e0 0x7f205c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076a00640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f205c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 -- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 msgr2=0x7f205c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f205c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 -- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c004d40 msgr2=0x7f205c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c004d40 0x7f205c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 -- 172.21.15.67:0/2443026673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c003520 con 0x7f205c0039e0 2024-02-26T10:41:25.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2076a00640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20780c5290 0x7f205c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.840+0000 7f2077400640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f205c0039e0 0x7f205c003e00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2068009fc0 tx=0x7f205c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9597f1c400de1b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068039070 con 0x7f205c0039e0 2024-02-26T10:41:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206802cb40 con 0x7f205c0039e0 2024-02-26T10:41:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f2076000640 1 -- 172.21.15.67:0/2443026673 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068009c10 con 0x7f205c0039e0 2024-02-26T10:41:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f207ca00640 1 -- 172.21.15.67:0/2443026673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f205c0039e0 msgr2=0x7f205c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f207ca00640 1 --2- 172.21.15.67:0/2443026673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f205c0039e0 0x7f205c003e00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2068009fc0 tx=0x7f205c005830 comp rx=0 tx=0).stop 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f2077e00640 1 -- 172.21.15.67:0/2443026673 reap_dead start 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f207ca00640 1 -- 172.21.15.67:0/2443026673 shutdown_connections 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.841+0000 7f207ca00640 1 -- 172.21.15.67:0/2443026673 >> 172.21.15.67:0/2443026673 conn(0x7f20780ac230 msgr2=0x7f20780ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.843+0000 7f207ca00640 1 -- 172.21.15.67:0/2443026673 shutdown_connections 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.843+0000 7f207ca00640 1 -- 172.21.15.67:0/2443026673 wait complete. 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.844+0000 7f207ca00640 1 Processor -- start 2024-02-26T10:41:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.844+0000 7f207ca00640 1 -- start start 2024-02-26T10:41:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f207ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f205c0039e0 0x7f20780c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f207ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 0x7f2078154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f207ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20781eb180 0x7f2078144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f207ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078155cc0 con 0x7f20781ee890 2024-02-26T10:41:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f207ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207806c3d0 con 0x7f20781eb180 2024-02-26T10:41:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f207ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780c5670 con 0x7f205c0039e0 2024-02-26T10:41:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.845+0000 7f2076a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 0x7f2078154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 0x7f2078154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34402/0 (socket says 172.21.15.67:34402) 2024-02-26T10:41:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 -- 172.21.15.67:0/1776338058 learned_addr learned my addr 172.21.15.67:0/1776338058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2077400640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f205c0039e0 0x7f20780c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 -- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f205c0039e0 msgr2=0x7f20780c1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2077e00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20781eb180 0x7f2078144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f205c0039e0 0x7f20780c1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 -- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20781eb180 msgr2=0x7f2078144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20781eb180 0x7f2078144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 -- 172.21.15.67:0/1776338058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780ae720 con 0x7f20781ee890 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2077e00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20781eb180 0x7f2078144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.846+0000 7f2076a00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 0x7f2078154320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f206000d8b0 tx=0x7f206000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.847+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060019070 con 0x7f20781ee890 2024-02-26T10:41:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.847+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20600074e0 con 0x7f20781ee890 2024-02-26T10:41:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.847+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20780aea00 con 0x7f20781ee890 2024-02-26T10:41:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.847+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20600049e0 con 0x7f20781ee890 2024-02-26T10:41:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.847+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20781eed10 con 0x7f20781ee890 2024-02-26T10:41:25.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.848+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078067f30 con 0x7f20781ee890 2024-02-26T10:41:25.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.849+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f206001c070 con 0x7f20781ee890 2024-02-26T10:41:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.850+0000 7f206f400640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f205406b500 0x7f205406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.850+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f206008a0e0 con 0x7f20781ee890 2024-02-26T10:41:25.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.850+0000 7f2077400640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f205406b500 0x7f205406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.852+0000 7f2077400640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f205406b500 0x7f205406d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2068036000 tx=0x7f206802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:25.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:25.855+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2078067f30 con 0x7f20781ee890 2024-02-26T10:41:26.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.056+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20780c6280 con 0x7f20781ee890 2024-02-26T10:41:26.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.057+0000 7f206f400640 1 -- 172.21.15.67:0/1776338058 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v61) v1 ==== 105+0+11 (secure 0 0 0) 0x7f2060058bc0 con 0x7f20781ee890 2024-02-26T10:41:26.055 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f205406b500 msgr2=0x7f205406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f205406b500 0x7f205406d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2068036000 tx=0x7f206802e000 comp rx=0 tx=0).stop 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 msgr2=0x7f2078154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 0x7f2078154320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f206000d8b0 tx=0x7f206000dd70 comp rx=0 tx=0).stop 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 shutdown_connections 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f205406b500 0x7f205406d9c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20781eb180 0x7f2078144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20781ee890 0x7f2078154320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 --2- 172.21.15.67:0/1776338058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f205c0039e0 0x7f20780c1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.060+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 >> 172.21.15.67:0/1776338058 conn(0x7f20780ac230 msgr2=0x7f20780aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.061+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 shutdown_connections 2024-02-26T10:41:26.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.061+0000 7f207ca00640 1 -- 172.21.15.67:0/1776338058 wait complete. 2024-02-26T10:41:26.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:41:26.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.312+0000 7f6fce600640 1 Processor -- start 2024-02-26T10:41:26.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.312+0000 7f6fce600640 1 -- start start 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.313+0000 7f6fce600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fc80c25a0 0x7f6fc80c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.313+0000 7f6fce600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6fc8151980 0x7f6fc8151d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.313+0000 7f6fce600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80c1be0 con 0x7f6fc80c1eb0 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fce600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80c1750 con 0x7f6fc80c25a0 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fce600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80c3660 con 0x7f6fc8151980 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fc80c25a0 0x7f6fc80c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fc80c25a0 0x7f6fc80c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34416/0 (socket says 172.21.15.67:34416) 2024-02-26T10:41:26.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 -- 172.21.15.67:0/3691610325 learned_addr learned my addr 172.21.15.67:0/3691610325 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:26.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcdc00640 1 --2- 172.21.15.67:0/3691610325 >> v2:172.21.15.82:3300/0 conn(0x7f6fc8151980 0x7f6fc8151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 -- 172.21.15.67:0/3691610325 >> v2:172.21.15.82:3300/0 conn(0x7f6fc8151980 msgr2=0x7f6fc8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:26.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 --2- 172.21.15.67:0/3691610325 >> v2:172.21.15.82:3300/0 conn(0x7f6fc8151980 0x7f6fc8151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 -- 172.21.15.67:0/3691610325 >> v1:172.21.15.67:6789/0 conn(0x7f6fc80c1eb0 legacy=0x7f6fc814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fcd200640 1 -- 172.21.15.67:0/3691610325 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80c0e20 con 0x7f6fc80c25a0 2024-02-26T10:41:26.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.314+0000 7f6fc7400640 1 -- 172.21.15.67:0/3691610325 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483300728 0 0) 0x7f6fc80c1be0 con 0x7f6fc80c1eb0 2024-02-26T10:41:26.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.316+0000 7f6fcd200640 1 --2- 172.21.15.67:0/3691610325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc80c2e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80c3b50 tx=0x7f6fb802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e80fbe8d23aea5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:26.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.316+0000 7f6fc7400640 1 -- 172.21.15.67:0/3691610325 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb802ca80 con 0x7f6fc80c25a0 2024-02-26T10:41:26.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.316+0000 7f6fc7400640 1 -- 172.21.15.67:0/3691610325 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb802cc20 con 0x7f6fc80c25a0 2024-02-26T10:41:26.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.316+0000 7f6fc7400640 1 -- 172.21.15.67:0/3691610325 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80343f0 con 0x7f6fc80c25a0 2024-02-26T10:41:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 -- 172.21.15.67:0/3691610325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 msgr2=0x7f6fc80c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 --2- 172.21.15.67:0/3691610325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc80c2e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80c3b50 tx=0x7f6fb802bee0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 -- 172.21.15.67:0/3691610325 shutdown_connections 2024-02-26T10:41:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 --2- 172.21.15.67:0/3691610325 >> v2:172.21.15.82:3300/0 conn(0x7f6fc8151980 0x7f6fc8151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 --2- 172.21.15.67:0/3691610325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc80c2e10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 -- 172.21.15.67:0/3691610325 >> 172.21.15.67:0/3691610325 conn(0x7f6fc80c6380 msgr2=0x7f6fc80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:26.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 -- 172.21.15.67:0/3691610325 shutdown_connections 2024-02-26T10:41:26.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.317+0000 7f6fce600640 1 -- 172.21.15.67:0/3691610325 wait complete. 2024-02-26T10:41:26.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.319+0000 7f6fce600640 1 Processor -- start 2024-02-26T10:41:26.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.319+0000 7f6fce600640 1 -- start start 2024-02-26T10:41:26.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fce600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6fc80c1eb0 0x7f6fc8147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fce600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc8141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fce600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8151980 0x7f6fc8142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fce600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80c3350 con 0x7f6fc80c25a0 2024-02-26T10:41:26.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fce600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80c13e0 con 0x7f6fc8151980 2024-02-26T10:41:26.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fce600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80c2980 con 0x7f6fc80c1eb0 2024-02-26T10:41:26.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc8141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc8141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34432/0 (socket says 172.21.15.67:34432) 2024-02-26T10:41:26.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.320+0000 7f6fc7e00640 1 -- 172.21.15.67:0/1516244782 learned_addr learned my addr 172.21.15.67:0/1516244782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fcdc00640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8151980 0x7f6fc8142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fcd200640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6fc80c1eb0 0x7f6fc8147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc7e00640 1 -- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6fc80c1eb0 msgr2=0x7f6fc8147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc7e00640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6fc80c1eb0 0x7f6fc8147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc7e00640 1 -- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8151980 msgr2=0x7f6fc8142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc7e00640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8151980 0x7f6fc8142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc7e00640 1 -- 172.21.15.67:0/1516244782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8148ff0 con 0x7f6fc80c25a0 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fcdc00640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8151980 0x7f6fc8142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc7e00640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc8141d10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c520 tx=0x7f6fb000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:26.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0017070 con 0x7f6fc80c25a0 2024-02-26T10:41:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb00035f0 con 0x7f6fc80c25a0 2024-02-26T10:41:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.321+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00039e0 con 0x7f6fc80c25a0 2024-02-26T10:41:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.322+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc814b660 con 0x7f6fc80c25a0 2024-02-26T10:41:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.322+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc81496b0 con 0x7f6fc80c25a0 2024-02-26T10:41:26.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.323+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc80678e0 con 0x7f6fc80c25a0 2024-02-26T10:41:26.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.324+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fb001a070 con 0x7f6fc80c25a0 2024-02-26T10:41:26.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.325+0000 7f6fc4c00640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fa406b4d0 0x7f6fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.325+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f6fb0089990 con 0x7f6fc80c25a0 2024-02-26T10:41:26.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.325+0000 7f6fcd200640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fa406b4d0 0x7f6fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.327+0000 7f6fcd200640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fa406b4d0 0x7f6fa406d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8035000 tx=0x7f6fb8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:26.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.330+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fb0058470 con 0x7f6fc80c25a0 2024-02-26T10:41:26.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.528+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f6fc8074a30 con 0x7f6fc80c25a0 2024-02-26T10:41:26.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.529+0000 7f6fc4c00640 1 -- 172.21.15.67:0/1516244782 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v61) v1 ==== 101+0+11 (secure 0 0 0) 0x7f6fb0057d70 con 0x7f6fc80c25a0 2024-02-26T10:41:26.527 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fa406b4d0 msgr2=0x7f6fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fa406b4d0 0x7f6fa406d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8035000 tx=0x7f6fb8030000 comp rx=0 tx=0).stop 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 msgr2=0x7f6fc8141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc8141d10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c520 tx=0x7f6fb000c9e0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 shutdown_connections 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6fa406b4d0 0x7f6fa406d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8151980 0x7f6fc8142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fc80c25a0 0x7f6fc8141d10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 --2- 172.21.15.67:0/1516244782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6fc80c1eb0 0x7f6fc8147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 >> 172.21.15.67:0/1516244782 conn(0x7f6fc80c6380 msgr2=0x7f6fc814bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 shutdown_connections 2024-02-26T10:41:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.532+0000 7f6fce600640 1 -- 172.21.15.67:0/1516244782 wait complete. 2024-02-26T10:41:26.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.775+0000 7fe2dda00640 1 Processor -- start 2024-02-26T10:41:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.775+0000 7fe2dda00640 1 -- start start 2024-02-26T10:41:26.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.776+0000 7fe2dda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8152460 0x7fe2d8158d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.776+0000 7fe2dda00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2d8156230 0x7fe2d80c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.776+0000 7fe2dda00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d8155840 con 0x7fe2d80c54e0 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.777+0000 7fe2dda00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d8158330 con 0x7fe2d8156230 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.777+0000 7fe2dda00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81588f0 con 0x7fe2d8152460 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.777+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8152460 0x7fe2d8158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.777+0000 7fe2d7400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2d8156230 0x7fe2d80c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.777+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8152460 0x7fe2d8158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:51324/0 (socket says 172.21.15.67:51324) 2024-02-26T10:41:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.777+0000 7fe2d7e00640 1 -- 172.21.15.67:0/3783850093 learned_addr learned my addr 172.21.15.67:0/3783850093 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.778+0000 7fe2d7400640 1 -- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8152460 msgr2=0x7fe2d8158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.778+0000 7fe2d7400640 1 --2- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8152460 0x7fe2d8158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.778+0000 7fe2d7400640 1 -- 172.21.15.67:0/3783850093 >> v1:172.21.15.82:6789/0 conn(0x7fe2d80c54e0 legacy=0x7fe2d8154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.778+0000 7fe2d7400640 1 -- 172.21.15.67:0/3783850093 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d8155b20 con 0x7fe2d8156230 2024-02-26T10:41:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.779+0000 7fe2d7400640 1 --2- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d8156230 0x7fe2d80c4fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009d90 tx=0x7fe2c002bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88e88f0c94b7ec server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.780+0000 7fe2d6a00640 1 -- 172.21.15.67:0/3783850093 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0039070 con 0x7fe2d8156230 2024-02-26T10:41:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.780+0000 7fe2d6a00640 1 -- 172.21.15.67:0/3783850093 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c002cb40 con 0x7fe2d8156230 2024-02-26T10:41:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.780+0000 7fe2d6a00640 1 -- 172.21.15.67:0/3783850093 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0034440 con 0x7fe2d8156230 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 -- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d8156230 msgr2=0x7fe2d80c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d8156230 0x7fe2d80c4fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009d90 tx=0x7fe2c002bd90 comp rx=0 tx=0).stop 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 -- 172.21.15.67:0/3783850093 shutdown_connections 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d8156230 0x7fe2d80c4fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3783850093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8152460 0x7fe2d8158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 -- 172.21.15.67:0/3783850093 >> 172.21.15.67:0/3783850093 conn(0x7fe2d80ac1b0 msgr2=0x7fe2d80ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 -- 172.21.15.67:0/3783850093 shutdown_connections 2024-02-26T10:41:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.781+0000 7fe2dda00640 1 -- 172.21.15.67:0/3783850093 wait complete. 2024-02-26T10:41:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.783+0000 7fe2dda00640 1 Processor -- start 2024-02-26T10:41:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.783+0000 7fe2dda00640 1 -- start start 2024-02-26T10:41:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dda00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d80c54e0 0x7fe2d8153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 0x7fe2d80c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8156230 0x7fe2d814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8155490 con 0x7fe2d8152460 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dda00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81592c0 con 0x7fe2d80c54e0 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dda00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80c3c00 con 0x7fe2d8156230 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 0x7fe2d80c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 0x7fe2d80c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34446/0 (socket says 172.21.15.67:34446) 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2d7400640 1 -- 172.21.15.67:0/3531227380 learned_addr learned my addr 172.21.15.67:0/3531227380 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2d7e00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d80c54e0 0x7fe2d8153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.785+0000 7fe2dd000640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8156230 0x7fe2d814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2d7400640 1 -- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8156230 msgr2=0x7fe2d814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2d7400640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8156230 0x7fe2d814d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2d7400640 1 -- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d80c54e0 msgr2=0x7fe2d8153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2d7400640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d80c54e0 0x7fe2d8153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2d7400640 1 -- 172.21.15.67:0/3531227380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d80c3ea0 con 0x7fe2d8152460 2024-02-26T10:41:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2d7400640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 0x7fe2d80c1ee0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0037040 tx=0x7fe2c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0039050 con 0x7fe2d8152460 2024-02-26T10:41:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.786+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0045070 con 0x7fe2d8152460 2024-02-26T10:41:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.787+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c003e850 con 0x7fe2d8152460 2024-02-26T10:41:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.787+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d80c4180 con 0x7fe2d8152460 2024-02-26T10:41:26.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.787+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d80c4740 con 0x7fe2d8152460 2024-02-26T10:41:26.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.788+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8154340 con 0x7fe2d8152460 2024-02-26T10:41:26.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.789+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2c003e9f0 con 0x7fe2d8152460 2024-02-26T10:41:26.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.790+0000 7fe2cfe00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:26.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.790+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7fe2c00ad3e0 con 0x7fe2d8152460 2024-02-26T10:41:26.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.791+0000 7fe2d7e00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.792+0000 7fe2d7e00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe2b406b4d0 0x7fe2b406d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009bd0 tx=0x7fe2c8009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:26.795+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2c007bfe0 con 0x7fe2d8152460 2024-02-26T10:41:27.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.034+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2d8074a30 con 0x7fe2d8152460 2024-02-26T10:41:27.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.035+0000 7fe2cfe00640 1 -- 172.21.15.67:0/3531227380 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+6182 (secure 0 0 0) 0x7fe2c007b8e0 con 0x7fe2d8152460 2024-02-26T10:41:27.033 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:27.034 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:27.034 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-02-26T10:41:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe2b406b4d0 msgr2=0x7fe2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe2b406b4d0 0x7fe2b406d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009bd0 tx=0x7fe2c8009240 comp rx=0 tx=0).stop 2024-02-26T10:41:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 msgr2=0x7fe2d80c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 0x7fe2d80c1ee0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0037040 tx=0x7fe2c0004a40 comp rx=0 tx=0).stop 2024-02-26T10:41:27.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 shutdown_connections 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2d8156230 0x7fe2d814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2d8152460 0x7fe2d80c1ee0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe2b406b4d0 0x7fe2b406d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 --2- 172.21.15.67:0/3531227380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2d80c54e0 0x7fe2d8153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 >> 172.21.15.67:0/3531227380 conn(0x7fe2d80ac1b0 msgr2=0x7fe2d80aa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 shutdown_connections 2024-02-26T10:41:27.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.038+0000 7fe2dda00640 1 -- 172.21.15.67:0/3531227380 wait complete. 2024-02-26T10:41:27.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:27.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.283+0000 7f2e45c00640 1 Processor -- start 2024-02-26T10:41:27.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.284+0000 7f2e45c00640 1 -- start start 2024-02-26T10:41:27.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.285+0000 7f2e45c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e400c7140 0x7f2e400c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.285+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 0x7f2e40070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.285+0000 7f2e45c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400c3750 con 0x7f2e400713e0 2024-02-26T10:41:27.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.285+0000 7f2e45c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400c39f0 con 0x7f2e400c7140 2024-02-26T10:41:27.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.285+0000 7f2e45c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c3cd0 con 0x7f2e40070230 2024-02-26T10:41:27.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.286+0000 7f2e3f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 0x7f2e40070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.286+0000 7f2e3f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 0x7f2e40070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:51342/0 (socket says 172.21.15.67:51342) 2024-02-26T10:41:27.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.286+0000 7f2e3f400640 1 -- 172.21.15.67:0/3562536982 learned_addr learned my addr 172.21.15.67:0/3562536982 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:27.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.286+0000 7f2e3e000640 1 --1- 172.21.15.67:0/3562536982 >> v1:172.21.15.82:6789/0 conn(0x7f2e400713e0 0x7f2e40152400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:33908/0 (socket says 172.21.15.67:33908) 2024-02-26T10:41:27.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.286+0000 7f2e3ea00640 1 --2- 172.21.15.67:0/3562536982 >> v2:172.21.15.82:3300/0 conn(0x7f2e400c7140 0x7f2e400c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.287+0000 7f2e3f400640 1 -- 172.21.15.67:0/3562536982 >> v2:172.21.15.82:3300/0 conn(0x7f2e400c7140 msgr2=0x7f2e400c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:27.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.287+0000 7f2e3f400640 1 --2- 172.21.15.67:0/3562536982 >> v2:172.21.15.82:3300/0 conn(0x7f2e400c7140 0x7f2e400c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.287+0000 7f2e3f400640 1 -- 172.21.15.67:0/3562536982 >> v1:172.21.15.82:6789/0 conn(0x7f2e400713e0 legacy=0x7f2e40152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.287+0000 7f2e3f400640 1 -- 172.21.15.67:0/3562536982 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400c3fb0 con 0x7f2e40070230 2024-02-26T10:41:27.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.288+0000 7f2e3f400640 1 --2- 172.21.15.67:0/3562536982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 0x7f2e40070680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009e50 tx=0x7f2e3802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ded1d90fcad25db server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:27.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.289+0000 7f2e3d600640 1 -- 172.21.15.67:0/3562536982 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3802ca80 con 0x7f2e40070230 2024-02-26T10:41:27.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.289+0000 7f2e3d600640 1 -- 172.21.15.67:0/3562536982 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3802cc20 con 0x7f2e40070230 2024-02-26T10:41:27.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.289+0000 7f2e3d600640 1 -- 172.21.15.67:0/3562536982 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380343f0 con 0x7f2e40070230 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 -- 172.21.15.67:0/3562536982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 msgr2=0x7f2e40070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3562536982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 0x7f2e40070680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009e50 tx=0x7f2e3802bee0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 -- 172.21.15.67:0/3562536982 shutdown_connections 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3562536982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e40070230 0x7f2e40070680 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3562536982 >> v2:172.21.15.82:3300/0 conn(0x7f2e400c7140 0x7f2e400c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 -- 172.21.15.67:0/3562536982 >> 172.21.15.67:0/3562536982 conn(0x7f2e4006b3b0 msgr2=0x7f2e400aa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 -- 172.21.15.67:0/3562536982 shutdown_connections 2024-02-26T10:41:27.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.290+0000 7f2e45c00640 1 -- 172.21.15.67:0/3562536982 wait complete. 2024-02-26T10:41:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.292+0000 7f2e45c00640 1 Processor -- start 2024-02-26T10:41:27.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.292+0000 7f2e45c00640 1 -- start start 2024-02-26T10:41:27.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.293+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 0x7f2e400ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.293+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e400713e0 0x7f2e401497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.293+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e400c7140 0x7f2e4014e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e45c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c7520 con 0x7f2e40070230 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e45c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c6a00 con 0x7f2e400713e0 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e45c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c76a0 con 0x7f2e400c7140 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 0x7f2e400ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 0x7f2e400ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34462/0 (socket says 172.21.15.67:34462) 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 -- 172.21.15.67:0/3970730183 learned_addr learned my addr 172.21.15.67:0/3970730183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3f400640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e400c7140 0x7f2e4014e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3e000640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e400713e0 0x7f2e401497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 -- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e400c7140 msgr2=0x7f2e4014e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e400c7140 0x7f2e4014e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 -- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e400713e0 msgr2=0x7f2e401497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e400713e0 0x7f2e401497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3ea00640 1 -- 172.21.15.67:0/3970730183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4014a810 con 0x7f2e40070230 2024-02-26T10:41:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3e000640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e400713e0 0x7f2e401497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.294+0000 7f2e3f400640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e400c7140 0x7f2e4014e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.295+0000 7f2e3ea00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 0x7f2e400ac040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9e0 tx=0x7f2e3000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.295+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30017070 con 0x7f2e40070230 2024-02-26T10:41:27.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.295+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4014aaf0 con 0x7f2e40070230 2024-02-26T10:41:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.295+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3000c9d0 con 0x7f2e40070230 2024-02-26T10:41:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.295+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000cdc0 con 0x7f2e40070230 2024-02-26T10:41:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.295+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e40148a00 con 0x7f2e40070230 2024-02-26T10:41:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.296+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e40074a30 con 0x7f2e40070230 2024-02-26T10:41:27.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.297+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e3001a070 con 0x7f2e40070230 2024-02-26T10:41:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.298+0000 7f2e36a00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2e2006b4d0 0x7f2e2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.298+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f2e3008a950 con 0x7f2e40070230 2024-02-26T10:41:27.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.299+0000 7f2e3e000640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2e2006b4d0 0x7f2e2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.300+0000 7f2e3e000640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2e2006b4d0 0x7f2e2006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e400a8140 tx=0x7f2e2c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:27.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.303+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e30059450 con 0x7f2e40070230 2024-02-26T10:41:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.505+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e400c5c70 con 0x7f2e40070230 2024-02-26T10:41:27.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.507+0000 7f2e36a00640 1 -- 172.21.15.67:0/3970730183 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+20472 (secure 0 0 0) 0x7f2e30058d50 con 0x7f2e40070230 2024-02-26T10:41:27.506 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:27.506 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":61,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:22.912213+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2e2006b4d0 msgr2=0x7f2e2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2e2006b4d0 0x7f2e2006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e400a8140 tx=0x7f2e2c009040 comp rx=0 tx=0).stop 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 msgr2=0x7f2e400ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 0x7f2e400ac040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000e9e0 tx=0x7f2e3000eea0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 shutdown_connections 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e400c7140 0x7f2e4014e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2e2006b4d0 0x7f2e2006d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e400713e0 0x7f2e401497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 --2- 172.21.15.67:0/3970730183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e40070230 0x7f2e400ac040 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 >> 172.21.15.67:0/3970730183 conn(0x7f2e4006b3b0 msgr2=0x7f2e4014f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 shutdown_connections 2024-02-26T10:41:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.510+0000 7f2e45c00640 1 -- 172.21.15.67:0/3970730183 wait complete. 2024-02-26T10:41:27.529 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T10:41:27.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:27.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.759+0000 7f9dd7e00640 1 Processor -- start 2024-02-26T10:41:27.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.759+0000 7f9dd7e00640 1 -- start start 2024-02-26T10:41:27.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.760+0000 7f9dd7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9dd806c1d0 0x7f9dd806c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.760+0000 7f9dd7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9dd8071d80 0x7f9dd80c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.760+0000 7f9dd7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd80c0b80 con 0x7f9dd8072580 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.760+0000 7f9dd7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd80aa140 con 0x7f9dd806c1d0 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.760+0000 7f9dd7e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd80c0930 con 0x7f9dd8071d80 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9dd806c1d0 0x7f9dd806c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9dd8072580 0x7f9dd8152310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47320/0 (socket says 172.21.15.67:47320) 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6000640 1 -- 172.21.15.67:0/3542234389 learned_addr learned my addr 172.21.15.67:0/3542234389 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:27.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9dd806c1d0 0x7f9dd806c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34472/0 (socket says 172.21.15.67:34472) 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd7400640 1 --2- 172.21.15.67:0/3542234389 >> v2:172.21.15.82:3300/0 conn(0x7f9dd8071d80 0x7f9dd80c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6a00640 1 -- 172.21.15.67:0/3542234389 >> v2:172.21.15.82:3300/0 conn(0x7f9dd8071d80 msgr2=0x7f9dd80c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6a00640 1 --2- 172.21.15.67:0/3542234389 >> v2:172.21.15.82:3300/0 conn(0x7f9dd8071d80 0x7f9dd80c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6a00640 1 -- 172.21.15.67:0/3542234389 >> v1:172.21.15.67:6789/0 conn(0x7f9dd8072580 legacy=0x7f9dd8152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd6a00640 1 -- 172.21.15.67:0/3542234389 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd80c0e60 con 0x7f9dd806c1d0 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.761+0000 7f9dd5600640 1 -- 172.21.15.67:0/3542234389 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555314416 0 0) 0x7f9dd80c0b80 con 0x7f9dd8072580 2024-02-26T10:41:27.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.762+0000 7f9dd6a00640 1 --2- 172.21.15.67:0/3542234389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd806c1d0 0x7f9dd806c5b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9dd806cd90 tx=0x7f9dc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93643440e1eacf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:27.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd5600640 1 -- 172.21.15.67:0/3542234389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0035020 con 0x7f9dd806c1d0 2024-02-26T10:41:27.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd5600640 1 -- 172.21.15.67:0/3542234389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0031830 con 0x7f9dd806c1d0 2024-02-26T10:41:27.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd5600640 1 -- 172.21.15.67:0/3542234389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0031c00 con 0x7f9dd806c1d0 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd7e00640 1 -- 172.21.15.67:0/3542234389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd806c1d0 msgr2=0x7f9dd806c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/3542234389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd806c1d0 0x7f9dd806c5b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9dd806cd90 tx=0x7f9dc002b8d0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd7e00640 1 -- 172.21.15.67:0/3542234389 shutdown_connections 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/3542234389 >> v2:172.21.15.82:3300/0 conn(0x7f9dd8071d80 0x7f9dd80c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/3542234389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd806c1d0 0x7f9dd806c5b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.763+0000 7f9dd7e00640 1 -- 172.21.15.67:0/3542234389 >> 172.21.15.67:0/3542234389 conn(0x7f9dd80c7950 msgr2=0x7f9dd80c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.764+0000 7f9dd7e00640 1 -- 172.21.15.67:0/3542234389 shutdown_connections 2024-02-26T10:41:27.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.764+0000 7f9dd7e00640 1 -- 172.21.15.67:0/3542234389 wait complete. 2024-02-26T10:41:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.765+0000 7f9dd7e00640 1 Processor -- start 2024-02-26T10:41:27.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.765+0000 7f9dd7e00640 1 -- start start 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.766+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dd806c1d0 0x7f9dd81de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.766+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 0x7f9dd81df720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.766+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dd8072580 0x7f9dd81dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.766+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd8154f50 con 0x7f9dd8071d80 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.766+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd80a9cb0 con 0x7f9dd8072580 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.766+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd81550d0 con 0x7f9dd806c1d0 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 0x7f9dd81df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 0x7f9dd81df720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34484/0 (socket says 172.21.15.67:34484) 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 -- 172.21.15.67:0/1028672852 learned_addr learned my addr 172.21.15.67:0/1028672852 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6a00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dd806c1d0 0x7f9dd81de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd7400640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dd8072580 0x7f9dd81dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 -- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dd806c1d0 msgr2=0x7f9dd81de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dd806c1d0 0x7f9dd81de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 -- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dd8072580 msgr2=0x7f9dd81dfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dd8072580 0x7f9dd81dfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 -- 172.21.15.67:0/1028672852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8150ba0 con 0x7f9dd8071d80 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6a00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dd806c1d0 0x7f9dd81de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.767+0000 7f9dd6000640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 0x7f9dd81df720 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c890 tx=0x7f9dc800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.768+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8018070 con 0x7f9dd8071d80 2024-02-26T10:41:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.768+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc800a640 con 0x7f9dd8071d80 2024-02-26T10:41:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.768+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd81dcb30 con 0x7f9dd8071d80 2024-02-26T10:41:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.768+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800aa30 con 0x7f9dd8071d80 2024-02-26T10:41:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.768+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd81dd0f0 con 0x7f9dd8071d80 2024-02-26T10:41:27.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.769+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd8074a30 con 0x7f9dd8071d80 2024-02-26T10:41:27.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.770+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dc801b070 con 0x7f9dd8071d80 2024-02-26T10:41:27.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.771+0000 7f9dcf400640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9db406b500 0x7f9db406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:27.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.772+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f9dc8089be0 con 0x7f9dd8071d80 2024-02-26T10:41:27.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.772+0000 7f9dd6a00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9db406b500 0x7f9db406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:27.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.773+0000 7f9dd6a00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9db406b500 0x7f9db406d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0036000 tx=0x7f9dc0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:27.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:27.776+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dc80586c0 con 0x7f9dd8071d80 2024-02-26T10:41:28.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.017+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dd80c4bd0 con 0x7f9dd8071d80 2024-02-26T10:41:28.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.018+0000 7f9dcf400640 1 -- 172.21.15.67:0/1028672852 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f9dc8057fc0 con 0x7f9dd8071d80 2024-02-26T10:41:28.017 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:28.017 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:28.017 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-02-26T10:41:28.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.021+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9db406b500 msgr2=0x7f9db406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.021+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9db406b500 0x7f9db406d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0036000 tx=0x7f9dc0008030 comp rx=0 tx=0).stop 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.021+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 msgr2=0x7f9dd81df720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.021+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 0x7f9dd81df720 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800c890 tx=0x7f9dc800cd50 comp rx=0 tx=0).stop 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.021+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 shutdown_connections 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9db406b500 0x7f9db406d9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dd8072580 0x7f9dd81dfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dd8071d80 0x7f9dd81df720 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 --2- 172.21.15.67:0/1028672852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dd806c1d0 0x7f9dd81de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 >> 172.21.15.67:0/1028672852 conn(0x7f9dd80c7950 msgr2=0x7f9dd80c7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 shutdown_connections 2024-02-26T10:41:28.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.022+0000 7f9dd7e00640 1 -- 172.21.15.67:0/1028672852 wait complete. 2024-02-26T10:41:28.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:28.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.268+0000 7f30cf600640 1 Processor -- start 2024-02-26T10:41:28.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.269+0000 7f30cf600640 1 -- start start 2024-02-26T10:41:28.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.270+0000 7f30cf600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c80c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.270+0000 7f30cf600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f30c8156230 0x7f30c80c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.270+0000 7f30cf600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c8152e50 con 0x7f30c806e410 2024-02-26T10:41:28.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.270+0000 7f30cf600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c81506e0 con 0x7f30c8156230 2024-02-26T10:41:28.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.270+0000 7f30cf600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8153a60 con 0x7f30c8150300 2024-02-26T10:41:28.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.271+0000 7f30c7e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f30c806e410 0x7f30c80aaba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:33918/0 (socket says 172.21.15.67:33918) 2024-02-26T10:41:28.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.271+0000 7f30c7e00640 1 -- 172.21.15.67:0/282372877 learned_addr learned my addr 172.21.15.67:0/282372877 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:28.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.271+0000 7f30ccc00640 1 --2- 172.21.15.67:0/282372877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c80c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.271+0000 7f30cd600640 1 --2- 172.21.15.67:0/282372877 >> v2:172.21.15.82:3300/0 conn(0x7f30c8156230 0x7f30c80c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.272+0000 7f30ccc00640 1 -- 172.21.15.67:0/282372877 >> v2:172.21.15.82:3300/0 conn(0x7f30c8156230 msgr2=0x7f30c80c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:28.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.272+0000 7f30ccc00640 1 --2- 172.21.15.67:0/282372877 >> v2:172.21.15.82:3300/0 conn(0x7f30c8156230 0x7f30c80c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.272+0000 7f30ccc00640 1 -- 172.21.15.67:0/282372877 >> v1:172.21.15.82:6789/0 conn(0x7f30c806e410 legacy=0x7f30c80aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.272+0000 7f30ccc00640 1 -- 172.21.15.67:0/282372877 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8153130 con 0x7f30c8150300 2024-02-26T10:41:28.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.272+0000 7f30c7400640 1 -- 172.21.15.67:0/282372877 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621172639 0 0) 0x7f30c8152e50 con 0x7f30c806e410 2024-02-26T10:41:28.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.273+0000 7f30ccc00640 1 --2- 172.21.15.67:0/282372877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c80c2ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f30b80097a0 tx=0x7f30b802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bac6f74dce1c6eb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:28.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.274+0000 7f30c7400640 1 -- 172.21.15.67:0/282372877 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8038070 con 0x7f30c8150300 2024-02-26T10:41:28.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.274+0000 7f30c7400640 1 -- 172.21.15.67:0/282372877 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8031700 con 0x7f30c8150300 2024-02-26T10:41:28.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.274+0000 7f30c7400640 1 -- 172.21.15.67:0/282372877 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8031ad0 con 0x7f30c8150300 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 -- 172.21.15.67:0/282372877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 msgr2=0x7f30c80c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 --2- 172.21.15.67:0/282372877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c80c2ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f30b80097a0 tx=0x7f30b802b7a0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 -- 172.21.15.67:0/282372877 shutdown_connections 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 --2- 172.21.15.67:0/282372877 >> v2:172.21.15.82:3300/0 conn(0x7f30c8156230 0x7f30c80c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 --2- 172.21.15.67:0/282372877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c80c2ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 -- 172.21.15.67:0/282372877 >> 172.21.15.67:0/282372877 conn(0x7f30c80a84a0 msgr2=0x7f30c80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 -- 172.21.15.67:0/282372877 shutdown_connections 2024-02-26T10:41:28.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.275+0000 7f30cf600640 1 -- 172.21.15.67:0/282372877 wait complete. 2024-02-26T10:41:28.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.277+0000 7f30cf600640 1 Processor -- start 2024-02-26T10:41:28.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.277+0000 7f30cf600640 1 -- start start 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cf600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30c806e410 0x7f30c80c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cf600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c814a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cf600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 0x7f30c814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cf600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806dd80 con 0x7f30c8156230 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cf600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81536f0 con 0x7f30c806e410 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cf600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80ab2b0 con 0x7f30c8150300 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 0x7f30c814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 0x7f30c814aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34494/0 (socket says 172.21.15.67:34494) 2024-02-26T10:41:28.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30cd600640 1 -- 172.21.15.67:0/2009142916 learned_addr learned my addr 172.21.15.67:0/2009142916 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:28.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30c7e00640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c814a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.278+0000 7f30ccc00640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30c806e410 0x7f30c80c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cd600640 1 -- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 msgr2=0x7f30c814a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cd600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c814a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cd600640 1 -- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30c806e410 msgr2=0x7f30c80c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cd600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30c806e410 0x7f30c80c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cd600640 1 -- 172.21.15.67:0/2009142916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80721b0 con 0x7f30c8156230 2024-02-26T10:41:28.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cd600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 0x7f30c814aba0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009cb0 tx=0x7f30c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0018020 con 0x7f30c8156230 2024-02-26T10:41:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0014480 con 0x7f30c8156230 2024-02-26T10:41:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8072460 con 0x7f30c8156230 2024-02-26T10:41:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0014870 con 0x7f30c8156230 2024-02-26T10:41:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.279+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c80c2180 con 0x7f30c8156230 2024-02-26T10:41:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.280+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8072d60 con 0x7f30c8156230 2024-02-26T10:41:28.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.281+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30c0005d10 con 0x7f30c8156230 2024-02-26T10:41:28.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.282+0000 7f30c4c00640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.282+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f30c008b2a0 con 0x7f30c8156230 2024-02-26T10:41:28.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.283+0000 7f30ccc00640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.284+0000 7f30ccc00640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30a806b4d0 0x7f30a806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f30b8035000 tx=0x7f30b8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:28.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.288+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30c8072d60 con 0x7f30c8156230 2024-02-26T10:41:28.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.530+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30c8074a30 con 0x7f30c8156230 2024-02-26T10:41:28.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.531+0000 7f30c4c00640 1 -- 172.21.15.67:0/2009142916 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+6182 (secure 0 0 0) 0x7f30c0022030 con 0x7f30c8156230 2024-02-26T10:41:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:28.529 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:40:45.064964+0000","modified":"2024-02-26T10:40:48.161474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6834/1127454674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1127454674},{"type":"v1","addr":"172.21.15.67:6834","nonce":1127454674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:28.529 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 37 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30a806b4d0 msgr2=0x7f30a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30a806b4d0 0x7f30a806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f30b8035000 tx=0x7f30b8008030 comp rx=0 tx=0).stop 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 msgr2=0x7f30c814aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 0x7f30c814aba0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009cb0 tx=0x7f30c000c490 comp rx=0 tx=0).stop 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 shutdown_connections 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30c8156230 0x7f30c814aba0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30c8150300 0x7f30c814a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.533+0000 7f30cf600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.534+0000 7f30cf600640 1 --2- 172.21.15.67:0/2009142916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30c806e410 0x7f30c80c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.534+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 >> 172.21.15.67:0/2009142916 conn(0x7f30c80a84a0 msgr2=0x7f30c80a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.534+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 shutdown_connections 2024-02-26T10:41:28.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.534+0000 7f30cf600640 1 -- 172.21.15.67:0/2009142916 wait complete. 2024-02-26T10:41:28.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:28.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.782+0000 7ff9a6000640 1 Processor -- start 2024-02-26T10:41:28.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.782+0000 7ff9a6000640 1 -- start start 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff9a6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff9a00c2310 0x7ff9a0153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff9a6000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a0155f10 con 0x7ff9a006b000 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff9a6000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a00c28d0 con 0x7ff9a006ac20 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff9a6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a006b430 con 0x7ff9a00c2310 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff99fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff9a00c2310 0x7ff9a0153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff99fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff9a00c2310 0x7ff9a0153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34502/0 (socket says 172.21.15.67:34502) 2024-02-26T10:41:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff99fe00640 1 -- 172.21.15.67:0/1336465473 learned_addr learned my addr 172.21.15.67:0/1336465473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.784+0000 7ff9a5600640 1 --1- 172.21.15.67:0/1336465473 >> v1:172.21.15.67:6789/0 conn(0x7ff9a006b000 0x7ff9a015c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47328/0 (socket says 172.21.15.67:47328) 2024-02-26T10:41:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.785+0000 7ff99fe00640 1 -- 172.21.15.67:0/1336465473 >> v1:172.21.15.82:6789/0 conn(0x7ff9a006ac20 legacy=0x7ff9a0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.785+0000 7ff99fe00640 1 -- 172.21.15.67:0/1336465473 >> v1:172.21.15.67:6789/0 conn(0x7ff9a006b000 legacy=0x7ff9a015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.785+0000 7ff99fe00640 1 -- 172.21.15.67:0/1336465473 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0155150 con 0x7ff9a00c2310 2024-02-26T10:41:28.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.786+0000 7ff99fe00640 1 --2- 172.21.15.67:0/1336465473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a0153630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff988009870 tx=0x7ff98802db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f9548918562f23 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:28.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff99f400640 1 -- 172.21.15.67:0/1336465473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98802f8e0 con 0x7ff9a00c2310 2024-02-26T10:41:28.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff99f400640 1 -- 172.21.15.67:0/1336465473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98802fa80 con 0x7ff9a00c2310 2024-02-26T10:41:28.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff99f400640 1 -- 172.21.15.67:0/1336465473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98802fe50 con 0x7ff9a00c2310 2024-02-26T10:41:28.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 -- 172.21.15.67:0/1336465473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 msgr2=0x7ff9a0153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 --2- 172.21.15.67:0/1336465473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a0153630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff988009870 tx=0x7ff98802db20 comp rx=0 tx=0).stop 2024-02-26T10:41:28.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 -- 172.21.15.67:0/1336465473 shutdown_connections 2024-02-26T10:41:28.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 --2- 172.21.15.67:0/1336465473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a0153630 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 -- 172.21.15.67:0/1336465473 >> 172.21.15.67:0/1336465473 conn(0x7ff9a00c63f0 msgr2=0x7ff9a00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:28.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 -- 172.21.15.67:0/1336465473 shutdown_connections 2024-02-26T10:41:28.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.787+0000 7ff9a6000640 1 -- 172.21.15.67:0/1336465473 wait complete. 2024-02-26T10:41:28.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.789+0000 7ff9a6000640 1 Processor -- start 2024-02-26T10:41:28.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.789+0000 7ff9a6000640 1 -- start start 2024-02-26T10:41:28.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9a006ac20 0x7ff9a0150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a6000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a006b000 0x7ff9a01ea610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a01eab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006e1b0 con 0x7ff9a00c2310 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a6000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006e030 con 0x7ff9a006b000 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006e330 con 0x7ff9a006ac20 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a01eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a01eab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34512/0 (socket says 172.21.15.67:34512) 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.790+0000 7ff9a5600640 1 -- 172.21.15.67:0/3806735306 learned_addr learned my addr 172.21.15.67:0/3806735306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a4c00640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9a006ac20 0x7ff9a0150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff99fe00640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a006b000 0x7ff9a01ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a5600640 1 -- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9a006ac20 msgr2=0x7ff9a0150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a5600640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9a006ac20 0x7ff9a0150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a5600640 1 -- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a006b000 msgr2=0x7ff9a01ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a5600640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a006b000 0x7ff9a01ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a5600640 1 -- 172.21.15.67:0/3806735306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a01eb090 con 0x7ff9a00c2310 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.791+0000 7ff9a5600640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a01eab50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c780 tx=0x7ff99800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.792+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998017070 con 0x7ff9a00c2310 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.792+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800a5e0 con 0x7ff9a00c2310 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.792+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800a990 con 0x7ff9a00c2310 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.792+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a00c4e90 con 0x7ff9a00c2310 2024-02-26T10:41:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.792+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a0150fb0 con 0x7ff9a00c2310 2024-02-26T10:41:28.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.793+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a0074a30 con 0x7ff9a00c2310 2024-02-26T10:41:28.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.794+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff99801a070 con 0x7ff9a00c2310 2024-02-26T10:41:28.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.795+0000 7ff99cc00640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff97c06b500 0x7ff97c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:28.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.795+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7ff998089a20 con 0x7ff9a00c2310 2024-02-26T10:41:28.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.795+0000 7ff9a4c00640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff97c06b500 0x7ff97c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:28.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.797+0000 7ff9a4c00640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff97c06b500 0x7ff97c06d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00ac350 tx=0x7ff990009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:28.800+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff998057eb0 con 0x7ff9a00c2310 2024-02-26T10:41:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.010+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9a00c20f0 con 0x7ff9a00c2310 2024-02-26T10:41:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.014+0000 7ff99cc00640 1 -- 172.21.15.67:0/3806735306 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+20472 (secure 0 0 0) 0x7ff99805bf50 con 0x7ff9a00c2310 2024-02-26T10:41:29.013 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:29.013 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":61,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:22.912213+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:40:43.472687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-02-26T10:40:44.096497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff97c06b500 msgr2=0x7ff97c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff97c06b500 0x7ff97c06d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00ac350 tx=0x7ff990009040 comp rx=0 tx=0).stop 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 msgr2=0x7ff9a01eab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a01eab50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c780 tx=0x7ff99800cc40 comp rx=0 tx=0).stop 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 shutdown_connections 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a00c2310 0x7ff9a01eab50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff97c06b500 0x7ff97c06d9c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a006b000 0x7ff9a01ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 --2- 172.21.15.67:0/3806735306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9a006ac20 0x7ff9a0150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 >> 172.21.15.67:0/3806735306 conn(0x7ff9a00c63f0 msgr2=0x7ff9a00bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 shutdown_connections 2024-02-26T10:41:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.017+0000 7ff9a6000640 1 -- 172.21.15.67:0/3806735306 wait complete. 2024-02-26T10:41:29.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T10:41:29.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.262+0000 7f83c7200640 1 Processor -- start 2024-02-26T10:41:29.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.262+0000 7f83c7200640 1 -- start start 2024-02-26T10:41:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83c806ac20 0x7f83c80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806db00 0x7f83c80abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c806b460 con 0x7f83c806b000 2024-02-26T10:41:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c806dfb0 con 0x7f83c806ac20 2024-02-26T10:41:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80a9a80 con 0x7f83c806db00 2024-02-26T10:41:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83c806ac20 0x7f83c80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.264+0000 7f83c5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83c806ac20 0x7f83c80a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34516/0 (socket says 172.21.15.67:34516) 2024-02-26T10:41:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.265+0000 7f83c5e00640 1 -- 172.21.15.67:0/3134423412 learned_addr learned my addr 172.21.15.67:0/3134423412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.265+0000 7f83c6800640 1 --2- 172.21.15.67:0/3134423412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806db00 0x7f83c80abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.265+0000 7f83c5e00640 1 -- 172.21.15.67:0/3134423412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806db00 msgr2=0x7f83c80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:29.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.265+0000 7f83c5e00640 1 --2- 172.21.15.67:0/3134423412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806db00 0x7f83c80abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.265+0000 7f83c5e00640 1 -- 172.21.15.67:0/3134423412 >> v1:172.21.15.82:6789/0 conn(0x7f83c806b000 legacy=0x7f83c8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:29.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.265+0000 7f83c5e00640 1 -- 172.21.15.67:0/3134423412 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80a9d60 con 0x7f83c806ac20 2024-02-26T10:41:29.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.266+0000 7f83c5e00640 1 --2- 172.21.15.67:0/3134423412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806ac20 0x7f83c80a94a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009870 tx=0x7f83b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596a9eeb68271d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.267+0000 7f83c5400640 1 -- 172.21.15.67:0/3134423412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8034070 con 0x7f83c806ac20 2024-02-26T10:41:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.267+0000 7f83c5400640 1 -- 172.21.15.67:0/3134423412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8030840 con 0x7f83c806ac20 2024-02-26T10:41:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.267+0000 7f83c5400640 1 -- 172.21.15.67:0/3134423412 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8030c10 con 0x7f83c806ac20 2024-02-26T10:41:29.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.267+0000 7f83c7200640 1 -- 172.21.15.67:0/3134423412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806ac20 msgr2=0x7f83c80a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.267+0000 7f83c7200640 1 --2- 172.21.15.67:0/3134423412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806ac20 0x7f83c80a94a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009870 tx=0x7f83b802b980 comp rx=0 tx=0).stop 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.268+0000 7f83c7200640 1 -- 172.21.15.67:0/3134423412 shutdown_connections 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.268+0000 7f83c7200640 1 --2- 172.21.15.67:0/3134423412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806db00 0x7f83c80abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.268+0000 7f83c7200640 1 --2- 172.21.15.67:0/3134423412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806ac20 0x7f83c80a94a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.268+0000 7f83c7200640 1 -- 172.21.15.67:0/3134423412 >> 172.21.15.67:0/3134423412 conn(0x7f83c80c63f0 msgr2=0x7f83c80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.268+0000 7f83c7200640 1 -- 172.21.15.67:0/3134423412 shutdown_connections 2024-02-26T10:41:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.268+0000 7f83c7200640 1 -- 172.21.15.67:0/3134423412 wait complete. 2024-02-26T10:41:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.269+0000 7f83c7200640 1 Processor -- start 2024-02-26T10:41:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.270+0000 7f83c7200640 1 -- start start 2024-02-26T10:41:29.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.270+0000 7f83c7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806ac20 0x7f83c814a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.270+0000 7f83c7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806b000 0x7f83c8147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 0x7f83c80c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80ac490 con 0x7f83c806db00 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80aba30 con 0x7f83c806b000 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c8150410 con 0x7f83c806ac20 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 0x7f83c80c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 0x7f83c80c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34518/0 (socket says 172.21.15.67:34518) 2024-02-26T10:41:29.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 -- 172.21.15.67:0/3206115329 learned_addr learned my addr 172.21.15.67:0/3206115329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c5e00640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806ac20 0x7f83c814a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83bfe00640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806b000 0x7f83c8147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 -- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806ac20 msgr2=0x7f83c814a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806ac20 0x7f83c814a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 -- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806b000 msgr2=0x7f83c8147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806b000 0x7f83c8147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.271+0000 7f83c6800640 1 -- 172.21.15.67:0/3206115329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c81ee5d0 con 0x7f83c806db00 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.272+0000 7f83c6800640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 0x7f83c80c5700 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f83c000cc60 tx=0x7f83c000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.272+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003990 con 0x7f83c806db00 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.272+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c0003b30 con 0x7f83c806db00 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.272+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0004020 con 0x7f83c806db00 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.272+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c81ee8b0 con 0x7f83c806db00 2024-02-26T10:41:29.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.272+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c81eee70 con 0x7f83c806db00 2024-02-26T10:41:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.273+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c814f570 con 0x7f83c806db00 2024-02-26T10:41:29.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.274+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83c00041c0 con 0x7f83c806db00 2024-02-26T10:41:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.275+0000 7f83be000640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f83a406b4d0 0x7f83a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.275+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6623+0+0 (secure 0 0 0) 0x7f83c008aa90 con 0x7f83c806db00 2024-02-26T10:41:29.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.276+0000 7f83c5e00640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f83a406b4d0 0x7f83a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.277+0000 7f83c5e00640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f83a406b4d0 0x7f83a406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f83b8036000 tx=0x7f83b803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:29.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.280+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83c814f570 con 0x7f83c806db00 2024-02-26T10:41:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.519+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83c81ef070 con 0x7f83c806db00 2024-02-26T10:41:29.519 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:41:29.521+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:41:29.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.524+0000 7f83be000640 1 -- 172.21.15.67:0/3206115329 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7f83c81ef070 con 0x7f83c806db00 2024-02-26T10:41:29.523 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:41:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f83a406b4d0 msgr2=0x7f83a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f83a406b4d0 0x7f83a406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f83b8036000 tx=0x7f83b803b040 comp rx=0 tx=0).stop 2024-02-26T10:41:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 msgr2=0x7f83c80c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 0x7f83c80c5700 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f83c000cc60 tx=0x7f83c000a5c0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 shutdown_connections 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c806db00 0x7f83c80c5700 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f83a406b4d0 0x7f83a406d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83c806b000 0x7f83c8147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 --2- 172.21.15.67:0/3206115329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83c806ac20 0x7f83c814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.528+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 >> 172.21.15.67:0/3206115329 conn(0x7f83c80c63f0 msgr2=0x7f83c80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.529+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 shutdown_connections 2024-02-26T10:41:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.529+0000 7f83c7200640 1 -- 172.21.15.67:0/3206115329 wait complete. 2024-02-26T10:41:29.536 INFO:tasks.ceph.mds.c.smithi067.stderr: -3233> 2024-02-26T10:40:49.923+0000 7f64f2e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:41:29.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T10:41:29.603 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:41:29.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.742+0000 7f1ee6e00640 1 Processor -- start 2024-02-26T10:41:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.742+0000 7f1ee6e00640 1 -- start start 2024-02-26T10:41:29.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.743+0000 7f1ee6e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00ab520 0x7f1ee0158470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.743+0000 7f1ee6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ee00bf820 0x7f1ee0155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.743+0000 7f1ee6e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee0154f90 con 0x7f1ee006e810 2024-02-26T10:41:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.743+0000 7f1ee6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00c18a0 con 0x7f1ee00bf820 2024-02-26T10:41:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.743+0000 7f1ee6e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00c1b80 con 0x7f1ee00ab520 2024-02-26T10:41:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ee00bf820 0x7f1ee0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ee00bf820 0x7f1ee0155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36516/0 (socket says 172.21.15.67:36516) 2024-02-26T10:41:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 -- 172.21.15.67:0/1479560653 learned_addr learned my addr 172.21.15.67:0/1479560653 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5a00640 1 --2- 172.21.15.67:0/1479560653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00ab520 0x7f1ee0158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 -- 172.21.15.67:0/1479560653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00ab520 msgr2=0x7f1ee0158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 --2- 172.21.15.67:0/1479560653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00ab520 0x7f1ee0158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 -- 172.21.15.67:0/1479560653 >> v1:172.21.15.67:6789/0 conn(0x7f1ee006e810 legacy=0x7f1ee00c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.744+0000 7f1ee5000640 1 -- 172.21.15.67:0/1479560653 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0155230 con 0x7f1ee00bf820 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.746+0000 7f1ee5000640 1 --2- 172.21.15.67:0/1479560653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee00bf820 0x7f1ee0155750 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0155160 tx=0x7f1ed402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e705ed5ad40865 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.746+0000 7f1edfe00640 1 -- 172.21.15.67:0/1479560653 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4035020 con 0x7f1ee00bf820 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.746+0000 7f1edfe00640 1 -- 172.21.15.67:0/1479560653 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4031640 con 0x7f1ee00bf820 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.746+0000 7f1edfe00640 1 -- 172.21.15.67:0/1479560653 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed402d040 con 0x7f1ee00bf820 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.746+0000 7f1ee6e00640 1 -- 172.21.15.67:0/1479560653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee00bf820 msgr2=0x7f1ee0155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.746+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/1479560653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee00bf820 0x7f1ee0155750 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0155160 tx=0x7f1ed402b8f0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.747+0000 7f1ee6e00640 1 -- 172.21.15.67:0/1479560653 shutdown_connections 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.747+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/1479560653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee00bf820 0x7f1ee0155750 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.747+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/1479560653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00ab520 0x7f1ee0158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.747+0000 7f1ee6e00640 1 -- 172.21.15.67:0/1479560653 >> 172.21.15.67:0/1479560653 conn(0x7f1ee006cd20 msgr2=0x7f1ee006d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:29.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.747+0000 7f1ee6e00640 1 -- 172.21.15.67:0/1479560653 shutdown_connections 2024-02-26T10:41:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.747+0000 7f1ee6e00640 1 -- 172.21.15.67:0/1479560653 wait complete. 2024-02-26T10:41:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.748+0000 7f1ee6e00640 1 Processor -- start 2024-02-26T10:41:29.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 -- start start 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 0x7f1ee014aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00ab520 0x7f1ee01eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00bf820 0x7f1ee01e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00c09d0 con 0x7f1ee006e810 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00c0850 con 0x7f1ee00ab520 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.749+0000 7f1ee6e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00c1450 con 0x7f1ee00bf820 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 0x7f1ee014aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 0x7f1ee014aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36532/0 (socket says 172.21.15.67:36532) 2024-02-26T10:41:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 -- 172.21.15.67:0/202485308 learned_addr learned my addr 172.21.15.67:0/202485308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 -- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00bf820 msgr2=0x7f1ee01e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00bf820 0x7f1ee01e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 -- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00ab520 msgr2=0x7f1ee01eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00ab520 0x7f1ee01eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 -- 172.21.15.67:0/202485308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0070f80 con 0x7f1ee006e810 2024-02-26T10:41:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.750+0000 7f1ee5a00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 0x7f1ee014aea0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000d800 tx=0x7f1ed000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.751+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000b6c0 con 0x7f1ee006e810 2024-02-26T10:41:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.751+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed0019070 con 0x7f1ee006e810 2024-02-26T10:41:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.751+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000ba80 con 0x7f1ee006e810 2024-02-26T10:41:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.751+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee014abc0 con 0x7f1ee006e810 2024-02-26T10:41:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.751+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee01eeb70 con 0x7f1ee006e810 2024-02-26T10:41:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.752+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee00c4c70 con 0x7f1ee006e810 2024-02-26T10:41:29.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.753+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ed000bc20 con 0x7f1ee006e810 2024-02-26T10:41:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.754+0000 7f1edd600640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ec406b4d0 0x7f1ec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.754+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1ed008aab0 con 0x7f1ee006e810 2024-02-26T10:41:29.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.755+0000 7f1ee5000640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ec406b4d0 0x7f1ec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.756+0000 7f1ee5000640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ec406b4d0 0x7f1ec406d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4036000 tx=0x7f1ed4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:29.759+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ed0059530 con 0x7f1ee006e810 2024-02-26T10:41:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.003+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ee0074a30 con 0x7f1ee006e810 2024-02-26T10:41:30.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.527+0000 7f1edd600640 1 -- 172.21.15.67:0/202485308 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ed0058e30 con 0x7f1ee006e810 2024-02-26T10:41:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ec406b4d0 msgr2=0x7f1ec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ec406b4d0 0x7f1ec406d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4036000 tx=0x7f1ed4030000 comp rx=0 tx=0).stop 2024-02-26T10:41:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 msgr2=0x7f1ee014aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 0x7f1ee014aea0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000d800 tx=0x7f1ed000dcc0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 shutdown_connections 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ee00bf820 0x7f1ee01e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1ec406b4d0 0x7f1ec406d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00ab520 0x7f1ee01eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 --2- 172.21.15.67:0/202485308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ee006e810 0x7f1ee014aea0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 >> 172.21.15.67:0/202485308 conn(0x7f1ee006cd20 msgr2=0x7f1ee01de4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 shutdown_connections 2024-02-26T10:41:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.531+0000 7f1ee6e00640 1 -- 172.21.15.67:0/202485308 wait complete. 2024-02-26T10:41:30.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T10:41:30.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:41:30.604 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:41:30.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.773+0000 7f4931a00640 1 Processor -- start 2024-02-26T10:41:30.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.774+0000 7f4931a00640 1 -- start start 2024-02-26T10:41:30.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.775+0000 7f4931a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 0x7f492c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.775+0000 7f4931a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c152760 con 0x7f492c0c9b30 2024-02-26T10:41:30.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.775+0000 7f4931a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0cb4e0 con 0x7f492c0c54d0 2024-02-26T10:41:30.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.775+0000 7f4931a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c06b900 con 0x7f492c152380 2024-02-26T10:41:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.776+0000 7f4931000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f492c0c9b30 0x7f492c15f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56278/0 (socket says 172.21.15.67:56278) 2024-02-26T10:41:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.776+0000 7f4931000640 1 -- 172.21.15.67:0/3778762334 learned_addr learned my addr 172.21.15.67:0/3778762334 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.776+0000 7f492be00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 0x7f492c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.776+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619946654 0 0) 0x7f4924001490 con 0x7f492c0c9b30 2024-02-26T10:41:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.776+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4910003520 con 0x7f492c0c9b30 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609224179 0 0) 0x7f4910003520 con 0x7f492c0c9b30 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 msgr2=0x7f492c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 0x7f492c0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 >> v1:172.21.15.82:6789/0 conn(0x7f492c0c54d0 legacy=0x7f492c0c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c159010 con 0x7f492c0c9b30 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f4924001bc0 con 0x7f492c0c9b30 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 >> v1:172.21.15.67:6789/0 conn(0x7f492c0c9b30 legacy=0x7f492c15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49100039e0 0x7f4910003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 0x7f4910004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4910004d40 0x7f49100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c06b900 con 0x7f49100039e0 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491000bb80 con 0x7f4910004d40 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491000bd80 con 0x7f492c152380 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.777+0000 7f492be00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49100039e0 0x7f4910003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f492b400640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 0x7f4910004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f4931000640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4910004d40 0x7f49100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f492be00640 1 -- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 msgr2=0x7f4910004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f492be00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c152380 0x7f4910004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f492be00640 1 -- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4910004d40 msgr2=0x7f49100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f492be00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4910004d40 0x7f49100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.778+0000 7f492be00640 1 -- 172.21.15.67:0/3778762334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910003520 con 0x7f49100039e0 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.779+0000 7f492be00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49100039e0 0x7f4910003e00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f491c009ac0 tx=0x7f4910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d69ec7254c0c46 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c039070 con 0x7f49100039e0 2024-02-26T10:41:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c02c6b0 con 0x7f49100039e0 2024-02-26T10:41:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f492aa00640 1 -- 172.21.15.67:0/3778762334 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c009690 con 0x7f49100039e0 2024-02-26T10:41:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931a00640 1 -- 172.21.15.67:0/3778762334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49100039e0 msgr2=0x7f4910003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931a00640 1 --2- 172.21.15.67:0/3778762334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49100039e0 0x7f4910003e00 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f491c009ac0 tx=0x7f4910005830 comp rx=0 tx=0).stop 2024-02-26T10:41:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931000640 1 -- 172.21.15.67:0/3778762334 reap_dead start 2024-02-26T10:41:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931a00640 1 -- 172.21.15.67:0/3778762334 shutdown_connections 2024-02-26T10:41:30.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931a00640 1 -- 172.21.15.67:0/3778762334 >> 172.21.15.67:0/3778762334 conn(0x7f492c0ca610 msgr2=0x7f492c0caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:30.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931a00640 1 -- 172.21.15.67:0/3778762334 shutdown_connections 2024-02-26T10:41:30.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.780+0000 7f4931a00640 1 -- 172.21.15.67:0/3778762334 wait complete. 2024-02-26T10:41:30.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.782+0000 7f4931a00640 1 Processor -- start 2024-02-26T10:41:30.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.782+0000 7f4931a00640 1 -- start start 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f4931a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 0x7f492c1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f4931a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c1494a0 0x7f492c1488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f4931a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f492c1eea40 0x7f492c1ecd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f4931a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c06dca0 con 0x7f492c1577f0 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f4931a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c8bf0 con 0x7f492c1eea40 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f4931a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c158c70 con 0x7f492c1494a0 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f492be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 0x7f492c1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f492be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 0x7f492c1483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36550/0 (socket says 172.21.15.67:36550) 2024-02-26T10:41:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.783+0000 7f492be00640 1 -- 172.21.15.67:0/3369116702 learned_addr learned my addr 172.21.15.67:0/3369116702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492b400640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c1494a0 0x7f492c1488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492be00640 1 -- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c1494a0 msgr2=0x7f492c1488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492be00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c1494a0 0x7f492c1488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f4931000640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f492c1eea40 0x7f492c1ecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492be00640 1 -- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f492c1eea40 msgr2=0x7f492c1ecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492be00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f492c1eea40 0x7f492c1ecd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492be00640 1 -- 172.21.15.67:0/3369116702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c1eeed0 con 0x7f492c1577f0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f492be00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 0x7f492c1483b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f491c037040 tx=0x7f491c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.784+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c039070 con 0x7f492c1577f0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.785+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c043070 con 0x7f492c1577f0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.785+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0044e0 con 0x7f492c1577f0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.785+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c1ebfd0 con 0x7f492c1577f0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.785+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c1ec590 con 0x7f492c1577f0 2024-02-26T10:41:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.786+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c06d290 con 0x7f492c1577f0 2024-02-26T10:41:30.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.787+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f491c003a10 con 0x7f492c1577f0 2024-02-26T10:41:30.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.788+0000 7f4923e00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f490c06b500 0x7f490c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:30.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.788+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f491c0ab880 con 0x7f492c1577f0 2024-02-26T10:41:30.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.788+0000 7f492b400640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f490c06b500 0x7f490c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:30.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.790+0000 7f492b400640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f490c06b500 0x7f490c06d9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4918007fd0 tx=0x7f4918009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.792+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f492c06d290 con 0x7f492c1577f0 2024-02-26T10:41:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:30.993+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f492c074a30 con 0x7f492c1577f0 2024-02-26T10:41:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.991+0000 7f4923e00640 1 -- 172.21.15.67:0/3369116702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7f491c04c020 con 0x7f492c1577f0 2024-02-26T10:41:31.991 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-02-26T10:41:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.997+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f490c06b500 msgr2=0x7f490c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.997+0000 7f4931a00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f490c06b500 0x7f490c06d9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4918007fd0 tx=0x7f4918009040 comp rx=0 tx=0).stop 2024-02-26T10:41:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.997+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 msgr2=0x7f492c1483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:31.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.997+0000 7f4931a00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 0x7f492c1483b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f491c037040 tx=0x7f491c004730 comp rx=0 tx=0).stop 2024-02-26T10:41:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.998+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 shutdown_connections 2024-02-26T10:41:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.998+0000 7f4931a00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f490c06b500 0x7f490c06d9c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.998+0000 7f4931a00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f492c1eea40 0x7f492c1ecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:31.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.998+0000 7f4931a00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f492c1494a0 0x7f492c1488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.998+0000 7f4931a00640 1 --2- 172.21.15.67:0/3369116702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f492c1577f0 0x7f492c1483b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:31.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:31.998+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 >> 172.21.15.67:0/3369116702 conn(0x7f492c0ca610 msgr2=0x7f492c0aad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.001+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 shutdown_connections 2024-02-26T10:41:32.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.001+0000 7f4931a00640 1 -- 172.21.15.67:0/3369116702 wait complete. 2024-02-26T10:41:32.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T10:41:32.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.234+0000 7f4655800640 1 Processor -- start 2024-02-26T10:41:32.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.234+0000 7f4655800640 1 -- start start 2024-02-26T10:41:32.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f4655800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46501503a0 0x7f46500ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:32.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f4655800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46500a9430 0x7f46501535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:32.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f4655800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4650153ae0 con 0x7f465006e2d0 2024-02-26T10:41:32.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f4655800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4650150780 con 0x7f46501503a0 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f4655800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46500a98c0 con 0x7f46500a9430 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f464fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46501503a0 0x7f46500ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f464fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46501503a0 0x7f46500ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36554/0 (socket says 172.21.15.67:36554) 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.236+0000 7f464fe00640 1 -- 172.21.15.67:0/1368338242 learned_addr learned my addr 172.21.15.67:0/1368338242 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.237+0000 7f464fe00640 1 -- 172.21.15.67:0/1368338242 >> v2:172.21.15.82:3300/0 conn(0x7f46500a9430 msgr2=0x7f46501535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.237+0000 7f464f400640 1 --2- 172.21.15.67:0/1368338242 >> v2:172.21.15.82:3300/0 conn(0x7f46500a9430 0x7f46501535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.237+0000 7f464fe00640 1 --2- 172.21.15.67:0/1368338242 >> v2:172.21.15.82:3300/0 conn(0x7f46500a9430 0x7f46501535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.237+0000 7f464fe00640 1 -- 172.21.15.67:0/1368338242 >> v1:172.21.15.82:6789/0 conn(0x7f465006e2d0 legacy=0x7f46500c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.237+0000 7f464fe00640 1 -- 172.21.15.67:0/1368338242 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500a9ba0 con 0x7f46501503a0 2024-02-26T10:41:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.237+0000 7f464f400640 1 --2- 172.21.15.67:0/1368338242 >> v2:172.21.15.82:3300/0 conn(0x7f46500a9430 0x7f46501535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:32.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.238+0000 7f464fe00640 1 --2- 172.21.15.67:0/1368338242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46501503a0 0x7f46500ab480 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f46500c73e0 tx=0x7f464002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f3d16412a379b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:32.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.238+0000 7f464ea00640 1 -- 172.21.15.67:0/1368338242 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464002ca80 con 0x7f46501503a0 2024-02-26T10:41:32.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.238+0000 7f464ea00640 1 -- 172.21.15.67:0/1368338242 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464002cc20 con 0x7f46501503a0 2024-02-26T10:41:32.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.238+0000 7f464ea00640 1 -- 172.21.15.67:0/1368338242 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400343f0 con 0x7f46501503a0 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 -- 172.21.15.67:0/1368338242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46501503a0 msgr2=0x7f46500ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 --2- 172.21.15.67:0/1368338242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46501503a0 0x7f46500ab480 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f46500c73e0 tx=0x7f464002bee0 comp rx=0 tx=0).stop 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 -- 172.21.15.67:0/1368338242 shutdown_connections 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 --2- 172.21.15.67:0/1368338242 >> v2:172.21.15.82:3300/0 conn(0x7f46500a9430 0x7f46501535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 --2- 172.21.15.67:0/1368338242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46501503a0 0x7f46500ab480 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 -- 172.21.15.67:0/1368338242 >> 172.21.15.67:0/1368338242 conn(0x7f46500c6380 msgr2=0x7f46500c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 -- 172.21.15.67:0/1368338242 shutdown_connections 2024-02-26T10:41:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.239+0000 7f4655800640 1 -- 172.21.15.67:0/1368338242 wait complete. 2024-02-26T10:41:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.241+0000 7f4655800640 1 Processor -- start 2024-02-26T10:41:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.241+0000 7f4655800640 1 -- start start 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f4655800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 0x7f4650149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f4655800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46501503a0 0x7f465013b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f4655800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501483a0 0x7f465013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f4655800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465006ddb0 con 0x7f465006e2d0 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f4655800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465006dad0 con 0x7f46501483a0 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f4655800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46500c0f80 con 0x7f46501503a0 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f464fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 0x7f4650149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f464fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 0x7f4650149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36568/0 (socket says 172.21.15.67:36568) 2024-02-26T10:41:32.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.242+0000 7f464fe00640 1 -- 172.21.15.67:0/1946772970 learned_addr learned my addr 172.21.15.67:0/1946772970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464f400640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46501503a0 0x7f465013b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464fe00640 1 -- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46501503a0 msgr2=0x7f465013b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f4654e00640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501483a0 0x7f465013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464fe00640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46501503a0 0x7f465013b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464fe00640 1 -- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501483a0 msgr2=0x7f465013be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464fe00640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501483a0 0x7f465013be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464fe00640 1 -- 172.21.15.67:0/1946772970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46501523d0 con 0x7f465006e2d0 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f464fe00640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 0x7f4650149f40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4640000f80 tx=0x7f4640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640043020 con 0x7f465006e2d0 2024-02-26T10:41:32.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464003e450 con 0x7f465006e2d0 2024-02-26T10:41:32.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46500bfd90 con 0x7f465006e2d0 2024-02-26T10:41:32.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.243+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464003e840 con 0x7f465006e2d0 2024-02-26T10:41:32.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.244+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46500c0350 con 0x7f465006e2d0 2024-02-26T10:41:32.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.245+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4650074a30 con 0x7f465006e2d0 2024-02-26T10:41:32.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.246+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4640046020 con 0x7f465006e2d0 2024-02-26T10:41:32.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.247+0000 7f4647e00640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462c06b4d0 0x7f462c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:32.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.247+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6271+0+0 (secure 0 0 0) 0x7f46400acd10 con 0x7f465006e2d0 2024-02-26T10:41:32.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.248+0000 7f464f400640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462c06b4d0 0x7f462c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.249+0000 7f464f400640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462c06b4d0 0x7f462c06d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f465014dac0 tx=0x7f4638008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:32.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.252+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f464004d030 con 0x7f465006e2d0 2024-02-26T10:41:32.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:32.458+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f46500c1650 con 0x7f465006e2d0 2024-02-26T10:41:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.024+0000 7f4647e00640 1 -- 172.21.15.67:0/1946772970 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7f464007b9f0 con 0x7f465006e2d0 2024-02-26T10:41:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-02-26T10:41:33.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.030+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462c06b4d0 msgr2=0x7f462c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.030+0000 7f4655800640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462c06b4d0 0x7f462c06d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f465014dac0 tx=0x7f4638008040 comp rx=0 tx=0).stop 2024-02-26T10:41:33.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.030+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 msgr2=0x7f4650149f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.030+0000 7f4655800640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 0x7f4650149f40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4640000f80 tx=0x7f4640004a40 comp rx=0 tx=0).stop 2024-02-26T10:41:33.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.031+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 shutdown_connections 2024-02-26T10:41:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.031+0000 7f4655800640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f462c06b4d0 0x7f462c06d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.031+0000 7f4655800640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501483a0 0x7f465013be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.031+0000 7f4655800640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46501503a0 0x7f465013b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.031+0000 7f4655800640 1 --2- 172.21.15.67:0/1946772970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465006e2d0 0x7f4650149f40 secure :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4640000f80 tx=0x7f4640004a40 comp rx=0 tx=0).stop 2024-02-26T10:41:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.031+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 >> 172.21.15.67:0/1946772970 conn(0x7f46500c6380 msgr2=0x7f46501515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.033+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 shutdown_connections 2024-02-26T10:41:33.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.033+0000 7f4655800640 1 -- 172.21.15.67:0/1946772970 wait complete. 2024-02-26T10:41:33.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.256+0000 7ff683e00640 1 Processor -- start 2024-02-26T10:41:33.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.257+0000 7ff683e00640 1 -- start start 2024-02-26T10:41:33.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.258+0000 7ff683e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff67c070150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.258+0000 7ff683e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0ae080 con 0x7ff67c0c69d0 2024-02-26T10:41:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.258+0000 7ff683e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c153840 con 0x7ff67c0adca0 2024-02-26T10:41:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.258+0000 7ff683e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c3540 con 0x7ff67c0c64c0 2024-02-26T10:41:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.259+0000 7ff681e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff67c0c69d0 0x7ff67c0c84a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56282/0 (socket says 172.21.15.67:56282) 2024-02-26T10:41:33.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.259+0000 7ff681e00640 1 -- 172.21.15.67:0/1889630711 learned_addr learned my addr 172.21.15.67:0/1889630711 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.259+0000 7ff680a00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff67c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.259+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412286016 0 0) 0x7ff67c0ae080 con 0x7ff67c0c69d0 2024-02-26T10:41:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.259+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff664003520 con 0x7ff67c0c69d0 2024-02-26T10:41:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3989006555 0 0) 0x7ff664003520 con 0x7ff67c0c69d0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 msgr2=0x7ff67c070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff67c070150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 >> v1:172.21.15.82:6789/0 conn(0x7ff67c0adca0 legacy=0x7ff67c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c153b20 con 0x7ff67c0c69d0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7ff6740019f0 con 0x7ff67c0c69d0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 >> v1:172.21.15.67:6789/0 conn(0x7ff67c0c69d0 legacy=0x7ff67c0c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6640039e0 0x7ff664003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff664004800 0x7ff664004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff664005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66400bb50 con 0x7ff664004800 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66400bcd0 con 0x7ff6640039e0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66400bed0 con 0x7ff67c0c64c0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.260+0000 7ff680a00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff664004800 0x7ff664004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff681e00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff664005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff681400640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6640039e0 0x7ff664003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff680a00640 1 -- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 msgr2=0x7ff664005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff680a00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff664005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff680a00640 1 -- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6640039e0 msgr2=0x7ff664003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff680a00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6640039e0 0x7ff664003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff680a00640 1 -- 172.21.15.67:0/1889630711 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664003520 con 0x7ff664004800 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.261+0000 7ff681e00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff67c0c64c0 0x7ff664005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.262+0000 7ff680a00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff664004800 0x7ff664004c80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009a90 tx=0x7ff664005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff264a5ed923f451 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.262+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c039070 con 0x7ff664004800 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.262+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c02c680 con 0x7ff664004800 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.262+0000 7ff67be00640 1 -- 172.21.15.67:0/1889630711 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c009690 con 0x7ff664004800 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff683e00640 1 -- 172.21.15.67:0/1889630711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff664004800 msgr2=0x7ff664004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff683e00640 1 --2- 172.21.15.67:0/1889630711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff664004800 0x7ff664004c80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009a90 tx=0x7ff664005830 comp rx=0 tx=0).stop 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff681e00640 1 -- 172.21.15.67:0/1889630711 reap_dead start 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff683e00640 1 -- 172.21.15.67:0/1889630711 shutdown_connections 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff683e00640 1 -- 172.21.15.67:0/1889630711 >> 172.21.15.67:0/1889630711 conn(0x7ff67c06b3b0 msgr2=0x7ff67c0aa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:33.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff683e00640 1 -- 172.21.15.67:0/1889630711 shutdown_connections 2024-02-26T10:41:33.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.263+0000 7ff683e00640 1 -- 172.21.15.67:0/1889630711 wait complete. 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.265+0000 7ff683e00640 1 Processor -- start 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.265+0000 7ff683e00640 1 -- start start 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff683e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff664004800 0x7ff67c151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff683e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff67c0c2890 0x7ff67c149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff683e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 0x7ff67c154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff683e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c8bb0 con 0x7ff67c14a0e0 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff683e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c2f80 con 0x7ff67c0c2890 2024-02-26T10:41:33.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff683e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c8d30 con 0x7ff664004800 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 0x7ff67c154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 0x7ff67c154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36580/0 (socket says 172.21.15.67:36580) 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 -- 172.21.15.67:0/4135081703 learned_addr learned my addr 172.21.15.67:0/4135081703 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681400640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff664004800 0x7ff67c151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff680a00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff67c0c2890 0x7ff67c149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 -- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff664004800 msgr2=0x7ff67c151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff664004800 0x7ff67c151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 -- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff67c0c2890 msgr2=0x7ff67c149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff67c0c2890 0x7ff67c149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.266+0000 7ff681e00640 1 -- 172.21.15.67:0/4135081703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c14e8b0 con 0x7ff67c14a0e0 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff681400640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff664004800 0x7ff67c151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff680a00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff67c0c2890 0x7ff67c149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff681e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 0x7ff67c154c40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff67c1508f0 tx=0x7ff67400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674004d60 con 0x7ff67c14a0e0 2024-02-26T10:41:33.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400bb20 con 0x7ff67c14a0e0 2024-02-26T10:41:33.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674007110 con 0x7ff67c14a0e0 2024-02-26T10:41:33.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c14eb90 con 0x7ff67c14a0e0 2024-02-26T10:41:33.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.267+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c14f150 con 0x7ff67c14a0e0 2024-02-26T10:41:33.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.268+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c074a30 con 0x7ff67c14a0e0 2024-02-26T10:41:33.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.269+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff67401b020 con 0x7ff67c14a0e0 2024-02-26T10:41:33.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.271+0000 7ff679600640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff66006b500 0x7ff66006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.272+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7ff67408a370 con 0x7ff67c14a0e0 2024-02-26T10:41:33.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.272+0000 7ff681400640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff66006b500 0x7ff66006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.273+0000 7ff681400640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff66006b500 0x7ff66006d9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff670000f60 tx=0x7ff670009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.276+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6740591e0 con 0x7ff67c14a0e0 2024-02-26T10:41:33.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.474+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff67c152c90 con 0x7ff67c14a0e0 2024-02-26T10:41:33.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.478+0000 7ff679600640 1 -- 172.21.15.67:0/4135081703 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16827 (secure 0 0 0) 0x7ff674058ae0 con 0x7ff67c14a0e0 2024-02-26T10:41:33.479 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:33.480 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":64,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:33.014527+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6834/1127454674":"2024-02-27T10:41:29.521347+0000","172.21.15.67:6832/1127454674":"2024-02-27T10:41:29.521347+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff66006b500 msgr2=0x7ff66006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff66006b500 0x7ff66006d9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff670000f60 tx=0x7ff670009040 comp rx=0 tx=0).stop 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 msgr2=0x7ff67c154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 0x7ff67c154c40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff67c1508f0 tx=0x7ff67400b4a0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 shutdown_connections 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14a0e0 0x7ff67c154c40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff66006b500 0x7ff66006d9c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff67c0c2890 0x7ff67c149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 --2- 172.21.15.67:0/4135081703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff664004800 0x7ff67c151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.480+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 >> 172.21.15.67:0/4135081703 conn(0x7ff67c06b3b0 msgr2=0x7ff67c0a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.481+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 shutdown_connections 2024-02-26T10:41:33.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.481+0000 7ff683e00640 1 -- 172.21.15.67:0/4135081703 wait complete. 2024-02-26T10:41:33.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:40:58.810192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:41:00.247724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:33.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:33.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.731+0000 7efce4c00640 1 Processor -- start 2024-02-26T10:41:33.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.731+0000 7efce4c00640 1 -- start start 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efce4c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efce0071710 0x7efce00c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efce4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efce006e3d0 0x7efce015c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efce4c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015cba0 0x7efce0156320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efce4c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00c2a80 con 0x7efce006e3d0 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efce4c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce015d0b0 con 0x7efce0071710 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efce4c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce015d390 con 0x7efce015cba0 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efcdea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efce006e3d0 0x7efce015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.733+0000 7efcdea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efce006e3d0 0x7efce015c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36586/0 (socket says 172.21.15.67:36586) 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdea00640 1 -- 172.21.15.67:0/956704030 learned_addr learned my addr 172.21.15.67:0/956704030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdfe00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015cba0 0x7efce0156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdf400640 1 --2- 172.21.15.67:0/956704030 >> v2:172.21.15.82:3300/0 conn(0x7efce0071710 0x7efce00c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdea00640 1 -- 172.21.15.67:0/956704030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015cba0 msgr2=0x7efce0156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdea00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015cba0 0x7efce0156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdea00640 1 -- 172.21.15.67:0/956704030 >> v2:172.21.15.82:3300/0 conn(0x7efce0071710 msgr2=0x7efce00c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdea00640 1 --2- 172.21.15.67:0/956704030 >> v2:172.21.15.82:3300/0 conn(0x7efce0071710 0x7efce00c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdea00640 1 -- 172.21.15.67:0/956704030 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce006dbd0 con 0x7efce006e3d0 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdf400640 1 --2- 172.21.15.67:0/956704030 >> v2:172.21.15.82:3300/0 conn(0x7efce0071710 0x7efce00c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.734+0000 7efcdfe00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015cba0 0x7efce0156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.735+0000 7efcdea00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce006e3d0 0x7efce015c660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efce00a91f0 tx=0x7efccc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f5371c2c15962a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efcde000640 1 -- 172.21.15.67:0/956704030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc0317d0 con 0x7efce006e3d0 2024-02-26T10:41:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efcde000640 1 -- 172.21.15.67:0/956704030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc037070 con 0x7efce006e3d0 2024-02-26T10:41:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efcde000640 1 -- 172.21.15.67:0/956704030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc031b90 con 0x7efce006e3d0 2024-02-26T10:41:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 -- 172.21.15.67:0/956704030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce006e3d0 msgr2=0x7efce015c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce006e3d0 0x7efce015c660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efce00a91f0 tx=0x7efccc02b870 comp rx=0 tx=0).stop 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 -- 172.21.15.67:0/956704030 shutdown_connections 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015cba0 0x7efce0156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 --2- 172.21.15.67:0/956704030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce006e3d0 0x7efce015c660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 --2- 172.21.15.67:0/956704030 >> v2:172.21.15.82:3300/0 conn(0x7efce0071710 0x7efce00c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.736+0000 7efce4c00640 1 -- 172.21.15.67:0/956704030 >> 172.21.15.67:0/956704030 conn(0x7efce00c6380 msgr2=0x7efce00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.737+0000 7efce4c00640 1 -- 172.21.15.67:0/956704030 shutdown_connections 2024-02-26T10:41:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.737+0000 7efce4c00640 1 -- 172.21.15.67:0/956704030 wait complete. 2024-02-26T10:41:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.738+0000 7efce4c00640 1 Processor -- start 2024-02-26T10:41:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.738+0000 7efce4c00640 1 -- start start 2024-02-26T10:41:33.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.739+0000 7efce4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce006e3d0 0x7efce01f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.739+0000 7efce4c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071710 0x7efce00aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.739+0000 7efce4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 0x7efce00aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.739+0000 7efce4c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0159410 con 0x7efce015cba0 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.739+0000 7efce4c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0159290 con 0x7efce006e3d0 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.739+0000 7efce4c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce006e860 con 0x7efce0071710 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 0x7efce00aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 0x7efce00aa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36590/0 (socket says 172.21.15.67:36590) 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 -- 172.21.15.67:0/3219844430 learned_addr learned my addr 172.21.15.67:0/3219844430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdea00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071710 0x7efce00aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 -- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071710 msgr2=0x7efce00aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdf400640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce006e3d0 0x7efce01f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071710 0x7efce00aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 -- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce006e3d0 msgr2=0x7efce01f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce006e3d0 0x7efce01f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.740+0000 7efcdfe00640 1 -- 172.21.15.67:0/3219844430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00ac330 con 0x7efce015cba0 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.741+0000 7efcdfe00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 0x7efce00aa400 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efcd800cd80 tx=0x7efcd800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.741+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800f680 con 0x7efce015cba0 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.741+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd800f840 con 0x7efce015cba0 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.741+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800fc30 con 0x7efce015cba0 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.741+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00a9640 con 0x7efce015cba0 2024-02-26T10:41:33.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.741+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce00a9c00 con 0x7efce015cba0 2024-02-26T10:41:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.742+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce00678e0 con 0x7efce015cba0 2024-02-26T10:41:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.743+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcd800e3e0 con 0x7efce015cba0 2024-02-26T10:41:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.744+0000 7efcd7400640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcc006b4d0 0x7efcc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.744+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7efcd808b080 con 0x7efce015cba0 2024-02-26T10:41:33.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.745+0000 7efcdf400640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcc006b4d0 0x7efcc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:33.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.746+0000 7efcdf400640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcc006b4d0 0x7efcc006d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efce00c1be0 tx=0x7efcd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:33.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.748+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcd8014030 con 0x7efce015cba0 2024-02-26T10:41:33.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.960+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efce0074a30 con 0x7efce015cba0 2024-02-26T10:41:33.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.961+0000 7efcd7400640 1 -- 172.21.15.67:0/3219844430 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcd8059e50 con 0x7efce015cba0 2024-02-26T10:41:33.960 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:33.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.963+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcc006b4d0 msgr2=0x7efcc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.963+0000 7efce4c00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcc006b4d0 0x7efcc006d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efce00c1be0 tx=0x7efcd0008040 comp rx=0 tx=0).stop 2024-02-26T10:41:33.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.963+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 msgr2=0x7efce00aa400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:33.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.963+0000 7efce4c00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 0x7efce00aa400 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efcd800cd80 tx=0x7efcd800a650 comp rx=0 tx=0).stop 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 shutdown_connections 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce015cba0 0x7efce00aa400 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071710 0x7efce00aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcc006b4d0 0x7efcc006d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 --2- 172.21.15.67:0/3219844430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce006e3d0 0x7efce01f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 >> 172.21.15.67:0/3219844430 conn(0x7efce00c6380 msgr2=0x7efce0157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 shutdown_connections 2024-02-26T10:41:33.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:33.964+0000 7efce4c00640 1 -- 172.21.15.67:0/3219844430 wait complete. 2024-02-26T10:41:33.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:41:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.207+0000 7fa209600640 1 Processor -- start 2024-02-26T10:41:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.207+0000 7fa209600640 1 -- start start 2024-02-26T10:41:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.208+0000 7fa209600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2040c0ef0 0x7fa204158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.208+0000 7fa209600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa2040c1400 0x7fa20406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.208+0000 7fa209600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa204072ce0 con 0x7fa2041591b0 2024-02-26T10:41:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.208+0000 7fa209600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa204073010 con 0x7fa2040c0ef0 2024-02-26T10:41:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.208+0000 7fa209600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20414f490 con 0x7fa2040c1400 2024-02-26T10:41:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa1fbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2040c0ef0 0x7fa204158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa202a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa2041591b0 0x7fa204070ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56298/0 (socket says 172.21.15.67:56298) 2024-02-26T10:41:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa202a00640 1 -- 172.21.15.67:0/3309369887 learned_addr learned my addr 172.21.15.67:0/3309369887 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa1fbe00640 1 -- 172.21.15.67:0/3309369887 >> v2:172.21.15.82:3300/0 conn(0x7fa2040c1400 msgr2=0x7fa20406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa203400640 1 --2- 172.21.15.67:0/3309369887 >> v2:172.21.15.82:3300/0 conn(0x7fa2040c1400 0x7fa20406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa1fbe00640 1 --2- 172.21.15.67:0/3309369887 >> v2:172.21.15.82:3300/0 conn(0x7fa2040c1400 0x7fa20406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa1fbe00640 1 -- 172.21.15.67:0/3309369887 >> v1:172.21.15.67:6789/0 conn(0x7fa2041591b0 legacy=0x7fa204070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa1fbe00640 1 -- 172.21.15.67:0/3309369887 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204155bf0 con 0x7fa2040c0ef0 2024-02-26T10:41:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.209+0000 7fa203400640 1 --2- 172.21.15.67:0/3309369887 >> v2:172.21.15.82:3300/0 conn(0x7fa2040c1400 0x7fa20406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.211+0000 7fa1fbe00640 1 --2- 172.21.15.67:0/3309369887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2040c0ef0 0x7fa204158580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa20406bc50 tx=0x7fa1ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a138e5ddb8157d17 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.211+0000 7fa202000640 1 -- 172.21.15.67:0/3309369887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0315a0 con 0x7fa2040c0ef0 2024-02-26T10:41:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.211+0000 7fa202000640 1 -- 172.21.15.67:0/3309369887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa2040c0ef0 2024-02-26T10:41:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.211+0000 7fa202000640 1 -- 172.21.15.67:0/3309369887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec031960 con 0x7fa2040c0ef0 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.211+0000 7fa209600640 1 -- 172.21.15.67:0/3309369887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2040c0ef0 msgr2=0x7fa204158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.211+0000 7fa209600640 1 --2- 172.21.15.67:0/3309369887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2040c0ef0 0x7fa204158580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa20406bc50 tx=0x7fa1ec02b840 comp rx=0 tx=0).stop 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.212+0000 7fa209600640 1 -- 172.21.15.67:0/3309369887 shutdown_connections 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.212+0000 7fa209600640 1 --2- 172.21.15.67:0/3309369887 >> v2:172.21.15.82:3300/0 conn(0x7fa2040c1400 0x7fa20406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.212+0000 7fa209600640 1 --2- 172.21.15.67:0/3309369887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2040c0ef0 0x7fa204158580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.212+0000 7fa209600640 1 -- 172.21.15.67:0/3309369887 >> 172.21.15.67:0/3309369887 conn(0x7fa2040c6da0 msgr2=0x7fa2040c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.212+0000 7fa209600640 1 -- 172.21.15.67:0/3309369887 shutdown_connections 2024-02-26T10:41:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.212+0000 7fa209600640 1 -- 172.21.15.67:0/3309369887 wait complete. 2024-02-26T10:41:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.214+0000 7fa209600640 1 Processor -- start 2024-02-26T10:41:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.214+0000 7fa209600640 1 -- start start 2024-02-26T10:41:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa209600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040c0ef0 0x7fa2040a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa209600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040c1400 0x7fa2041eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa209600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 0x7fa2040c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa209600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040a8ba0 con 0x7fa2041591b0 2024-02-26T10:41:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa209600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040715e0 con 0x7fa2040c0ef0 2024-02-26T10:41:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa209600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040a8d20 con 0x7fa2040c1400 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa203400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 0x7fa2040c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa203400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 0x7fa2040c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36610/0 (socket says 172.21.15.67:36610) 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa203400640 1 -- 172.21.15.67:0/3140393624 learned_addr learned my addr 172.21.15.67:0/3140393624 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa202a00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040c0ef0 0x7fa2040a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.215+0000 7fa1fbe00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040c1400 0x7fa2041eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa203400640 1 -- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040c1400 msgr2=0x7fa2041eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa203400640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040c1400 0x7fa2041eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa203400640 1 -- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040c0ef0 msgr2=0x7fa2040a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa203400640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040c0ef0 0x7fa2040a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa203400640 1 -- 172.21.15.67:0/3140393624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2041ebe00 con 0x7fa2041591b0 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa202a00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040c0ef0 0x7fa2040a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa1fbe00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040c1400 0x7fa2041eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa203400640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 0x7fa2040c5380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00cef0 tx=0x7fa1fc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc01a070 con 0x7fa2041591b0 2024-02-26T10:41:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00d860 con 0x7fa2041591b0 2024-02-26T10:41:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00dc50 con 0x7fa2041591b0 2024-02-26T10:41:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.216+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20414f610 con 0x7fa2041591b0 2024-02-26T10:41:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.217+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20414fbd0 con 0x7fa2041591b0 2024-02-26T10:41:34.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.218+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20414fe10 con 0x7fa2041591b0 2024-02-26T10:41:34.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.219+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1fc01d070 con 0x7fa2041591b0 2024-02-26T10:41:34.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.220+0000 7fa1faa00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa1e006b4d0 0x7fa1e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.220+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7fa1fc08ab80 con 0x7fa2041591b0 2024-02-26T10:41:34.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.220+0000 7fa202a00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa1e006b4d0 0x7fa1e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.221+0000 7fa202a00640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa1e006b4d0 0x7fa1e006d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000f60 tx=0x7fa1f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:34.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.224+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa20414fe10 con 0x7fa2041591b0 2024-02-26T10:41:34.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.422+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa20406f010 con 0x7fa2041591b0 2024-02-26T10:41:34.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.423+0000 7fa1faa00640 1 -- 172.21.15.67:0/3140393624 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v64) v1 ==== 105+0+11 (secure 0 0 0) 0x7fa1fc059a10 con 0x7fa2041591b0 2024-02-26T10:41:34.422 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:41:34.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa1e006b4d0 msgr2=0x7fa1e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa1e006b4d0 0x7fa1e006d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000f60 tx=0x7fa1f4009040 comp rx=0 tx=0).stop 2024-02-26T10:41:34.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 msgr2=0x7fa2040c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 0x7fa2040c5380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00cef0 tx=0x7fa1fc00a710 comp rx=0 tx=0).stop 2024-02-26T10:41:34.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 shutdown_connections 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2041591b0 0x7fa2040c5380 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040c1400 0x7fa2041eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa1e006b4d0 0x7fa1e006d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 --2- 172.21.15.67:0/3140393624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040c0ef0 0x7fa2040a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 >> 172.21.15.67:0/3140393624 conn(0x7fa2040c6da0 msgr2=0x7fa204154420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 shutdown_connections 2024-02-26T10:41:34.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.426+0000 7fa209600640 1 -- 172.21.15.67:0/3140393624 wait complete. 2024-02-26T10:41:34.445 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:41:34.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.673+0000 7f9fc3800640 1 Processor -- start 2024-02-26T10:41:34.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.673+0000 7f9fc3800640 1 -- start start 2024-02-26T10:41:34.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.674+0000 7f9fc3800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9fbc1503d0 0x7f9fbc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.674+0000 7f9fc3800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9fbc06e390 0x7f9fbc1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.674+0000 7f9fc3800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0a6eb0 con 0x7f9fbc0c2950 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.674+0000 7f9fc3800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0a7190 con 0x7f9fbc1503d0 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.674+0000 7f9fc3800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0a7470 con 0x7f9fbc06e390 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9fbc1503d0 0x7f9fbc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc1800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9fbc0c2950 0x7f9fbc156320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56300/0 (socket says 172.21.15.67:56300) 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc1800640 1 -- 172.21.15.67:0/2690147691 learned_addr learned my addr 172.21.15.67:0/2690147691 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:34.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fbbe00640 1 --2- 172.21.15.67:0/2690147691 >> v2:172.21.15.82:3300/0 conn(0x7f9fbc06e390 0x7f9fbc1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc0e00640 1 -- 172.21.15.67:0/2690147691 >> v2:172.21.15.82:3300/0 conn(0x7f9fbc06e390 msgr2=0x7f9fbc1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:34.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc0e00640 1 --2- 172.21.15.67:0/2690147691 >> v2:172.21.15.82:3300/0 conn(0x7f9fbc06e390 0x7f9fbc1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc0e00640 1 -- 172.21.15.67:0/2690147691 >> v1:172.21.15.67:6789/0 conn(0x7f9fbc0c2950 legacy=0x7f9fbc156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.675+0000 7f9fc0e00640 1 -- 172.21.15.67:0/2690147691 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc0a7750 con 0x7f9fbc1503d0 2024-02-26T10:41:34.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.677+0000 7f9fc0e00640 1 --2- 172.21.15.67:0/2690147691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc06de50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0734b0 tx=0x7f9fb002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665e045a33690143 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:34.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.677+0000 7f9fbb400640 1 -- 172.21.15.67:0/2690147691 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0038070 con 0x7f9fbc1503d0 2024-02-26T10:41:34.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.677+0000 7f9fbb400640 1 -- 172.21.15.67:0/2690147691 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb0031710 con 0x7f9fbc1503d0 2024-02-26T10:41:34.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.677+0000 7f9fbb400640 1 -- 172.21.15.67:0/2690147691 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0031ae0 con 0x7f9fbc1503d0 2024-02-26T10:41:34.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.677+0000 7f9fc3800640 1 -- 172.21.15.67:0/2690147691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc1503d0 msgr2=0x7f9fbc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.677+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2690147691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc06de50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0734b0 tx=0x7f9fb002b7b0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.678+0000 7f9fc3800640 1 -- 172.21.15.67:0/2690147691 shutdown_connections 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.678+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2690147691 >> v2:172.21.15.82:3300/0 conn(0x7f9fbc06e390 0x7f9fbc1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.678+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2690147691 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc06de50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.678+0000 7f9fc3800640 1 -- 172.21.15.67:0/2690147691 >> 172.21.15.67:0/2690147691 conn(0x7f9fbc0c6da0 msgr2=0x7f9fbc0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.678+0000 7f9fc3800640 1 -- 172.21.15.67:0/2690147691 shutdown_connections 2024-02-26T10:41:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.678+0000 7f9fc3800640 1 -- 172.21.15.67:0/2690147691 wait complete. 2024-02-26T10:41:34.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.679+0000 7f9fc3800640 1 Processor -- start 2024-02-26T10:41:34.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.680+0000 7f9fc3800640 1 -- start start 2024-02-26T10:41:34.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc3800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fbc06e390 0x7f9fbc154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc3800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 0x7f9fbc14ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc3800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc3800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc1507b0 con 0x7f9fbc0c2950 2024-02-26T10:41:34.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc3800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc06ef00 con 0x7f9fbc1503d0 2024-02-26T10:41:34.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc3800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0c0ef0 con 0x7f9fbc06e390 2024-02-26T10:41:34.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 0x7f9fbc14ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 0x7f9fbc14ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36634/0 (socket says 172.21.15.67:36634) 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 -- 172.21.15.67:0/2645281489 learned_addr learned my addr 172.21.15.67:0/2645281489 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc0e00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fbc06e390 0x7f9fbc154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 -- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fbc06e390 msgr2=0x7f9fbc154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fc1800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fbc06e390 0x7f9fbc154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 -- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fbc1503d0 msgr2=0x7f9fbc155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.681+0000 7f9fbbe00640 1 -- 172.21.15.67:0/2645281489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc14e970 con 0x7f9fbc0c2950 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.682+0000 7f9fbbe00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 0x7f9fbc14ab00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800d800 tx=0x7f9fa800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.682+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800b660 con 0x7f9fbc0c2950 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.682+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8019070 con 0x7f9fbc0c2950 2024-02-26T10:41:34.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.682+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800ba20 con 0x7f9fbc0c2950 2024-02-26T10:41:34.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.682+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc14ec50 con 0x7f9fbc0c2950 2024-02-26T10:41:34.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.682+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc14f210 con 0x7f9fbc0c2950 2024-02-26T10:41:34.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.683+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc074a30 con 0x7f9fbc0c2950 2024-02-26T10:41:34.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.684+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fa800bbc0 con 0x7f9fbc0c2950 2024-02-26T10:41:34.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.686+0000 7f9fb8c00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9f9c06b500 0x7f9f9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:34.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.686+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f9fa808a620 con 0x7f9fbc0c2950 2024-02-26T10:41:34.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.686+0000 7f9fc0e00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9f9c06b500 0x7f9f9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:34.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.687+0000 7f9fc0e00640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9f9c06b500 0x7f9f9c06d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0035000 tx=0x7f9fb0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:34.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.691+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fa8059490 con 0x7f9fbc0c2950 2024-02-26T10:41:34.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.888+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f9fbc06ec50 con 0x7f9fbc0c2950 2024-02-26T10:41:34.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.889+0000 7f9fb8c00640 1 -- 172.21.15.67:0/2645281489 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v64) v1 ==== 101+0+11 (secure 0 0 0) 0x7f9fa8058d90 con 0x7f9fbc0c2950 2024-02-26T10:41:34.888 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.891+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9f9c06b500 msgr2=0x7f9f9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.891+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9f9c06b500 0x7f9f9c06d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0035000 tx=0x7f9fb0008030 comp rx=0 tx=0).stop 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 msgr2=0x7f9fbc14ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 0x7f9fbc14ab00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800d800 tx=0x7f9fa800dcc0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 shutdown_connections 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9f9c06b500 0x7f9f9c06d9c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fbc1503d0 0x7f9fbc155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fbc0c2950 0x7f9fbc14ab00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 --2- 172.21.15.67:0/2645281489 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fbc06e390 0x7f9fbc154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 >> 172.21.15.67:0/2645281489 conn(0x7f9fbc0c6da0 msgr2=0x7f9fbc0c2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 shutdown_connections 2024-02-26T10:41:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:34.892+0000 7f9fc3800640 1 -- 172.21.15.67:0/2645281489 wait complete. 2024-02-26T10:41:34.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.128+0000 7fb195200640 1 Processor -- start 2024-02-26T10:41:35.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.129+0000 7fb195200640 1 -- start start 2024-02-26T10:41:35.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.130+0000 7fb195200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb1900c8510 0x7fb1900c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.130+0000 7fb195200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb190152760 con 0x7fb1900c7020 2024-02-26T10:41:35.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.130+0000 7fb195200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19006d9a0 con 0x7fb190152380 2024-02-26T10:41:35.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.130+0000 7fb195200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1900c7550 con 0x7fb1900c8510 2024-02-26T10:41:35.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.130+0000 7fb18ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb1900c7020 0x7fb19006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56306/0 (socket says 172.21.15.67:56306) 2024-02-26T10:41:35.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.130+0000 7fb18ea00640 1 -- 172.21.15.67:0/466549572 learned_addr learned my addr 172.21.15.67:0/466549572 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18fe00640 1 --2- 172.21.15.67:0/466549572 >> v2:172.21.15.82:3300/0 conn(0x7fb1900c8510 0x7fb1900c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041274992 0 0) 0x7fb190152760 con 0x7fb1900c7020 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb174003520 con 0x7fb1900c7020 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158772507 0 0) 0x7fb174003520 con 0x7fb1900c7020 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 >> v2:172.21.15.82:3300/0 conn(0x7fb1900c8510 msgr2=0x7fb1900c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 --2- 172.21.15.67:0/466549572 >> v2:172.21.15.82:3300/0 conn(0x7fb1900c8510 0x7fb1900c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 >> v1:172.21.15.82:6789/0 conn(0x7fb190152380 legacy=0x7fb190155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1900c7830 con 0x7fb1900c7020 2024-02-26T10:41:35.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fb178001670 con 0x7fb1900c7020 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 >> v1:172.21.15.67:6789/0 conn(0x7fb1900c7020 legacy=0x7fb19006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1740039b0 0x7fb174003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb174004800 0x7fb174004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.131+0000 7fb18e000640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1740051c0 0x7fb17400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900c7550 con 0x7fb1740051c0 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17400c160 con 0x7fb1740039b0 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17400c360 con 0x7fb174004800 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18fe00640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1740051c0 0x7fb17400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18f400640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1740039b0 0x7fb174003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18fe00640 1 -- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb174004800 msgr2=0x7fb174004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18fe00640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb174004800 0x7fb174004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18fe00640 1 -- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1740039b0 msgr2=0x7fb174003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18fe00640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1740039b0 0x7fb174003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.132+0000 7fb18fe00640 1 -- 172.21.15.67:0/466549572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174003520 con 0x7fb1740051c0 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.133+0000 7fb18fe00640 1 -- 172.21.15.67:0/466549572 reap_dead start 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.133+0000 7fb18fe00640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1740051c0 0x7fb17400b9c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb188009fc0 tx=0x7fb174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4392e1e5320ec71b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188009c10 con 0x7fb1740051c0 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18802cca0 con 0x7fb1740051c0 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb18e000640 1 -- 172.21.15.67:0/466549572 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188003bd0 con 0x7fb1740051c0 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 -- 172.21.15.67:0/466549572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1740051c0 msgr2=0x7fb17400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1740051c0 0x7fb17400b9c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb188009fc0 tx=0x7fb174005830 comp rx=0 tx=0).stop 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 -- 172.21.15.67:0/466549572 shutdown_connections 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 --2- 172.21.15.67:0/466549572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1740051c0 0x7fb17400b9c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 -- 172.21.15.67:0/466549572 >> 172.21.15.67:0/466549572 conn(0x7fb1900ac1a0 msgr2=0x7fb1900ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 -- 172.21.15.67:0/466549572 shutdown_connections 2024-02-26T10:41:35.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.134+0000 7fb195200640 1 -- 172.21.15.67:0/466549572 wait complete. 2024-02-26T10:41:35.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.136+0000 7fb195200640 1 Processor -- start 2024-02-26T10:41:35.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.136+0000 7fb195200640 1 -- start start 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.137+0000 7fb195200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1740051c0 0x7fb1901ea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.137+0000 7fb195200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1900ae6a0 0x7fb1901483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.137+0000 7fb195200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 0x7fb1900af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.137+0000 7fb195200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19006d290 con 0x7fb1901563a0 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.137+0000 7fb195200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19006cd70 con 0x7fb1900ae6a0 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.137+0000 7fb195200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19006d410 con 0x7fb1740051c0 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 0x7fb1900af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 0x7fb1900af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36660/0 (socket says 172.21.15.67:36660) 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 -- 172.21.15.67:0/2490660582 learned_addr learned my addr 172.21.15.67:0/2490660582 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18ea00640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1900ae6a0 0x7fb1901483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18f400640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1740051c0 0x7fb1901ea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 -- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1740051c0 msgr2=0x7fb1901ea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1740051c0 0x7fb1901ea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 -- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1900ae6a0 msgr2=0x7fb1901483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1900ae6a0 0x7fb1901483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.138+0000 7fb18fe00640 1 -- 172.21.15.67:0/2490660582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1900ad600 con 0x7fb1901563a0 2024-02-26T10:41:35.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.139+0000 7fb18fe00640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 0x7fb1900af610 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb188035d50 tx=0x7fb188009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:35.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.139+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188003990 con 0x7fb1901563a0 2024-02-26T10:41:35.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.139+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188044070 con 0x7fb1901563a0 2024-02-26T10:41:35.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.139+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188003d90 con 0x7fb1901563a0 2024-02-26T10:41:35.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.139+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1900ad8e0 con 0x7fb1901563a0 2024-02-26T10:41:35.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.139+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1900adea0 con 0x7fb1901563a0 2024-02-26T10:41:35.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.140+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1900aea80 con 0x7fb1901563a0 2024-02-26T10:41:35.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.141+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb188005d10 con 0x7fb1901563a0 2024-02-26T10:41:35.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.142+0000 7fb187400640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb16c06b500 0x7fb16c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.142+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7fb1880ac960 con 0x7fb1901563a0 2024-02-26T10:41:35.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.143+0000 7fb18f400640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb16c06b500 0x7fb16c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.144+0000 7fb18f400640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb16c06b500 0x7fb16c06d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb180000c00 tx=0x7fb180009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:35.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.147+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb18807b730 con 0x7fb1901563a0 2024-02-26T10:41:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.383+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1900c5bb0 con 0x7fb1901563a0 2024-02-26T10:41:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.385+0000 7fb187400640 1 -- 172.21.15.67:0/2490660582 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fb1900c5bb0 con 0x7fb1901563a0 2024-02-26T10:41:35.383 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:35.384 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/2899333510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2899333510},{"type":"v1","addr":"172.21.15.67:6834","nonce":2899333510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-02-26T10:41:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb16c06b500 msgr2=0x7fb16c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb16c06b500 0x7fb16c06d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb180000c00 tx=0x7fb180009040 comp rx=0 tx=0).stop 2024-02-26T10:41:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 msgr2=0x7fb1900af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 0x7fb1900af610 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb188035d50 tx=0x7fb188009750 comp rx=0 tx=0).stop 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 shutdown_connections 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1901563a0 0x7fb1900af610 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb16c06b500 0x7fb16c06d9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1900ae6a0 0x7fb1901483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 --2- 172.21.15.67:0/2490660582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1740051c0 0x7fb1901ea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.387+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 >> 172.21.15.67:0/2490660582 conn(0x7fb1900ac1a0 msgr2=0x7fb190150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.388+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 shutdown_connections 2024-02-26T10:41:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.388+0000 7fb195200640 1 -- 172.21.15.67:0/2490660582 wait complete. 2024-02-26T10:41:35.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:35.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.623+0000 7f25e5c00640 1 Processor -- start 2024-02-26T10:41:35.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.623+0000 7f25e5c00640 1 -- start start 2024-02-26T10:41:35.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.625+0000 7f25e5c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25e00c2310 0x7f25e0153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.625+0000 7f25e5c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006b000 0x7f25e01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.625+0000 7f25e5c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e0155f10 con 0x7f25e006ac20 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.625+0000 7f25e5c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e006b470 con 0x7f25e00c2310 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.625+0000 7f25e5c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00c28d0 con 0x7f25e006b000 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25df400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25e00c2310 0x7f25e0153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25dfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f25e006ac20 0x7f25e0152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56310/0 (socket says 172.21.15.67:56310) 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25dfe00640 1 -- 172.21.15.67:0/742655591 learned_addr learned my addr 172.21.15.67:0/742655591 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25e5200640 1 --2- 172.21.15.67:0/742655591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006b000 0x7f25e01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25df400640 1 -- 172.21.15.67:0/742655591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006b000 msgr2=0x7f25e01524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25df400640 1 --2- 172.21.15.67:0/742655591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006b000 0x7f25e01524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25df400640 1 -- 172.21.15.67:0/742655591 >> v1:172.21.15.67:6789/0 conn(0x7f25e006ac20 legacy=0x7f25e0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25df400640 1 -- 172.21.15.67:0/742655591 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0155150 con 0x7f25e00c2310 2024-02-26T10:41:35.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.626+0000 7f25dea00640 1 -- 172.21.15.67:0/742655591 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820543155 0 0) 0x7f25e0155f10 con 0x7f25e006ac20 2024-02-26T10:41:35.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.627+0000 7f25df400640 1 --2- 172.21.15.67:0/742655591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e00c2310 0x7f25e0153630 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097a0 tx=0x7f25cc02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa8da784e9da21e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25dea00640 1 -- 172.21.15.67:0/742655591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc039070 con 0x7f25e00c2310 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25dea00640 1 -- 172.21.15.67:0/742655591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc0316f0 con 0x7f25e00c2310 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25dea00640 1 -- 172.21.15.67:0/742655591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc031ac0 con 0x7f25e00c2310 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25e5c00640 1 -- 172.21.15.67:0/742655591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e00c2310 msgr2=0x7f25e0153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25e5c00640 1 --2- 172.21.15.67:0/742655591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e00c2310 0x7f25e0153630 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097a0 tx=0x7f25cc02b790 comp rx=0 tx=0).stop 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25e5c00640 1 -- 172.21.15.67:0/742655591 shutdown_connections 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25e5c00640 1 --2- 172.21.15.67:0/742655591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006b000 0x7f25e01524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25e5c00640 1 --2- 172.21.15.67:0/742655591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e00c2310 0x7f25e0153630 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.628+0000 7f25e5c00640 1 -- 172.21.15.67:0/742655591 >> 172.21.15.67:0/742655591 conn(0x7f25e00c63f0 msgr2=0x7f25e00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.629+0000 7f25e5c00640 1 -- 172.21.15.67:0/742655591 shutdown_connections 2024-02-26T10:41:35.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.629+0000 7f25e5c00640 1 -- 172.21.15.67:0/742655591 wait complete. 2024-02-26T10:41:35.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.630+0000 7f25e5c00640 1 Processor -- start 2024-02-26T10:41:35.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.630+0000 7f25e5c00640 1 -- start start 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.631+0000 7f25e5c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006ac20 0x7f25e015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.631+0000 7f25e5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 0x7f25e0141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.631+0000 7f25e5c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e00c2310 0x7f25e0142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.631+0000 7f25e5c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01559f0 con 0x7f25e006b000 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.631+0000 7f25e5c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0155710 con 0x7f25e00c2310 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.631+0000 7f25e5c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0148850 con 0x7f25e006ac20 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 0x7f25e0141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 0x7f25e0141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36678/0 (socket says 172.21.15.67:36678) 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 -- 172.21.15.67:0/3401326103 learned_addr learned my addr 172.21.15.67:0/3401326103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:35.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25dfe00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006ac20 0x7f25e015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 -- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006ac20 msgr2=0x7f25e015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25e5200640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e00c2310 0x7f25e0142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006ac20 0x7f25e015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 -- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e00c2310 msgr2=0x7f25e0142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e00c2310 0x7f25e0142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 -- 172.21.15.67:0/3401326103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0142650 con 0x7f25e006b000 2024-02-26T10:41:35.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.632+0000 7f25df400640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 0x7f25e0141bd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f25cc000c00 tx=0x7f25cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.633+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc043070 con 0x7f25e006b000 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.633+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc034680 con 0x7f25e006b000 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.633+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc039040 con 0x7f25e006b000 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.633+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00c60c0 con 0x7f25e006b000 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.633+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e00c42a0 con 0x7f25e006b000 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.634+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0074a30 con 0x7f25e006b000 2024-02-26T10:41:35.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.635+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25cc046070 con 0x7f25e006b000 2024-02-26T10:41:35.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.636+0000 7f25d7e00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:35.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.636+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f25cc0abea0 con 0x7f25e006b000 2024-02-26T10:41:35.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.637+0000 7f25dfe00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:35.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.638+0000 7f25dfe00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25c006b4d0 0x7f25c006d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f25d0000f60 tx=0x7f25d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:35.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.640+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25cc07abe0 con 0x7f25e006b000 2024-02-26T10:41:35.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.841+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25e00c3ae0 con 0x7f25e006b000 2024-02-26T10:41:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.843+0000 7f25d7e00640 1 -- 172.21.15.67:0/3401326103 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16827 (secure 0 0 0) 0x7f25cc07a4e0 con 0x7f25e006b000 2024-02-26T10:41:35.843 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:35.843 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":64,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:33.014527+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6834/1127454674":"2024-02-27T10:41:29.521347+0000","172.21.15.67:6832/1127454674":"2024-02-27T10:41:29.521347+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.846+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25c006b4d0 msgr2=0x7f25c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.846+0000 7f25e5c00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25c006b4d0 0x7f25c006d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f25d0000f60 tx=0x7f25d0009040 comp rx=0 tx=0).stop 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.846+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 msgr2=0x7f25e0141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.846+0000 7f25e5c00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 0x7f25e0141bd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f25cc000c00 tx=0x7f25cc0046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 shutdown_connections 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e00c2310 0x7f25e0142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25e006b000 0x7f25e0141bd0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 --2- 172.21.15.67:0/3401326103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e006ac20 0x7f25e015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 >> 172.21.15.67:0/3401326103 conn(0x7f25e00c63f0 msgr2=0x7f25e0147760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 shutdown_connections 2024-02-26T10:41:35.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:35.847+0000 7f25e5c00640 1 -- 172.21.15.67:0/3401326103 wait complete. 2024-02-26T10:41:35.863 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-02-26T10:41:35.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:36.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.073+0000 7f8ad3a00640 1 Processor -- start 2024-02-26T10:41:36.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.073+0000 7f8ad3a00640 1 -- start start 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad3a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8acc06eb40 0x7f8acc072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad3a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 0x7f8acc072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad3a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8acc1546a0 con 0x7f8acc070f10 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad3a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8acc154980 con 0x7f8acc06eb40 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad3a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc154c60 con 0x7f8acc06f270 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad1a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8acc070f10 0x7f8acc152400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56316/0 (socket says 172.21.15.67:56316) 2024-02-26T10:41:36.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.075+0000 7f8ad1a00640 1 -- 172.21.15.67:0/3131620264 learned_addr learned my addr 172.21.15.67:0/3131620264 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acbe00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 0x7f8acc072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8ad1000640 1 --2- 172.21.15.67:0/3131620264 >> v2:172.21.15.82:3300/0 conn(0x7f8acc06eb40 0x7f8acc072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048200629 0 0) 0x7f8acc1546a0 con 0x7f8acc070f10 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab4003520 con 0x7f8acc070f10 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794247777 0 0) 0x7f8ab4003520 con 0x7f8acc070f10 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 msgr2=0x7f8acc072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 0x7f8acc072ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 >> v2:172.21.15.82:3300/0 conn(0x7f8acc06eb40 msgr2=0x7f8acc072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 --2- 172.21.15.67:0/3131620264 >> v2:172.21.15.82:3300/0 conn(0x7f8acc06eb40 0x7f8acc072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc152b10 con 0x7f8acc070f10 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8ac40019f0 con 0x7f8acc070f10 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 >> v1:172.21.15.67:6789/0 conn(0x7f8acc070f10 legacy=0x7f8acc152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 0x7f8ab40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab4003f30 0x7f8ab4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8ab40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc154980 con 0x7f8ab4004d40 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc154c60 con 0x7f8ab4003f30 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.076+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab400b9c0 con 0x7f8acc06f270 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1a00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8ab40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1a00640 1 -- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 msgr2=0x7f8ab40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1000640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 0x7f8ab40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8acbe00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab4003f30 0x7f8ab4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1a00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc06f270 0x7f8ab40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1a00640 1 -- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab4003f30 msgr2=0x7f8ab4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1a00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab4003f30 0x7f8ab4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8ad1a00640 1 -- 172.21.15.67:0/3131620264 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4003520 con 0x7f8ab4004d40 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.077+0000 7f8acbe00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab4003f30 0x7f8ab4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:36.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.078+0000 7f8ad1a00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8ab40051c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400a8a0 tx=0x7f8ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad1cf789dd247bc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:36.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.078+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400a4c0 con 0x7f8ab4004d40 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.078+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4034650 con 0x7f8ab4004d40 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.078+0000 7f8acb400640 1 -- 172.21.15.67:0/3131620264 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4034a80 con 0x7f8ab4004d40 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad3a00640 1 -- 172.21.15.67:0/3131620264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 msgr2=0x7f8ab40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/3131620264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8ab40051c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400a8a0 tx=0x7f8ab4005830 comp rx=0 tx=0).stop 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad1a00640 1 -- 172.21.15.67:0/3131620264 reap_dead start 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad3a00640 1 -- 172.21.15.67:0/3131620264 shutdown_connections 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad3a00640 1 -- 172.21.15.67:0/3131620264 >> 172.21.15.67:0/3131620264 conn(0x7f8acc0a84a0 msgr2=0x7f8acc0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad3a00640 1 -- 172.21.15.67:0/3131620264 shutdown_connections 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.079+0000 7f8ad3a00640 1 -- 172.21.15.67:0/3131620264 wait complete. 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.081+0000 7f8ad3a00640 1 Processor -- start 2024-02-26T10:41:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.081+0000 7f8ad3a00640 1 -- start start 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad3a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8acc149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad3a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc14a6e0 0x7f8acc0aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad3a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8acc145910 0x7f8acc1441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad3a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc072ed0 con 0x7f8ab4004d40 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad3a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc152ef0 con 0x7f8acc145910 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad3a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc073050 con 0x7f8acc14a6e0 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8acc149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8acc149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36708/0 (socket says 172.21.15.67:36708) 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.082+0000 7f8ad1000640 1 -- 172.21.15.67:0/2612428318 learned_addr learned my addr 172.21.15.67:0/2612428318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8acbe00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc14a6e0 0x7f8acc0aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8acc145910 0x7f8acc1441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1000640 1 -- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc14a6e0 msgr2=0x7f8acc0aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1000640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc14a6e0 0x7f8acc0aa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1000640 1 -- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8acc145910 msgr2=0x7f8acc1441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1000640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8acc145910 0x7f8acc1441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1000640 1 -- 172.21.15.67:0/2612428318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc148010 con 0x7f8ab4004d40 2024-02-26T10:41:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8acc145910 0x7f8acc1441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.083+0000 7f8ad1000640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8acc149750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0010940 tx=0x7f8ac0010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.084+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac000f040 con 0x7f8ab4004d40 2024-02-26T10:41:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.084+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac000ec20 con 0x7f8ab4004d40 2024-02-26T10:41:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.084+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac001c400 con 0x7f8ab4004d40 2024-02-26T10:41:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.084+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc144f00 con 0x7f8ab4004d40 2024-02-26T10:41:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.084+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc148680 con 0x7f8ab4004d40 2024-02-26T10:41:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.085+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc148a60 con 0x7f8ab4004d40 2024-02-26T10:41:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.086+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ac001c5a0 con 0x7f8ab4004d40 2024-02-26T10:41:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.087+0000 7f8ac8c00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ab006b500 0x7f8ab006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.087+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f8ac008a920 con 0x7f8ab4004d40 2024-02-26T10:41:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.087+0000 7f8acbe00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ab006b500 0x7f8ab006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.088+0000 7f8acbe00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ab006b500 0x7f8ab006d9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8acc150b40 tx=0x7f8abc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:36.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.091+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ac0018030 con 0x7f8ab4004d40 2024-02-26T10:41:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.329+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8acc06e840 con 0x7f8ab4004d40 2024-02-26T10:41:36.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.330+0000 7f8ac8c00640 1 -- 172.21.15.67:0/2612428318 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f8ac00590a0 con 0x7f8ab4004d40 2024-02-26T10:41:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:36.330 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/2899333510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2899333510},{"type":"v1","addr":"172.21.15.67:6834","nonce":2899333510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ab006b500 msgr2=0x7f8ab006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ab006b500 0x7f8ab006d9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8acc150b40 tx=0x7f8abc008040 comp rx=0 tx=0).stop 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 msgr2=0x7f8acc149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8acc149750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0010940 tx=0x7f8ac0010e00 comp rx=0 tx=0).stop 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 shutdown_connections 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8ab006b500 0x7f8ab006d9c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8acc145910 0x7f8acc1441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8acc14a6e0 0x7f8acc0aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 --2- 172.21.15.67:0/2612428318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab4004d40 0x7f8acc149750 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.333+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 >> 172.21.15.67:0/2612428318 conn(0x7f8acc0a84a0 msgr2=0x7f8acc0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.334+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 shutdown_connections 2024-02-26T10:41:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.334+0000 7f8ad3a00640 1 -- 172.21.15.67:0/2612428318 wait complete. 2024-02-26T10:41:36.350 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.571+0000 7f421ac00640 1 Processor -- start 2024-02-26T10:41:36.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.572+0000 7f421ac00640 1 -- start start 2024-02-26T10:41:36.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421ac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4214152380 0x7f42140c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421ac00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f42140c7020 0x7f4214155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421ac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4214152760 con 0x7f421406d290 2024-02-26T10:41:36.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421ac00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4214155cd0 con 0x7f42140c7020 2024-02-26T10:41:36.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421ac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42140c7550 con 0x7f4214152380 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421a200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f421406d290 0x7f42140c4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56326/0 (socket says 172.21.15.67:56326) 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f421a200640 1 -- 172.21.15.67:0/2683134752 learned_addr learned my addr 172.21.15.67:0/2683134752 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.573+0000 7f4218e00640 1 --2- 172.21.15.67:0/2683134752 >> v2:172.21.15.67:3300/0 conn(0x7f42140c7020 0x7f4214155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4219800640 1 --2- 172.21.15.67:0/2683134752 >> v2:172.21.15.82:3300/0 conn(0x7f4214152380 0x7f42140c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4218e00640 1 -- 172.21.15.67:0/2683134752 >> v2:172.21.15.82:3300/0 conn(0x7f4214152380 msgr2=0x7f42140c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4218e00640 1 --2- 172.21.15.67:0/2683134752 >> v2:172.21.15.82:3300/0 conn(0x7f4214152380 0x7f42140c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4218e00640 1 -- 172.21.15.67:0/2683134752 >> v1:172.21.15.67:6789/0 conn(0x7f421406d290 legacy=0x7f42140c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4213e00640 1 -- 172.21.15.67:0/2683134752 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972538737 0 0) 0x7f4214152760 con 0x7f421406d290 2024-02-26T10:41:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4218e00640 1 -- 172.21.15.67:0/2683134752 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140c7830 con 0x7f42140c7020 2024-02-26T10:41:36.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.574+0000 7f4219800640 1 --2- 172.21.15.67:0/2683134752 >> v2:172.21.15.82:3300/0 conn(0x7f4214152380 0x7f42140c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.575+0000 7f4218e00640 1 --2- 172.21.15.67:0/2683134752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140c7020 0x7f4214155790 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f421406ddd0 tx=0x7f420002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2693b7c4bde59a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.575+0000 7f4213e00640 1 -- 172.21.15.67:0/2683134752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000315a0 con 0x7f42140c7020 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.575+0000 7f4213e00640 1 -- 172.21.15.67:0/2683134752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200037070 con 0x7f42140c7020 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.575+0000 7f4213e00640 1 -- 172.21.15.67:0/2683134752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200031960 con 0x7f42140c7020 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 -- 172.21.15.67:0/2683134752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140c7020 msgr2=0x7f4214155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 --2- 172.21.15.67:0/2683134752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140c7020 0x7f4214155790 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f421406ddd0 tx=0x7f420002b840 comp rx=0 tx=0).stop 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 -- 172.21.15.67:0/2683134752 shutdown_connections 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 --2- 172.21.15.67:0/2683134752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f42140c7020 0x7f4214155790 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 --2- 172.21.15.67:0/2683134752 >> v2:172.21.15.82:3300/0 conn(0x7f4214152380 0x7f42140c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 -- 172.21.15.67:0/2683134752 >> 172.21.15.67:0/2683134752 conn(0x7f42140ac1a0 msgr2=0x7f42140ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:36.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 -- 172.21.15.67:0/2683134752 shutdown_connections 2024-02-26T10:41:36.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.576+0000 7f421ac00640 1 -- 172.21.15.67:0/2683134752 wait complete. 2024-02-26T10:41:36.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.578+0000 7f421ac00640 1 Processor -- start 2024-02-26T10:41:36.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.578+0000 7f421ac00640 1 -- start start 2024-02-26T10:41:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f421ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 0x7f42140bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f421ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42140c7020 0x7f42140ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f421ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4214152380 0x7f421414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f421ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42140c56a0 con 0x7f421406d290 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f421ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421406cd70 con 0x7f4214152380 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f421ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42140c5820 con 0x7f42140c7020 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f4219800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 0x7f42140bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f4219800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 0x7f42140bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36734/0 (socket says 172.21.15.67:36734) 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.579+0000 7f4219800640 1 -- 172.21.15.67:0/952560918 learned_addr learned my addr 172.21.15.67:0/952560918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4218e00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42140c7020 0x7f42140ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f421a200640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4214152380 0x7f421414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4219800640 1 -- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42140c7020 msgr2=0x7f42140ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4219800640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42140c7020 0x7f42140ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4219800640 1 -- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4214152380 msgr2=0x7f421414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4219800640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4214152380 0x7f421414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4219800640 1 -- 172.21.15.67:0/952560918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214151530 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4219800640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 0x7f42140bfb70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f420400c7f0 tx=0x7f420400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.580+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204015070 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.581+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4204007820 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.581+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204007bd0 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.581+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42140c00b0 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.581+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42140c0820 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.582+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4214074a30 con 0x7f421406d290 2024-02-26T10:41:36.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.583+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4204018070 con 0x7f421406d290 2024-02-26T10:41:36.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.584+0000 7f4211600640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f41f406b4d0 0x7f41f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.584+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f420408a3b0 con 0x7f421406d290 2024-02-26T10:41:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.585+0000 7f4218e00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f41f406b4d0 0x7f41f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.586+0000 7f4218e00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f41f406b4d0 0x7f41f406d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4200036000 tx=0x7f420003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.588+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4204059180 con 0x7f421406d290 2024-02-26T10:41:36.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.826+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42140c7c50 con 0x7f421406d290 2024-02-26T10:41:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.827+0000 7f4211600640 1 -- 172.21.15.67:0/952560918 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f4204058a80 con 0x7f421406d290 2024-02-26T10:41:36.826 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:36.826 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/2899333510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2899333510},{"type":"v1","addr":"172.21.15.67:6834","nonce":2899333510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:01.229137+0000","modified":"2024-02-26T10:41:04.306213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5179":{"gid":5179,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":7,"addr":"172.21.15.82:6837/2903170369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2903170369},{"type":"v1","addr":"172.21.15.82:6837","nonce":2903170369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-02-26T10:41:36.826 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f41f406b4d0 msgr2=0x7f41f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f41f406b4d0 0x7f41f406d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4200036000 tx=0x7f420003d040 comp rx=0 tx=0).stop 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 msgr2=0x7f42140bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 0x7f42140bfb70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f420400c7f0 tx=0x7f420400ccb0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 shutdown_connections 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f41f406b4d0 0x7f41f406d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4214152380 0x7f421414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42140c7020 0x7f42140ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 --2- 172.21.15.67:0/952560918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f421406d290 0x7f42140bfb70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 >> 172.21.15.67:0/952560918 conn(0x7f42140ac1a0 msgr2=0x7f42140ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 shutdown_connections 2024-02-26T10:41:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:36.830+0000 7f421ac00640 1 -- 172.21.15.67:0/952560918 wait complete. 2024-02-26T10:41:36.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.073+0000 7f6822e00640 1 Processor -- start 2024-02-26T10:41:37.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.073+0000 7f6822e00640 1 -- start start 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c0c1c10 0x7f681c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f681c0c0850 0x7f681c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f681c06b160 0x7f681c06e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c155cf0 con 0x7f681c06b160 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c06dca0 con 0x7f681c0c0850 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0c2a70 con 0x7f681c0c1c10 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.075+0000 7f6822400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f681c06b160 0x7f681c06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f681c06b160 0x7f681c06e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36738/0 (socket says 172.21.15.67:36738) 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 -- 172.21.15.67:0/4157142744 learned_addr learned my addr 172.21.15.67:0/4157142744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6821a00640 1 --2- 172.21.15.67:0/4157142744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c0c1c10 0x7f681c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6821000640 1 --2- 172.21.15.67:0/4157142744 >> v2:172.21.15.82:3300/0 conn(0x7f681c0c0850 0x7f681c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 -- 172.21.15.67:0/4157142744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c0c1c10 msgr2=0x7f681c1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 --2- 172.21.15.67:0/4157142744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c0c1c10 0x7f681c1557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 -- 172.21.15.67:0/4157142744 >> v2:172.21.15.82:3300/0 conn(0x7f681c0c0850 msgr2=0x7f681c06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 --2- 172.21.15.67:0/4157142744 >> v2:172.21.15.82:3300/0 conn(0x7f681c0c0850 0x7f681c06ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6822400640 1 -- 172.21.15.67:0/4157142744 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0c21d0 con 0x7f681c06b160 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.076+0000 7f6821000640 1 --2- 172.21.15.67:0/4157142744 >> v2:172.21.15.82:3300/0 conn(0x7f681c0c0850 0x7f681c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.077+0000 7f6822400640 1 --2- 172.21.15.67:0/4157142744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c06b160 0x7f681c06e180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f681c06ed30 tx=0x7f681402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1614d5170bc373 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f681be00640 1 -- 172.21.15.67:0/4157142744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681402c950 con 0x7f681c06b160 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f681be00640 1 -- 172.21.15.67:0/4157142744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681402caf0 con 0x7f681c06b160 2024-02-26T10:41:37.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f681be00640 1 -- 172.21.15.67:0/4157142744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68140343f0 con 0x7f681c06b160 2024-02-26T10:41:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 -- 172.21.15.67:0/4157142744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c06b160 msgr2=0x7f681c06e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 --2- 172.21.15.67:0/4157142744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c06b160 0x7f681c06e180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f681c06ed30 tx=0x7f681402bdb0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 -- 172.21.15.67:0/4157142744 shutdown_connections 2024-02-26T10:41:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 --2- 172.21.15.67:0/4157142744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c06b160 0x7f681c06e180 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 --2- 172.21.15.67:0/4157142744 >> v2:172.21.15.82:3300/0 conn(0x7f681c0c0850 0x7f681c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 --2- 172.21.15.67:0/4157142744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c0c1c10 0x7f681c1557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.078+0000 7f6822e00640 1 -- 172.21.15.67:0/4157142744 >> 172.21.15.67:0/4157142744 conn(0x7f681c0c63f0 msgr2=0x7f681c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.079+0000 7f6822e00640 1 -- 172.21.15.67:0/4157142744 shutdown_connections 2024-02-26T10:41:37.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.079+0000 7f6822e00640 1 -- 172.21.15.67:0/4157142744 wait complete. 2024-02-26T10:41:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.080+0000 7f6822e00640 1 Processor -- start 2024-02-26T10:41:37.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.080+0000 7f6822e00640 1 -- start start 2024-02-26T10:41:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.081+0000 7f6822e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c06b160 0x7f681c0ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.081+0000 7f6822e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c0c0850 0x7f681c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.081+0000 7f6822e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 0x7f681c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.081+0000 7f6822e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0c0df0 con 0x7f681c0c1c10 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.081+0000 7f6822e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0c2790 con 0x7f681c0c0850 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.081+0000 7f6822e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c0c0f70 con 0x7f681c06b160 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 0x7f681c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 0x7f681c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36748/0 (socket says 172.21.15.67:36748) 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 -- 172.21.15.67:0/432991295 learned_addr learned my addr 172.21.15.67:0/432991295 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6821a00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c06b160 0x7f681c0ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6821000640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c0c0850 0x7f681c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 -- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c06b160 msgr2=0x7f681c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c06b160 0x7f681c0ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 -- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c0c0850 msgr2=0x7f681c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c0c0850 0x7f681c157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6822400640 1 -- 172.21.15.67:0/432991295 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c14b710 con 0x7f681c0c1c10 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.082+0000 7f6821000640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c0c0850 0x7f681c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:37.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.083+0000 7f6822400640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 0x7f681c150a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6814009e20 tx=0x7f6814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.083+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814043020 con 0x7f681c0c1c10 2024-02-26T10:41:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.083+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681403e450 con 0x7f681c0c1c10 2024-02-26T10:41:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.083+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c14b9f0 con 0x7f681c0c1c10 2024-02-26T10:41:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.083+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681403e820 con 0x7f681c0c1c10 2024-02-26T10:41:37.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.083+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c158c00 con 0x7f681c0c1c10 2024-02-26T10:41:37.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.084+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c0aa3b0 con 0x7f681c0c1c10 2024-02-26T10:41:37.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.085+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6814046020 con 0x7f681c0c1c10 2024-02-26T10:41:37.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.086+0000 7f6819600640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b520 0x7f67fc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.086+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f68140aca60 con 0x7f681c0c1c10 2024-02-26T10:41:37.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.087+0000 7f6821a00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b520 0x7f67fc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.088+0000 7f6821a00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b520 0x7f67fc06d9e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6810006fd0 tx=0x7f6810008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:37.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.091+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f681404d030 con 0x7f681c0c1c10 2024-02-26T10:41:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.291+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f681c074a30 con 0x7f681c0c1c10 2024-02-26T10:41:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.294+0000 7f6819600640 1 -- 172.21.15.67:0/432991295 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16827 (secure 0 0 0) 0x7f681407b8b0 con 0x7f681c0c1c10 2024-02-26T10:41:37.293 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:37.293 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":64,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:33.014527+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:40:58.810192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:41:00.247724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6834/1127454674":"2024-02-27T10:41:29.521347+0000","172.21.15.67:6832/1127454674":"2024-02-27T10:41:29.521347+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b520 msgr2=0x7f67fc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b520 0x7f67fc06d9e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6810006fd0 tx=0x7f6810008040 comp rx=0 tx=0).stop 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 msgr2=0x7f681c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 0x7f681c150a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6814009e20 tx=0x7f6814004a40 comp rx=0 tx=0).stop 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 shutdown_connections 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f681c0c1c10 0x7f681c150a10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b520 0x7f67fc06d9e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c0c0850 0x7f681c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 --2- 172.21.15.67:0/432991295 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f681c06b160 0x7f681c0ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 >> 172.21.15.67:0/432991295 conn(0x7f681c0c63f0 msgr2=0x7f681c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 shutdown_connections 2024-02-26T10:41:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.297+0000 7f6822e00640 1 -- 172.21.15.67:0/432991295 wait complete. 2024-02-26T10:41:37.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-02-26T10:41:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.538+0000 7f8313200640 1 Processor -- start 2024-02-26T10:41:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.538+0000 7f8313200640 1 -- start start 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.539+0000 7f8313200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f830c06ea30 0x7f830c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.539+0000 7f8313200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f830c0c1070 0x7f830c072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.539+0000 7f8313200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ac20 0x7f830c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.539+0000 7f8313200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c151670 con 0x7f830c06ea30 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.539+0000 7f8313200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c06b100 con 0x7f830c0c1070 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.539+0000 7f8313200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c06b3e0 con 0x7f830c06ac20 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.540+0000 7f830be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f830c06ea30 0x7f830c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.540+0000 7f830be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f830c06ea30 0x7f830c06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36758/0 (socket says 172.21.15.67:36758) 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.540+0000 7f830be00640 1 -- 172.21.15.67:0/3862539567 learned_addr learned my addr 172.21.15.67:0/3862539567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.540+0000 7f8311200640 1 --2- 172.21.15.67:0/3862539567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ac20 0x7f830c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.540+0000 7f830b400640 1 --2- 172.21.15.67:0/3862539567 >> v2:172.21.15.82:3300/0 conn(0x7f830c0c1070 0x7f830c072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.541+0000 7f830be00640 1 -- 172.21.15.67:0/3862539567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ac20 msgr2=0x7f830c151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.541+0000 7f830be00640 1 --2- 172.21.15.67:0/3862539567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ac20 0x7f830c151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.541+0000 7f830be00640 1 -- 172.21.15.67:0/3862539567 >> v2:172.21.15.82:3300/0 conn(0x7f830c0c1070 msgr2=0x7f830c072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.541+0000 7f830be00640 1 --2- 172.21.15.67:0/3862539567 >> v2:172.21.15.82:3300/0 conn(0x7f830c0c1070 0x7f830c072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.541+0000 7f830be00640 1 -- 172.21.15.67:0/3862539567 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c06b6c0 con 0x7f830c06ea30 2024-02-26T10:41:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.542+0000 7f830be00640 1 --2- 172.21.15.67:0/3862539567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c06ea30 0x7f830c06dd80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009e50 tx=0x7f82fc02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e01662263ac947f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.542+0000 7f830aa00640 1 -- 172.21.15.67:0/3862539567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc02ec90 con 0x7f830c06ea30 2024-02-26T10:41:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.542+0000 7f830aa00640 1 -- 172.21.15.67:0/3862539567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc02ee30 con 0x7f830c06ea30 2024-02-26T10:41:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.542+0000 7f830aa00640 1 -- 172.21.15.67:0/3862539567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0365f0 con 0x7f830c06ea30 2024-02-26T10:41:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 -- 172.21.15.67:0/3862539567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c06ea30 msgr2=0x7f830c06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 --2- 172.21.15.67:0/3862539567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c06ea30 0x7f830c06dd80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009e50 tx=0x7f82fc02e0f0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 -- 172.21.15.67:0/3862539567 shutdown_connections 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 --2- 172.21.15.67:0/3862539567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ac20 0x7f830c151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 --2- 172.21.15.67:0/3862539567 >> v2:172.21.15.82:3300/0 conn(0x7f830c0c1070 0x7f830c072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 --2- 172.21.15.67:0/3862539567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c06ea30 0x7f830c06dd80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 -- 172.21.15.67:0/3862539567 >> 172.21.15.67:0/3862539567 conn(0x7f830c0a84a0 msgr2=0x7f830c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 -- 172.21.15.67:0/3862539567 shutdown_connections 2024-02-26T10:41:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.543+0000 7f8313200640 1 -- 172.21.15.67:0/3862539567 wait complete. 2024-02-26T10:41:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.545+0000 7f8313200640 1 Processor -- start 2024-02-26T10:41:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.545+0000 7f8313200640 1 -- start start 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8313200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830c06ac20 0x7f830c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8313200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ea30 0x7f830c0c0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8313200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 0x7f830c1ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8313200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1521c0 con 0x7f830c1503c0 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8313200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c152040 con 0x7f830c06ac20 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8313200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c0c3100 con 0x7f830c06ea30 2024-02-26T10:41:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 0x7f830c1ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 0x7f830c1ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36774/0 (socket says 172.21.15.67:36774) 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 -- 172.21.15.67:0/4277095608 learned_addr learned my addr 172.21.15.67:0/4277095608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f830b400640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ea30 0x7f830c0c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f830be00640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830c06ac20 0x7f830c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 -- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ea30 msgr2=0x7f830c0c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ea30 0x7f830c0c0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 -- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830c06ac20 msgr2=0x7f830c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830c06ac20 0x7f830c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f8311200640 1 -- 172.21.15.67:0/4277095608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c153c50 con 0x7f830c1503c0 2024-02-26T10:41:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.546+0000 7f830be00640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830c06ac20 0x7f830c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.547+0000 7f8311200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 0x7f830c1ee430 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8304004650 tx=0x7f830400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.547+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304017070 con 0x7f830c1503c0 2024-02-26T10:41:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.547+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830400c8e0 con 0x7f830c1503c0 2024-02-26T10:41:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.547+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c152680 con 0x7f830c1503c0 2024-02-26T10:41:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.547+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400ccd0 con 0x7f830c1503c0 2024-02-26T10:41:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.547+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c0c0980 con 0x7f830c1503c0 2024-02-26T10:41:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.548+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c074a30 con 0x7f830c1503c0 2024-02-26T10:41:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.550+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f830401a070 con 0x7f830c1503c0 2024-02-26T10:41:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.550+0000 7f8303e00640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f82e806b4d0 0x7f82e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.551+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f830408a1d0 con 0x7f830c1503c0 2024-02-26T10:41:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.551+0000 7f830be00640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f82e806b4d0 0x7f82e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.552+0000 7f830be00640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f82e806b4d0 0x7f82e806d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f82fc037000 tx=0x7f82fc032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.555+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8304058fa0 con 0x7f830c1503c0 2024-02-26T10:41:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.791+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f830c152920 con 0x7f830c1503c0 2024-02-26T10:41:37.791 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:41:37.793+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:41:37.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.796+0000 7f8303e00640 1 -- 172.21.15.67:0/4277095608 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 163+0+0 (secure 0 0 0) 0x7f83040588a0 con 0x7f830c1503c0 2024-02-26T10:41:37.795 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f82e806b4d0 msgr2=0x7f82e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f82e806b4d0 0x7f82e806d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f82fc037000 tx=0x7f82fc032000 comp rx=0 tx=0).stop 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 msgr2=0x7f830c1ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 0x7f830c1ee430 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8304004650 tx=0x7f830400ece0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 shutdown_connections 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f830c1503c0 0x7f830c1ee430 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f830c06ea30 0x7f830c0c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f82e806b4d0 0x7f82e806d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 --2- 172.21.15.67:0/4277095608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830c06ac20 0x7f830c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 >> 172.21.15.67:0/4277095608 conn(0x7f830c0a84a0 msgr2=0x7f830c14e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 shutdown_connections 2024-02-26T10:41:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:37.799+0000 7f8313200640 1 -- 172.21.15.67:0/4277095608 wait complete. 2024-02-26T10:41:37.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-02-26T10:41:37.822 INFO:tasks.ceph.mds.d.smithi082.stderr: -2444> 2024-02-26T10:41:04.924+0000 7fc2b8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:41:37.912 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-02-26T10:41:38.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.042+0000 7f4a8ce00640 1 Processor -- start 2024-02-26T10:41:38.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.042+0000 7f4a8ce00640 1 -- start start 2024-02-26T10:41:38.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.043+0000 7f4a8ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a88152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a8ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a8806e4f0 0x7f4a8806e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a8ce00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a880c34d0 con 0x7f4a88072da0 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a8ce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a881530f0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a8ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88157150 con 0x7f4a88150580 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a8806e4f0 0x7f4a8806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a8806e4f0 0x7f4a8806e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36778/0 (socket says 172.21.15.67:36778) 2024-02-26T10:41:38.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a87e00640 1 -- 172.21.15.67:0/3104409265 learned_addr learned my addr 172.21.15.67:0/3104409265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a87e00640 1 -- 172.21.15.67:0/3104409265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 msgr2=0x7f4a88152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.044+0000 7f4a87400640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a88152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.045+0000 7f4a87e00640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a88152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.045+0000 7f4a87e00640 1 -- 172.21.15.67:0/3104409265 >> v1:172.21.15.82:6789/0 conn(0x7f4a88072da0 legacy=0x7f4a88156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.045+0000 7f4a87e00640 1 -- 172.21.15.67:0/3104409265 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a880c37b0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.045+0000 7f4a87400640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a88152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:38.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.046+0000 7f4a87e00640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a8806e960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009e50 tx=0x7f4a8002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c0abd7e9a38d48 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.046+0000 7f4a86000640 1 -- 172.21.15.67:0/3104409265 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8002c9f0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.046+0000 7f4a86000640 1 -- 172.21.15.67:0/3104409265 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8002cb90 con 0x7f4a8806e4f0 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.046+0000 7f4a86000640 1 -- 172.21.15.67:0/3104409265 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800343f0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 -- 172.21.15.67:0/3104409265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 msgr2=0x7f4a8806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a8806e960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009e50 tx=0x7f4a8002be50 comp rx=0 tx=0).stop 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 -- 172.21.15.67:0/3104409265 shutdown_connections 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a8806e960 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/3104409265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a88152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 -- 172.21.15.67:0/3104409265 >> 172.21.15.67:0/3104409265 conn(0x7f4a8806cd20 msgr2=0x7f4a8806d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 -- 172.21.15.67:0/3104409265 shutdown_connections 2024-02-26T10:41:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.047+0000 7f4a8ce00640 1 -- 172.21.15.67:0/3104409265 wait complete. 2024-02-26T10:41:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.049+0000 7f4a8ce00640 1 Processor -- start 2024-02-26T10:41:38.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.049+0000 7f4a8ce00640 1 -- start start 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a8ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a88140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a8ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a88072da0 0x7f4a88140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a8ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a881e9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a8ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88156520 con 0x7f4a8806e4f0 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a8ce00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88156240 con 0x7f4a88072da0 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a8ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88073200 con 0x7f4a88150580 2024-02-26T10:41:38.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a88140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a88140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36786/0 (socket says 172.21.15.67:36786) 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a87400640 1 -- 172.21.15.67:0/1371716612 learned_addr learned my addr 172.21.15.67:0/1371716612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a86a00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a88072da0 0x7f4a88140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a87e00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a881e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.050+0000 7f4a87400640 1 -- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 msgr2=0x7f4a881e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a87400640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a881e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a87400640 1 -- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a88072da0 msgr2=0x7f4a88140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a87400640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a88072da0 0x7f4a88140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a87400640 1 -- 172.21.15.67:0/1371716612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a881e7240 con 0x7f4a8806e4f0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a87400640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a88140640 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c7f0 tx=0x7f4a7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78015070 con 0x7f4a8806e4f0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a78007a30 con 0x7f4a8806e4f0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78007e00 con 0x7f4a8806e4f0 2024-02-26T10:41:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a880aab40 con 0x7f4a8806e4f0 2024-02-26T10:41:38.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.051+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a880ab3e0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.053+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a880aad10 con 0x7f4a8806e4f0 2024-02-26T10:41:38.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.054+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a78018070 con 0x7f4a8806e4f0 2024-02-26T10:41:38.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.054+0000 7f4a7f400640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6806b4d0 0x7f4a6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:38.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.055+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f4a7808a1f0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.055+0000 7f4a86a00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6806b4d0 0x7f4a6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.056+0000 7f4a86a00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6806b4d0 0x7f4a6806d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006fd0 tx=0x7f4a74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.059+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a78058ff0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.297+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4a88074a30 con 0x7f4a8806e4f0 2024-02-26T10:41:38.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.799+0000 7f4a7f400640 1 -- 172.21.15.67:0/1371716612 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4a780588f0 con 0x7f4a8806e4f0 2024-02-26T10:41:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6806b4d0 msgr2=0x7f4a6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6806b4d0 0x7f4a6806d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006fd0 tx=0x7f4a74008040 comp rx=0 tx=0).stop 2024-02-26T10:41:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 msgr2=0x7f4a88140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a88140640 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800c7f0 tx=0x7f4a7800cbf0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 shutdown_connections 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a88150580 0x7f4a881e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4a6806b4d0 0x7f4a6806d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a88072da0 0x7f4a88140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 --2- 172.21.15.67:0/1371716612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a8806e4f0 0x7f4a88140640 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.803+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 >> 172.21.15.67:0/1371716612 conn(0x7f4a8806cd20 msgr2=0x7f4a880c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.804+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 shutdown_connections 2024-02-26T10:41:38.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:38.804+0000 7f4a8ce00640 1 -- 172.21.15.67:0/1371716612 wait complete. 2024-02-26T10:41:38.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-02-26T10:41:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.056+0000 7f94c5800640 1 Processor -- start 2024-02-26T10:41:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.057+0000 7f94c5800640 1 -- start start 2024-02-26T10:41:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.058+0000 7f94c5800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94c014cc80 0x7f94c0153650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.058+0000 7f94c5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 0x7f94c00ab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.058+0000 7f94c5800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c0153b90 con 0x7f94c006dd80 2024-02-26T10:41:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.058+0000 7f94c5800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c014d060 con 0x7f94c014cc80 2024-02-26T10:41:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.058+0000 7f94c5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01562f0 con 0x7f94c00aabc0 2024-02-26T10:41:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.059+0000 7f94b7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f94c006dd80 0x7f94c00c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56340/0 (socket says 172.21.15.67:56340) 2024-02-26T10:41:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.059+0000 7f94b7e00640 1 -- 172.21.15.67:0/3997217847 learned_addr learned my addr 172.21.15.67:0/3997217847 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.059+0000 7f94bf400640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 0x7f94c00ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.059+0000 7f94bea00640 1 --2- 172.21.15.67:0/3997217847 >> v2:172.21.15.82:3300/0 conn(0x7f94c014cc80 0x7f94c0153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.059+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004354028 0 0) 0x7f94a8001110 con 0x7f94c006dd80 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.059+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a4003520 con 0x7f94c006dd80 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215092777 0 0) 0x7f94a4003520 con 0x7f94c006dd80 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 msgr2=0x7f94c00ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 0x7f94c00ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 >> v2:172.21.15.82:3300/0 conn(0x7f94c014cc80 msgr2=0x7f94c0153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 --2- 172.21.15.67:0/3997217847 >> v2:172.21.15.82:3300/0 conn(0x7f94c014cc80 0x7f94c0153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c01565d0 con 0x7f94c006dd80 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f94a8001840 con 0x7f94c006dd80 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 >> v1:172.21.15.67:6789/0 conn(0x7f94c006dd80 legacy=0x7f94c00c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a40039b0 0x7f94a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a4004800 0x7f94a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 0x7f94a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c014d060 con 0x7f94a4004800 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01562f0 con 0x7f94a40039b0 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a400baf0 con 0x7f94c00aabc0 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.060+0000 7f94b7e00640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a4004800 0x7f94a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.061+0000 7f94bea00640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a40039b0 0x7f94a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.061+0000 7f94b7e00640 1 -- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 msgr2=0x7f94a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.061+0000 7f94b7e00640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00aabc0 0x7f94a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.061+0000 7f94b7e00640 1 -- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a40039b0 msgr2=0x7f94a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.061+0000 7f94b7e00640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a40039b0 0x7f94a4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.061+0000 7f94b7e00640 1 -- 172.21.15.67:0/3997217847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4003520 con 0x7f94a4004800 2024-02-26T10:41:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.062+0000 7f94b7e00640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a4004800 0x7f94a4004c80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f94a800c2f0 tx=0x7f94a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed63431a6a224be server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.062+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a802cbe0 con 0x7f94a4004800 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.062+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a8030070 con 0x7f94a4004800 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.062+0000 7f94be000640 1 -- 172.21.15.67:0/3997217847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80058d0 con 0x7f94a4004800 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94c5800640 1 -- 172.21.15.67:0/3997217847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a4004800 msgr2=0x7f94a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94c5800640 1 --2- 172.21.15.67:0/3997217847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a4004800 0x7f94a4004c80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f94a800c2f0 tx=0x7f94a4005830 comp rx=0 tx=0).stop 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94bf400640 1 -- 172.21.15.67:0/3997217847 reap_dead start 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94c5800640 1 -- 172.21.15.67:0/3997217847 shutdown_connections 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94c5800640 1 -- 172.21.15.67:0/3997217847 >> 172.21.15.67:0/3997217847 conn(0x7f94c00a84a0 msgr2=0x7f94c00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94c5800640 1 -- 172.21.15.67:0/3997217847 shutdown_connections 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.063+0000 7f94c5800640 1 -- 172.21.15.67:0/3997217847 wait complete. 2024-02-26T10:41:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.065+0000 7f94c5800640 1 Processor -- start 2024-02-26T10:41:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.065+0000 7f94c5800640 1 -- start start 2024-02-26T10:41:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94c5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a4004800 0x7f94c014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94c5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 0x7f94c00c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94c5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00c4a10 0x7f94c01429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94c5800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00c37a0 con 0x7f94c014bf10 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94c5800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0156b90 con 0x7f94a4004800 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94c5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00c3920 con 0x7f94c00c4a10 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 0x7f94c00c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 0x7f94c00c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36794/0 (socket says 172.21.15.67:36794) 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.066+0000 7f94b7e00640 1 -- 172.21.15.67:0/212403678 learned_addr learned my addr 172.21.15.67:0/212403678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94bf400640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00c4a10 0x7f94c01429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94bea00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a4004800 0x7f94c014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94b7e00640 1 -- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00c4a10 msgr2=0x7f94c01429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94b7e00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00c4a10 0x7f94c01429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94b7e00640 1 -- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a4004800 msgr2=0x7f94c014b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94b7e00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a4004800 0x7f94c014b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94b7e00640 1 -- 172.21.15.67:0/212403678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0152b80 con 0x7f94c014bf10 2024-02-26T10:41:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94bea00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a4004800 0x7f94c014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.067+0000 7f94b7e00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 0x7f94c00c44d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f94a802c650 tx=0x7f94c00aa450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.068+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8045070 con 0x7f94c014bf10 2024-02-26T10:41:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.068+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a8007e50 con 0x7f94c014bf10 2024-02-26T10:41:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.068+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c01eadc0 con 0x7f94c014bf10 2024-02-26T10:41:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.068+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8004e30 con 0x7f94c014bf10 2024-02-26T10:41:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.069+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c01eb380 con 0x7f94c014bf10 2024-02-26T10:41:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.070+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c0074a30 con 0x7f94c014bf10 2024-02-26T10:41:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.071+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94a8007070 con 0x7f94c014bf10 2024-02-26T10:41:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.071+0000 7f94b6a00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949c06b4d0 0x7f949c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.072+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5943+0+0 (secure 0 0 0) 0x7f94a80ab2d0 con 0x7f94c014bf10 2024-02-26T10:41:39.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.072+0000 7f94bea00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949c06b4d0 0x7f949c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.073+0000 7f94bea00640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949c06b4d0 0x7f949c06d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f94b00099e0 tx=0x7f94b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.076+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94c0152b80 con 0x7f94c014bf10 2024-02-26T10:41:39.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:39.274+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94c00c1120 con 0x7f94c014bf10 2024-02-26T10:41:40.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.080+0000 7f94b6a00640 1 -- 172.21.15.67:0/212403678 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v66) v1 ==== 184+0+0 (secure 0 0 0) 0x7f94c0153b90 con 0x7f94c014bf10 2024-02-26T10:41:40.079 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_metadata' removed 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949c06b4d0 msgr2=0x7f949c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949c06b4d0 0x7f949c06d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f94b00099e0 tx=0x7f94b0008040 comp rx=0 tx=0).stop 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 msgr2=0x7f94c00c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 0x7f94c00c44d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f94a802c650 tx=0x7f94c00aa450 comp rx=0 tx=0).stop 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 shutdown_connections 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c00c4a10 0x7f94c01429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94c014bf10 0x7f94c00c44d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949c06b4d0 0x7f949c06d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 --2- 172.21.15.67:0/212403678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a4004800 0x7f94c014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 >> 172.21.15.67:0/212403678 conn(0x7f94c00a84a0 msgr2=0x7f94c00c1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:40.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.084+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 shutdown_connections 2024-02-26T10:41:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.085+0000 7f94c5800640 1 -- 172.21.15.67:0/212403678 wait complete. 2024-02-26T10:41:40.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-02-26T10:41:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.311+0000 7ffb73200640 1 Processor -- start 2024-02-26T10:41:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.311+0000 7ffb73200640 1 -- start start 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb73200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb741503a0 0x7ffb740ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb73200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb7406e2d0 0x7ffb740c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb73200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb74150780 con 0x7ffb740a9430 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb73200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb740a98c0 con 0x7ffb741503a0 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb73200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb740a9ba0 con 0x7ffb7406e2d0 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb71e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb741503a0 0x7ffb740ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb71e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb741503a0 0x7ffb740ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51960/0 (socket says 172.21.15.67:51960) 2024-02-26T10:41:40.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.313+0000 7ffb71e00640 1 -- 172.21.15.67:0/1213902342 learned_addr learned my addr 172.21.15.67:0/1213902342 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.314+0000 7ffb71e00640 1 -- 172.21.15.67:0/1213902342 >> v2:172.21.15.82:3300/0 conn(0x7ffb7406e2d0 msgr2=0x7ffb740c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.314+0000 7ffb71e00640 1 --2- 172.21.15.67:0/1213902342 >> v2:172.21.15.82:3300/0 conn(0x7ffb7406e2d0 0x7ffb740c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.314+0000 7ffb71e00640 1 -- 172.21.15.67:0/1213902342 >> v1:172.21.15.67:6789/0 conn(0x7ffb740a9430 legacy=0x7ffb741535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.314+0000 7ffb71e00640 1 -- 172.21.15.67:0/1213902342 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb740a9e80 con 0x7ffb741503a0 2024-02-26T10:41:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.315+0000 7ffb71e00640 1 --2- 172.21.15.67:0/1213902342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb740ab480 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ffb740c3f40 tx=0x7ffb6402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68cb189a3886cde server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.315+0000 7ffb70a00640 1 -- 172.21.15.67:0/1213902342 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64034070 con 0x7ffb741503a0 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.315+0000 7ffb70a00640 1 -- 172.21.15.67:0/1213902342 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64030870 con 0x7ffb741503a0 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.315+0000 7ffb70a00640 1 -- 172.21.15.67:0/1213902342 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64030c40 con 0x7ffb741503a0 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 -- 172.21.15.67:0/1213902342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 msgr2=0x7ffb740ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 --2- 172.21.15.67:0/1213902342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb740ab480 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ffb740c3f40 tx=0x7ffb6402b9b0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 -- 172.21.15.67:0/1213902342 shutdown_connections 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 --2- 172.21.15.67:0/1213902342 >> v2:172.21.15.82:3300/0 conn(0x7ffb7406e2d0 0x7ffb740c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 --2- 172.21.15.67:0/1213902342 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb740ab480 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 -- 172.21.15.67:0/1213902342 >> 172.21.15.67:0/1213902342 conn(0x7ffb740c6380 msgr2=0x7ffb740c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 -- 172.21.15.67:0/1213902342 shutdown_connections 2024-02-26T10:41:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.316+0000 7ffb73200640 1 -- 172.21.15.67:0/1213902342 wait complete. 2024-02-26T10:41:40.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.318+0000 7ffb73200640 1 Processor -- start 2024-02-26T10:41:40.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.318+0000 7ffb73200640 1 -- start start 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb73200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb7406e2d0 0x7ffb7414a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb73200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb74149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb73200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb7414dd50 0x7ffb7414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb73200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb740c0f80 con 0x7ffb741503a0 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb73200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7406ddb0 con 0x7ffb7414dd50 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb73200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7406e740 con 0x7ffb7406e2d0 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb71400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb74149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb71400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb74149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51964/0 (socket says 172.21.15.67:51964) 2024-02-26T10:41:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.319+0000 7ffb71400640 1 -- 172.21.15.67:0/3140026549 learned_addr learned my addr 172.21.15.67:0/3140026549 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb72800640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb7414dd50 0x7ffb7414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71e00640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb7406e2d0 0x7ffb7414a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71400640 1 -- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb7406e2d0 msgr2=0x7ffb7414a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71400640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb7406e2d0 0x7ffb7414a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71400640 1 -- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb7414dd50 msgr2=0x7ffb7414e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71400640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb7414dd50 0x7ffb7414e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71400640 1 -- 172.21.15.67:0/3140026549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74149ea0 con 0x7ffb741503a0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb71400640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb74149960 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c470 tx=0x7ffb60001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600052d0 con 0x7ffb741503a0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60005470 con 0x7ffb741503a0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60007260 con 0x7ffb741503a0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.320+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb741ebce0 con 0x7ffb741503a0 2024-02-26T10:41:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.321+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb741eb9b0 con 0x7ffb741503a0 2024-02-26T10:41:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.322+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb740678e0 con 0x7ffb741503a0 2024-02-26T10:41:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.323+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb60007400 con 0x7ffb741503a0 2024-02-26T10:41:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.324+0000 7ffb6a000640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5406b4d0 0x7ffb5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.324+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5503+0+0 (secure 0 0 0) 0x7ffb60089650 con 0x7ffb741503a0 2024-02-26T10:41:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.324+0000 7ffb71e00640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5406b4d0 0x7ffb5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.325+0000 7ffb71e00640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5406b4d0 0x7ffb5406d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffb64036000 tx=0x7ffb6403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:40.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.328+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb60058590 con 0x7ffb741503a0 2024-02-26T10:41:40.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:40.533+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb7404f810 con 0x7ffb741503a0 2024-02-26T10:41:41.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.107+0000 7ffb6a000640 1 -- 172.21.15.67:0/3140026549 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v67) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffb7404f810 con 0x7ffb741503a0 2024-02-26T10:41:41.107 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_data' removed 2024-02-26T10:41:41.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.111+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5406b4d0 msgr2=0x7ffb5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.111+0000 7ffb73200640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5406b4d0 0x7ffb5406d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffb64036000 tx=0x7ffb6403b040 comp rx=0 tx=0).stop 2024-02-26T10:41:41.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.111+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 msgr2=0x7ffb74149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.111+0000 7ffb73200640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb74149960 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c470 tx=0x7ffb60001e70 comp rx=0 tx=0).stop 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 shutdown_connections 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffb5406b4d0 0x7ffb5406d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb7414dd50 0x7ffb7414e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb741503a0 0x7ffb74149960 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 --2- 172.21.15.67:0/3140026549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb7406e2d0 0x7ffb7414a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 >> 172.21.15.67:0/3140026549 conn(0x7ffb740c6380 msgr2=0x7ffb740c2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 shutdown_connections 2024-02-26T10:41:41.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.112+0000 7ffb73200640 1 -- 172.21.15.67:0/3140026549 wait complete. 2024-02-26T10:41:41.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-26T10:41:41.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.349+0000 7f37ba600640 1 Processor -- start 2024-02-26T10:41:41.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.350+0000 7f37ba600640 1 -- start start 2024-02-26T10:41:41.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.351+0000 7f37ba600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37b41525b0 0x7f37b4155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.351+0000 7f37ba600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b406c850 con 0x7f37b40c5330 2024-02-26T10:41:41.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.351+0000 7f37ba600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b4155c90 con 0x7f37b406bb80 2024-02-26T10:41:41.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.351+0000 7f37ba600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b41552c0 con 0x7f37b41525b0 2024-02-26T10:41:41.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.351+0000 7f37b3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f37b40c5330 0x7f37b40c8520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54748/0 (socket says 172.21.15.67:54748) 2024-02-26T10:41:41.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.351+0000 7f37b3e00640 1 -- 172.21.15.67:0/2120809240 learned_addr learned my addr 172.21.15.67:0/2120809240 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:41.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515019104 0 0) 0x7f37b406c850 con 0x7f37b40c5330 2024-02-26T10:41:41.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3798003520 con 0x7f37b40c5330 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b9200640 1 --2- 172.21.15.67:0/2120809240 >> v2:172.21.15.82:3300/0 conn(0x7f37b41525b0 0x7f37b4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143259787 0 0) 0x7f3798003520 con 0x7f37b40c5330 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 >> v2:172.21.15.82:3300/0 conn(0x7f37b41525b0 msgr2=0x7f37b4155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 --2- 172.21.15.67:0/2120809240 >> v2:172.21.15.82:3300/0 conn(0x7f37b41525b0 0x7f37b4155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 >> v1:172.21.15.82:6789/0 conn(0x7f37b406bb80 legacy=0x7f37b40c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b406c290 con 0x7f37b40c5330 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f379c001670 con 0x7f37b40c5330 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 >> v1:172.21.15.67:6789/0 conn(0x7f37b40c5330 legacy=0x7f37b40c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980039b0 0x7f3798003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3798004800 0x7f3798004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37980053c0 0x7f379800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41552c0 con 0x7f37980053c0 2024-02-26T10:41:41.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379800c250 con 0x7f3798004800 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.352+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379800c450 con 0x7f37980039b0 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9c00640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37980053c0 0x7f379800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9200640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980039b0 0x7f3798003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9c00640 1 -- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980039b0 msgr2=0x7f3798003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9c00640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980039b0 0x7f3798003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9c00640 1 -- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3798004800 msgr2=0x7f3798004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b3e00640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3798004800 0x7f3798004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9c00640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3798004800 0x7f3798004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9c00640 1 -- 172.21.15.67:0/2120809240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798003520 con 0x7f37980053c0 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b9200640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980039b0 0x7f3798003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.353+0000 7f37b3e00640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3798004800 0x7f3798004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:41.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.354+0000 7f37b9c00640 1 -- 172.21.15.67:0/2120809240 reap_dead start 2024-02-26T10:41:41.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.354+0000 7f37b9c00640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37980053c0 0x7f379800b9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f37ac001750 tx=0x7f3798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad3fb851b3baa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac001370 con 0x7f37980053c0 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac005310 con 0x7f37980053c0 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37b3400640 1 -- 172.21.15.67:0/2120809240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac003830 con 0x7f37980053c0 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 -- 172.21.15.67:0/2120809240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37980053c0 msgr2=0x7f379800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37980053c0 0x7f379800b9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f37ac001750 tx=0x7f3798005830 comp rx=0 tx=0).stop 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 -- 172.21.15.67:0/2120809240 shutdown_connections 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 --2- 172.21.15.67:0/2120809240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37980053c0 0x7f379800b9c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 -- 172.21.15.67:0/2120809240 >> 172.21.15.67:0/2120809240 conn(0x7f37b40c70a0 msgr2=0x7f37b40c74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 -- 172.21.15.67:0/2120809240 shutdown_connections 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.355+0000 7f37ba600640 1 -- 172.21.15.67:0/2120809240 wait complete. 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.357+0000 7f37ba600640 1 Processor -- start 2024-02-26T10:41:41.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.357+0000 7f37ba600640 1 -- start start 2024-02-26T10:41:41.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37ba600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980053c0 0x7f37b4158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37ba600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 0x7f37b40af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37ba600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37b40afcb0 0x7f37b40c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37ba600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b406b670 con 0x7f37b40c3d00 2024-02-26T10:41:41.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37ba600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4154f50 con 0x7f37b40afcb0 2024-02-26T10:41:41.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37ba600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b406b7f0 con 0x7f37980053c0 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37b3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 0x7f37b40af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37b3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 0x7f37b40af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51986/0 (socket says 172.21.15.67:51986) 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.358+0000 7f37b3e00640 1 -- 172.21.15.67:0/1177573377 learned_addr learned my addr 172.21.15.67:0/1177573377 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b9200640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980053c0 0x7f37b4158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b9c00640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37b40afcb0 0x7f37b40c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b3e00640 1 -- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980053c0 msgr2=0x7f37b4158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b3e00640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980053c0 0x7f37b4158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b3e00640 1 -- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37b40afcb0 msgr2=0x7f37b40c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b3e00640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37b40afcb0 0x7f37b40c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b3e00640 1 -- 172.21.15.67:0/1177573377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40c4840 con 0x7f37b40c3d00 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b3e00640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 0x7f37b40af770 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f379c00d720 tx=0x7f379c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c019070 con 0x7f37b40c3d00 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c0072d0 con 0x7f37b40c3d00 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0047e0 con 0x7f37b40c3d00 2024-02-26T10:41:41.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.359+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b41eb4a0 con 0x7f37b40c3d00 2024-02-26T10:41:41.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.360+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b40c1890 con 0x7f37b40c3d00 2024-02-26T10:41:41.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.361+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b406ac20 con 0x7f37b40c3d00 2024-02-26T10:41:41.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.362+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f379c01c070 con 0x7f37b40c3d00 2024-02-26T10:41:41.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.362+0000 7f37b0c00640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f379006b4d0 0x7f379006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.363+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f379c089910 con 0x7f37b40c3d00 2024-02-26T10:41:41.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.363+0000 7f37b9200640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f379006b4d0 0x7f379006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.364+0000 7f37b9200640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f379006b4d0 0x7f379006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fd0 tx=0x7f37a400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:41.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.367+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37b406ac20 con 0x7f37b40c3d00 2024-02-26T10:41:41.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.570+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f37b4074a30 con 0x7f37b40c3d00 2024-02-26T10:41:41.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.577+0000 7f37b0c00640 1 -- 172.21.15.67:0/1177573377 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f379c058a20 con 0x7f37b40c3d00 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.579+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f379006b4d0 msgr2=0x7f379006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.579+0000 7f37ba600640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f379006b4d0 0x7f379006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fd0 tx=0x7f37a400b040 comp rx=0 tx=0).stop 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 msgr2=0x7f37b40af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 0x7f37b40af770 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f379c00d720 tx=0x7f379c00dbe0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 shutdown_connections 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f379006b4d0 0x7f379006d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37b40afcb0 0x7f37b40c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37b40c3d00 0x7f37b40af770 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 --2- 172.21.15.67:0/1177573377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37980053c0 0x7f37b4158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 >> 172.21.15.67:0/1177573377 conn(0x7f37b40c70a0 msgr2=0x7f37b40aaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 shutdown_connections 2024-02-26T10:41:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.580+0000 7f37ba600640 1 -- 172.21.15.67:0/1177573377 wait complete. 2024-02-26T10:41:41.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent' 2024-02-26T10:41:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.819+0000 7f8e2de00640 1 Processor -- start 2024-02-26T10:41:41.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.820+0000 7f8e2de00640 1 -- start start 2024-02-26T10:41:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.821+0000 7f8e2de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e2806ac20 0x7f8e280a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.822+0000 7f8e2de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2806b460 con 0x7f8e2806b000 2024-02-26T10:41:41.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.822+0000 7f8e2de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2806df90 con 0x7f8e2806db00 2024-02-26T10:41:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.822+0000 7f8e2de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e280a9a80 con 0x7f8e2806ac20 2024-02-26T10:41:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.822+0000 7f8e2ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e2806ac20 0x7f8e280a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.822+0000 7f8e27e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8e2806b000 0x7f8e28153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54764/0 (socket says 172.21.15.67:54764) 2024-02-26T10:41:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.822+0000 7f8e27e00640 1 -- 172.21.15.67:0/520304217 learned_addr learned my addr 172.21.15.67:0/520304217 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.823+0000 7f8e27400640 1 -- 172.21.15.67:0/520304217 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153660505 0 0) 0x7f8e2806b460 con 0x7f8e2806b000 2024-02-26T10:41:41.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.823+0000 7f8e27400640 1 -- 172.21.15.67:0/520304217 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e0c003520 con 0x7f8e2806b000 2024-02-26T10:41:41.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.823+0000 7f8e2ca00640 1 -- 172.21.15.67:0/520304217 >> v1:172.21.15.82:6789/0 conn(0x7f8e2806db00 legacy=0x7f8e280abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.823+0000 7f8e2ca00640 1 -- 172.21.15.67:0/520304217 >> v1:172.21.15.67:6789/0 conn(0x7f8e2806b000 legacy=0x7f8e28153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.823+0000 7f8e2ca00640 1 -- 172.21.15.67:0/520304217 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280a9d60 con 0x7f8e2806ac20 2024-02-26T10:41:41.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.824+0000 7f8e2ca00640 1 --2- 172.21.15.67:0/520304217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806ac20 0x7f8e280a94a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8e180097a0 tx=0x7f8e1802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7904c3146e9a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:41.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.824+0000 7f8e27400640 1 -- 172.21.15.67:0/520304217 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18034070 con 0x7f8e2806ac20 2024-02-26T10:41:41.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.824+0000 7f8e27400640 1 -- 172.21.15.67:0/520304217 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e18030610 con 0x7f8e2806ac20 2024-02-26T10:41:41.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.824+0000 7f8e27400640 1 -- 172.21.15.67:0/520304217 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180309e0 con 0x7f8e2806ac20 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 -- 172.21.15.67:0/520304217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806ac20 msgr2=0x7f8e280a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 --2- 172.21.15.67:0/520304217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806ac20 0x7f8e280a94a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8e180097a0 tx=0x7f8e1802b960 comp rx=0 tx=0).stop 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 -- 172.21.15.67:0/520304217 shutdown_connections 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 --2- 172.21.15.67:0/520304217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806ac20 0x7f8e280a94a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 -- 172.21.15.67:0/520304217 >> 172.21.15.67:0/520304217 conn(0x7f8e280c63f0 msgr2=0x7f8e280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 -- 172.21.15.67:0/520304217 shutdown_connections 2024-02-26T10:41:41.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.825+0000 7f8e2de00640 1 -- 172.21.15.67:0/520304217 wait complete. 2024-02-26T10:41:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.827+0000 7f8e2de00640 1 Processor -- start 2024-02-26T10:41:41.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.827+0000 7f8e2de00640 1 -- start start 2024-02-26T10:41:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e2806ac20 0x7f8e280bf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 0x7f8e28140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e2de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e2806db00 0x7f8e281ee040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e2de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28150410 con 0x7f8e2806b000 2024-02-26T10:41:41.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e2de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280aba30 con 0x7f8e2806db00 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e2de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28150590 con 0x7f8e2806ac20 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 0x7f8e28140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 0x7f8e28140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51996/0 (socket says 172.21.15.67:51996) 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.828+0000 7f8e27e00640 1 -- 172.21.15.67:0/2098777519 learned_addr learned my addr 172.21.15.67:0/2098777519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e2ca00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e2806ac20 0x7f8e280bf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e27e00640 1 -- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e2806ac20 msgr2=0x7f8e280bf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e2d400640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e2806db00 0x7f8e281ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e27e00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e2806ac20 0x7f8e280bf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e27e00640 1 -- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e2806db00 msgr2=0x7f8e281ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e27e00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e2806db00 0x7f8e281ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e27e00640 1 -- 172.21.15.67:0/2098777519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e281ea680 con 0x7f8e2806b000 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e2ca00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e2806ac20 0x7f8e280bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e2d400640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e2806db00 0x7f8e281ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e27e00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 0x7f8e28140c20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c4d0 tx=0x7f8e1400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:41.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14017070 con 0x7f8e2806b000 2024-02-26T10:41:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e14005ce0 con 0x7f8e2806b000 2024-02-26T10:41:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14007270 con 0x7f8e2806b000 2024-02-26T10:41:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.829+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e28072b80 con 0x7f8e2806b000 2024-02-26T10:41:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.830+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e281eb5f0 con 0x7f8e2806b000 2024-02-26T10:41:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.831+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e28074a30 con 0x7f8e2806b000 2024-02-26T10:41:41.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.832+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e1401a070 con 0x7f8e2806b000 2024-02-26T10:41:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.832+0000 7f8e24c00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e0806b4d0 0x7f8e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.833+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8e14089210 con 0x7f8e2806b000 2024-02-26T10:41:41.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.833+0000 7f8e2ca00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e0806b4d0 0x7f8e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:41.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.834+0000 7f8e2ca00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e0806b4d0 0x7f8e0806d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e18036000 tx=0x7f8e1803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:41.837+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e281ea680 con 0x7f8e2806b000 2024-02-26T10:41:42.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:42.110+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]} v 0) v1 -- 0x7f8e280c0df0 con 0x7f8e2806b000 2024-02-26T10:41:43.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.121+0000 7f8e24c00640 1 -- 172.21.15.67:0/2098777519 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]}]=0 v197) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8e14057cd0 con 0x7f8e2806b000 2024-02-26T10:41:43.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.126+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e0806b4d0 msgr2=0x7f8e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.126+0000 7f8e2de00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e0806b4d0 0x7f8e0806d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e18036000 tx=0x7f8e1803b040 comp rx=0 tx=0).stop 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.126+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 msgr2=0x7f8e28140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.126+0000 7f8e2de00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 0x7f8e28140c20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c4d0 tx=0x7f8e1400c990 comp rx=0 tx=0).stop 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 shutdown_connections 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e0806b4d0 0x7f8e0806d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e2806db00 0x7f8e281ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e2806b000 0x7f8e28140c20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 --2- 172.21.15.67:0/2098777519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e2806ac20 0x7f8e280bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 >> 172.21.15.67:0/2098777519 conn(0x7f8e280c63f0 msgr2=0x7f8e2814c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 shutdown_connections 2024-02-26T10:41:43.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.127+0000 7f8e2de00640 1 -- 172.21.15.67:0/2098777519 wait complete. 2024-02-26T10:41:43.148 INFO:tasks.cephfs_test_runner:test_mount_mon_and_osd_caps_present_mds_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) ... ok 2024-02-26T10:41:43.148 INFO:tasks.cephfs_test_runner:Starting test: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-02-26T10:41:43.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:43.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.377+0000 7fd435000640 1 Processor -- start 2024-02-26T10:41:43.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.377+0000 7fd435000640 1 -- start start 2024-02-26T10:41:43.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.378+0000 7fd435000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4300c2310 0x7fd430153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.378+0000 7fd435000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd43006b000 0x7fd4301524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.378+0000 7fd435000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd430155f10 con 0x7fd43006ac20 2024-02-26T10:41:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.378+0000 7fd435000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4300c28d0 con 0x7fd4300c2310 2024-02-26T10:41:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd435000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43006b430 con 0x7fd43006b000 2024-02-26T10:41:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4300c2310 0x7fd430153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd43006ac20 0x7fd430152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54768/0 (socket says 172.21.15.67:54768) 2024-02-26T10:41:43.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42f400640 1 -- 172.21.15.67:0/3960765543 learned_addr learned my addr 172.21.15.67:0/3960765543 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42fe00640 1 --2- 172.21.15.67:0/3960765543 >> v2:172.21.15.82:3300/0 conn(0x7fd43006b000 0x7fd4301524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42ea00640 1 -- 172.21.15.67:0/3960765543 >> v2:172.21.15.82:3300/0 conn(0x7fd43006b000 msgr2=0x7fd4301524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:43.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42ea00640 1 --2- 172.21.15.67:0/3960765543 >> v2:172.21.15.82:3300/0 conn(0x7fd43006b000 0x7fd4301524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42ea00640 1 -- 172.21.15.67:0/3960765543 >> v1:172.21.15.67:6789/0 conn(0x7fd43006ac20 legacy=0x7fd430152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42ea00640 1 -- 172.21.15.67:0/3960765543 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd430155150 con 0x7fd4300c2310 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.379+0000 7fd42fe00640 1 --2- 172.21.15.67:0/3960765543 >> v2:172.21.15.82:3300/0 conn(0x7fd43006b000 0x7fd4301524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.380+0000 7fd42ea00640 1 --2- 172.21.15.67:0/3960765543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4300c2310 0x7fd430153630 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd4300c2c60 tx=0x7fd41802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16709c8b1ca8b9d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.381+0000 7fd42e000640 1 -- 172.21.15.67:0/3960765543 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418038070 con 0x7fd4300c2310 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.381+0000 7fd42e000640 1 -- 172.21.15.67:0/3960765543 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd418031520 con 0x7fd4300c2310 2024-02-26T10:41:43.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.381+0000 7fd42e000640 1 -- 172.21.15.67:0/3960765543 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4180318f0 con 0x7fd4300c2310 2024-02-26T10:41:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.381+0000 7fd435000640 1 -- 172.21.15.67:0/3960765543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4300c2310 msgr2=0x7fd430153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.381+0000 7fd435000640 1 --2- 172.21.15.67:0/3960765543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4300c2310 0x7fd430153630 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd4300c2c60 tx=0x7fd41802b7c0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.382+0000 7fd435000640 1 -- 172.21.15.67:0/3960765543 shutdown_connections 2024-02-26T10:41:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.382+0000 7fd435000640 1 --2- 172.21.15.67:0/3960765543 >> v2:172.21.15.82:3300/0 conn(0x7fd43006b000 0x7fd4301524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.382+0000 7fd435000640 1 --2- 172.21.15.67:0/3960765543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4300c2310 0x7fd430153630 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.382+0000 7fd435000640 1 -- 172.21.15.67:0/3960765543 >> 172.21.15.67:0/3960765543 conn(0x7fd4300c63f0 msgr2=0x7fd4300c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.382+0000 7fd435000640 1 -- 172.21.15.67:0/3960765543 shutdown_connections 2024-02-26T10:41:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.382+0000 7fd435000640 1 -- 172.21.15.67:0/3960765543 wait complete. 2024-02-26T10:41:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.383+0000 7fd435000640 1 Processor -- start 2024-02-26T10:41:43.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.383+0000 7fd435000640 1 -- start start 2024-02-26T10:41:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.384+0000 7fd435000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43006ac20 0x7fd430072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.384+0000 7fd435000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 0x7fd4301df710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.384+0000 7fd435000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd4300c2310 0x7fd430141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.384+0000 7fd435000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4301559f0 con 0x7fd43006b000 2024-02-26T10:41:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.384+0000 7fd435000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd430155710 con 0x7fd43006ac20 2024-02-26T10:41:43.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.384+0000 7fd435000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd430148850 con 0x7fd4300c2310 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 0x7fd4301df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 0x7fd4301df710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52026/0 (socket says 172.21.15.67:52026) 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 -- 172.21.15.67:0/456679920 learned_addr learned my addr 172.21.15.67:0/456679920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42f400640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43006ac20 0x7fd430072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42fe00640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd4300c2310 0x7fd430141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 -- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd4300c2310 msgr2=0x7fd430141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd4300c2310 0x7fd430141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 -- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43006ac20 msgr2=0x7fd430072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43006ac20 0x7fd430072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42ea00640 1 -- 172.21.15.67:0/456679920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4301eabe0 con 0x7fd43006b000 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.385+0000 7fd42fe00640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd4300c2310 0x7fd430141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.386+0000 7fd42ea00640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 0x7fd4301df710 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd418009890 tx=0x7fd4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.386+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418042020 con 0x7fd43006b000 2024-02-26T10:41:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.386+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41803d950 con 0x7fd43006b000 2024-02-26T10:41:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.386+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418038040 con 0x7fd43006b000 2024-02-26T10:41:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.386+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4301eae60 con 0x7fd43006b000 2024-02-26T10:41:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.386+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4301eb420 con 0x7fd43006b000 2024-02-26T10:41:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.387+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4300c20f0 con 0x7fd43006b000 2024-02-26T10:41:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.388+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd418045020 con 0x7fd43006b000 2024-02-26T10:41:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.389+0000 7fd427400640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.389+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd4180ab9c0 con 0x7fd43006b000 2024-02-26T10:41:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.390+0000 7fd42f400640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.391+0000 7fd42f400640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd40c06b4d0 0x7fd40c06d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd420000f60 tx=0x7fd420009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:43.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.394+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd41804c030 con 0x7fd43006b000 2024-02-26T10:41:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.595+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd430074a10 con 0x7fd43006b000 2024-02-26T10:41:43.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.597+0000 7fd427400640 1 -- 172.21.15.67:0/456679920 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13079 (secure 0 0 0) 0x7fd4301eabe0 con 0x7fd43006b000 2024-02-26T10:41:43.596 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:43.596 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":67,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:41.096741+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/2903170369":"2024-02-27T10:41:37.793656+0000","172.21.15.82:6836/2903170369":"2024-02-27T10:41:37.793656+0000","172.21.15.67:6834/1127454674":"2024-02-27T10:41:29.521347+0000","172.21.15.67:6832/1127454674":"2024-02-27T10:41:29.521347+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd40c06b4d0 msgr2=0x7fd40c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd40c06b4d0 0x7fd40c06d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd420000f60 tx=0x7fd420009040 comp rx=0 tx=0).stop 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 msgr2=0x7fd4301df710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 0x7fd4301df710 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd418009890 tx=0x7fd4180046c0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 shutdown_connections 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd4300c2310 0x7fd430141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd43006b000 0x7fd4301df710 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 --2- 172.21.15.67:0/456679920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43006ac20 0x7fd430072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 >> 172.21.15.67:0/456679920 conn(0x7fd4300c63f0 msgr2=0x7fd430147430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 shutdown_connections 2024-02-26T10:41:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.600+0000 7fd435000640 1 -- 172.21.15.67:0/456679920 wait complete. 2024-02-26T10:41:43.619 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:43.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:43.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.844+0000 7f19f2200640 1 Processor -- start 2024-02-26T10:41:43.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.844+0000 7f19f2200640 1 -- start start 2024-02-26T10:41:43.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec1503a0 0x7f19ec0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f2200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f19ec0a9430 0x7f19ec1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f19ec06e2d0 0x7f19ec0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec153ae0 con 0x7f19ec06e2d0 2024-02-26T10:41:43.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f2200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec150780 con 0x7f19ec0a9430 2024-02-26T10:41:43.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0a98c0 con 0x7f19ec1503a0 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f19ec06e2d0 0x7f19ec0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f19ec06e2d0 0x7f19ec0c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52040/0 (socket says 172.21.15.67:52040) 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.846+0000 7f19f1800640 1 -- 172.21.15.67:0/626419358 learned_addr learned my addr 172.21.15.67:0/626419358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19f0e00640 1 --2- 172.21.15.67:0/626419358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec1503a0 0x7f19ec0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19ebe00640 1 --2- 172.21.15.67:0/626419358 >> v2:172.21.15.82:3300/0 conn(0x7f19ec0a9430 0x7f19ec1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19f1800640 1 -- 172.21.15.67:0/626419358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec1503a0 msgr2=0x7f19ec0ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19f1800640 1 --2- 172.21.15.67:0/626419358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec1503a0 0x7f19ec0ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19f1800640 1 -- 172.21.15.67:0/626419358 >> v2:172.21.15.82:3300/0 conn(0x7f19ec0a9430 msgr2=0x7f19ec1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19f1800640 1 --2- 172.21.15.67:0/626419358 >> v2:172.21.15.82:3300/0 conn(0x7f19ec0a9430 0x7f19ec1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.847+0000 7f19f1800640 1 -- 172.21.15.67:0/626419358 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0a9ba0 con 0x7f19ec06e2d0 2024-02-26T10:41:43.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.848+0000 7f19f1800640 1 --2- 172.21.15.67:0/626419358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec06e2d0 0x7f19ec0c2e10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0c7250 tx=0x7f19e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206e21dd3b2a8572 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:43.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.848+0000 7f19eb400640 1 -- 172.21.15.67:0/626419358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e402ca70 con 0x7f19ec06e2d0 2024-02-26T10:41:43.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.848+0000 7f19eb400640 1 -- 172.21.15.67:0/626419358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e402cc10 con 0x7f19ec06e2d0 2024-02-26T10:41:43.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.848+0000 7f19eb400640 1 -- 172.21.15.67:0/626419358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40343f0 con 0x7f19ec06e2d0 2024-02-26T10:41:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 -- 172.21.15.67:0/626419358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec06e2d0 msgr2=0x7f19ec0c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 --2- 172.21.15.67:0/626419358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec06e2d0 0x7f19ec0c2e10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0c7250 tx=0x7f19e402bed0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 -- 172.21.15.67:0/626419358 shutdown_connections 2024-02-26T10:41:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 --2- 172.21.15.67:0/626419358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec06e2d0 0x7f19ec0c2e10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 --2- 172.21.15.67:0/626419358 >> v2:172.21.15.82:3300/0 conn(0x7f19ec0a9430 0x7f19ec1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 --2- 172.21.15.67:0/626419358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec1503a0 0x7f19ec0ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 -- 172.21.15.67:0/626419358 >> 172.21.15.67:0/626419358 conn(0x7f19ec0c6380 msgr2=0x7f19ec0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 -- 172.21.15.67:0/626419358 shutdown_connections 2024-02-26T10:41:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.849+0000 7f19f2200640 1 -- 172.21.15.67:0/626419358 wait complete. 2024-02-26T10:41:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.851+0000 7f19f2200640 1 Processor -- start 2024-02-26T10:41:43.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.851+0000 7f19f2200640 1 -- start start 2024-02-26T10:41:43.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec06e2d0 0x7f19ec152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec1503a0 0x7f19ec0c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 0x7f19ec1eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec06ddb0 con 0x7f19ec147e60 2024-02-26T10:41:43.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec06dad0 con 0x7f19ec1503a0 2024-02-26T10:41:43.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0c0f80 con 0x7f19ec06e2d0 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 0x7f19ec1eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 0x7f19ec1eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52046/0 (socket says 172.21.15.67:52046) 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f1800640 1 -- 172.21.15.67:0/2785638346 learned_addr learned my addr 172.21.15.67:0/2785638346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.852+0000 7f19f0e00640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec06e2d0 0x7f19ec152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19ebe00640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec1503a0 0x7f19ec0c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19f1800640 1 -- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec06e2d0 msgr2=0x7f19ec152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19f1800640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec06e2d0 0x7f19ec152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19f1800640 1 -- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec1503a0 msgr2=0x7f19ec0c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19f1800640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec1503a0 0x7f19ec0c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19f1800640 1 -- 172.21.15.67:0/2785638346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0c5ea0 con 0x7f19ec147e60 2024-02-26T10:41:43.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19ebe00640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec1503a0 0x7f19ec0c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:43.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.853+0000 7f19f1800640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 0x7f19ec1eaa90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f19e4036040 tx=0x7f19e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:43.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.854+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4043020 con 0x7f19ec147e60 2024-02-26T10:41:43.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.854+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e403e450 con 0x7f19ec147e60 2024-02-26T10:41:43.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.854+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4009510 con 0x7f19ec147e60 2024-02-26T10:41:43.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.854+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec071280 con 0x7f19ec147e60 2024-02-26T10:41:43.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.854+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1eeab0 con 0x7f19ec147e60 2024-02-26T10:41:43.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.855+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec0678e0 con 0x7f19ec147e60 2024-02-26T10:41:43.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.856+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19e4046020 con 0x7f19ec147e60 2024-02-26T10:41:43.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.857+0000 7f19e8c00640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f19d006b4d0 0x7f19d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:43.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.857+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f19e40ac720 con 0x7f19ec147e60 2024-02-26T10:41:43.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.857+0000 7f19f0e00640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f19d006b4d0 0x7f19d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.859+0000 7f19f0e00640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f19d006b4d0 0x7f19d006d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:43.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:43.863+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19e404d030 con 0x7f19ec147e60 2024-02-26T10:41:44.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.061+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19ec074a30 con 0x7f19ec147e60 2024-02-26T10:41:44.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.062+0000 7f19e8c00640 1 -- 172.21.15.67:0/2785638346 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19e407b830 con 0x7f19ec147e60 2024-02-26T10:41:44.061 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f19d006b4d0 msgr2=0x7f19d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f19d006b4d0 0x7f19d006d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0008040 comp rx=0 tx=0).stop 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 msgr2=0x7f19ec1eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 0x7f19ec1eaa90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f19e4036040 tx=0x7f19e4004a40 comp rx=0 tx=0).stop 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 shutdown_connections 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19ec147e60 0x7f19ec1eaa90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f19d006b4d0 0x7f19d006d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec1503a0 0x7f19ec0c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 --2- 172.21.15.67:0/2785638346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f19ec06e2d0 0x7f19ec152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.065+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 >> 172.21.15.67:0/2785638346 conn(0x7f19ec0c6380 msgr2=0x7f19ec14c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.066+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 shutdown_connections 2024-02-26T10:41:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.066+0000 7f19f2200640 1 -- 172.21.15.67:0/2785638346 wait complete. 2024-02-26T10:41:44.082 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap' 2024-02-26T10:41:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.303+0000 7f88bf400640 1 Processor -- start 2024-02-26T10:41:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.303+0000 7f88bf400640 1 -- start start 2024-02-26T10:41:44.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88b8152420 0x7f88b80c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bf400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f88b80c35d0 0x7f88b815f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bf400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80c15e0 con 0x7f88b80bfa30 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bf400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b806e660 con 0x7f88b8152420 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bf400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b806e920 con 0x7f88b80c35d0 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88b8152420 0x7f88b80c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88b8152420 0x7f88b80c3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52054/0 (socket says 172.21.15.67:52054) 2024-02-26T10:41:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.305+0000 7f88bca00640 1 -- 172.21.15.67:0/3439213681 learned_addr learned my addr 172.21.15.67:0/3439213681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.306+0000 7f88bca00640 1 -- 172.21.15.67:0/3439213681 >> v2:172.21.15.82:3300/0 conn(0x7f88b80c35d0 msgr2=0x7f88b815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.306+0000 7f88bd400640 1 --2- 172.21.15.67:0/3439213681 >> v2:172.21.15.82:3300/0 conn(0x7f88b80c35d0 0x7f88b815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.306+0000 7f88bca00640 1 --2- 172.21.15.67:0/3439213681 >> v2:172.21.15.82:3300/0 conn(0x7f88b80c35d0 0x7f88b815f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.306+0000 7f88bca00640 1 -- 172.21.15.67:0/3439213681 >> v1:172.21.15.82:6789/0 conn(0x7f88b80bfa30 legacy=0x7f88b8158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:44.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.306+0000 7f88bca00640 1 -- 172.21.15.67:0/3439213681 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80c1880 con 0x7f88b8152420 2024-02-26T10:41:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.307+0000 7f88bca00640 1 --2- 172.21.15.67:0/3439213681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b80c3090 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f88b806f670 tx=0x7f88ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef4ff4e31780e22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.307+0000 7f88b7400640 1 -- 172.21.15.67:0/3439213681 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac02e070 con 0x7f88b8152420 2024-02-26T10:41:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.307+0000 7f88b7400640 1 -- 172.21.15.67:0/3439213681 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac037070 con 0x7f88b8152420 2024-02-26T10:41:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.307+0000 7f88b7400640 1 -- 172.21.15.67:0/3439213681 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac031a00 con 0x7f88b8152420 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 -- 172.21.15.67:0/3439213681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 msgr2=0x7f88b80c3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 --2- 172.21.15.67:0/3439213681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b80c3090 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f88b806f670 tx=0x7f88ac02b880 comp rx=0 tx=0).stop 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 -- 172.21.15.67:0/3439213681 shutdown_connections 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 --2- 172.21.15.67:0/3439213681 >> v2:172.21.15.82:3300/0 conn(0x7f88b80c35d0 0x7f88b815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 --2- 172.21.15.67:0/3439213681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b80c3090 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 -- 172.21.15.67:0/3439213681 >> 172.21.15.67:0/3439213681 conn(0x7f88b80a84a0 msgr2=0x7f88b80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 -- 172.21.15.67:0/3439213681 shutdown_connections 2024-02-26T10:41:44.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.308+0000 7f88bf400640 1 -- 172.21.15.67:0/3439213681 wait complete. 2024-02-26T10:41:44.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.310+0000 7f88bf400640 1 Processor -- start 2024-02-26T10:41:44.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.310+0000 7f88bf400640 1 -- start start 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bf400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b80bfa30 0x7f88b814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bf400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88b80c35d0 0x7f88b81eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b81eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bf400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8152800 con 0x7f88b8152420 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bf400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8158400 con 0x7f88b80bfa30 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bf400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8154e10 con 0x7f88b80c35d0 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b81eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b81eb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52062/0 (socket says 172.21.15.67:52062) 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bd400640 1 -- 172.21.15.67:0/2186912701 learned_addr learned my addr 172.21.15.67:0/2186912701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88b7e00640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88b80c35d0 0x7f88b81eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.311+0000 7f88bca00640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b80bfa30 0x7f88b814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bd400640 1 -- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88b80c35d0 msgr2=0x7f88b81eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bd400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88b80c35d0 0x7f88b81eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bd400640 1 -- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b80bfa30 msgr2=0x7f88b814fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bd400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b80bfa30 0x7f88b814fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bd400640 1 -- 172.21.15.67:0/2186912701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8154150 con 0x7f88b8152420 2024-02-26T10:41:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bca00640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b80bfa30 0x7f88b814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bd400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b81eb210 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f88b000cda0 tx=0x7f88b000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0003b90 con 0x7f88b8152420 2024-02-26T10:41:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b0003d30 con 0x7f88b8152420 2024-02-26T10:41:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0004240 con 0x7f88b8152420 2024-02-26T10:41:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b8153b40 con 0x7f88b8152420 2024-02-26T10:41:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.312+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b8153de0 con 0x7f88b8152420 2024-02-26T10:41:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.314+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b8074a30 con 0x7f88b8152420 2024-02-26T10:41:44.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.315+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88b00043e0 con 0x7f88b8152420 2024-02-26T10:41:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.316+0000 7f88b4c00640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.316+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f88b008aa80 con 0x7f88b8152420 2024-02-26T10:41:44.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.316+0000 7f88bca00640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:44.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.317+0000 7f88bca00640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f889c06b4d0 0x7f889c06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f88ac036000 tx=0x7f88ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.320+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88b001d340 con 0x7f88b8152420 2024-02-26T10:41:44.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:44.587+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]} v 0) v1 -- 0x7f88b806e440 con 0x7f88b8152420 2024-02-26T10:41:45.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.133+0000 7f88b4c00640 1 -- 172.21.15.67:0/2186912701 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]}]=0 v199) v1 ==== 162+0+0 (secure 0 0 0) 0x7f88b0059ba0 con 0x7f88b8152420 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f889c06b4d0 msgr2=0x7f889c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f889c06b4d0 0x7f889c06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f88ac036000 tx=0x7f88ac008030 comp rx=0 tx=0).stop 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 msgr2=0x7f88b81eb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b81eb210 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f88b000cda0 tx=0x7f88b000a660 comp rx=0 tx=0).stop 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 shutdown_connections 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b8152420 0x7f88b81eb210 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88b80c35d0 0x7f88b81eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 --2- 172.21.15.67:0/2186912701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b80bfa30 0x7f88b814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.137+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 >> 172.21.15.67:0/2186912701 conn(0x7f88b80a84a0 msgr2=0x7f88b80a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:45.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.138+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 shutdown_connections 2024-02-26T10:41:45.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.138+0000 7f88bf400640 1 -- 172.21.15.67:0/2186912701 wait complete. 2024-02-26T10:41:45.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-26T10:41:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.382+0000 7fb686000640 1 Processor -- start 2024-02-26T10:41:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.383+0000 7fb686000640 1 -- start start 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.384+0000 7fb686000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6800c5290 0x7fb6801562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.384+0000 7fb686000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb68006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb686000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68006c110 con 0x7fb68006ce30 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb686000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800c72c0 con 0x7fb6800c5290 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb686000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800c75a0 con 0x7fb680155880 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb685600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb68006ce30 0x7fb6800af8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54784/0 (socket says 172.21.15.67:54784) 2024-02-26T10:41:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb685600640 1 -- 172.21.15.67:0/3488194519 learned_addr learned my addr 172.21.15.67:0/3488194519 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb67fe00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb68006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb684c00640 1 --2- 172.21.15.67:0/3488194519 >> v2:172.21.15.82:3300/0 conn(0x7fb6800c5290 0x7fb6801562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988427331 0 0) 0x7fb68006c110 con 0x7fb68006ce30 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.385+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb660003520 con 0x7fb68006ce30 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893715720 0 0) 0x7fb660003520 con 0x7fb68006ce30 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 msgr2=0x7fb68006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb68006c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 >> v2:172.21.15.82:3300/0 conn(0x7fb6800c5290 msgr2=0x7fb6801562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 --2- 172.21.15.67:0/3488194519 >> v2:172.21.15.82:3300/0 conn(0x7fb6800c5290 0x7fb6801562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800c7880 con 0x7fb68006ce30 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fb678001670 con 0x7fb68006ce30 2024-02-26T10:41:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb684c00640 1 --2- 172.21.15.67:0/3488194519 >> v2:172.21.15.82:3300/0 conn(0x7fb6800c5290 0x7fb6801562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 >> v1:172.21.15.67:6789/0 conn(0x7fb68006ce30 legacy=0x7fb6800af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6600039b0 0x7fb660003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb660004d40 0x7fb6600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800c72c0 con 0x7fb6600039b0 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800c75a0 con 0x7fb660004d40 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66000baf0 con 0x7fb680155880 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.386+0000 7fb684c00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6600039b0 0x7fb660003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb685600640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb660004d40 0x7fb6600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb67fe00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb684c00640 1 -- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 msgr2=0x7fb660004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb684c00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb660004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb684c00640 1 -- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb660004d40 msgr2=0x7fb6600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:45.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb684c00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb660004d40 0x7fb6600051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb684c00640 1 -- 172.21.15.67:0/3488194519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660003520 con 0x7fb6600039b0 2024-02-26T10:41:45.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.387+0000 7fb67fe00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb680155880 0x7fb660004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.388+0000 7fb684c00640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6600039b0 0x7fb660003dd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb6700098a0 tx=0x7fb660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee818234b7144775 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.388+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000f040 con 0x7fb6600039b0 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.388+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670004230 con 0x7fb6600039b0 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.388+0000 7fb67f400640 1 -- 172.21.15.67:0/3488194519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670003610 con 0x7fb6600039b0 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb686000640 1 -- 172.21.15.67:0/3488194519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6600039b0 msgr2=0x7fb660003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb686000640 1 --2- 172.21.15.67:0/3488194519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6600039b0 0x7fb660003dd0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb6700098a0 tx=0x7fb660005830 comp rx=0 tx=0).stop 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb685600640 1 -- 172.21.15.67:0/3488194519 reap_dead start 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb686000640 1 -- 172.21.15.67:0/3488194519 shutdown_connections 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb686000640 1 -- 172.21.15.67:0/3488194519 >> 172.21.15.67:0/3488194519 conn(0x7fb6800ac230 msgr2=0x7fb6800ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb686000640 1 -- 172.21.15.67:0/3488194519 shutdown_connections 2024-02-26T10:41:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.389+0000 7fb686000640 1 -- 172.21.15.67:0/3488194519 wait complete. 2024-02-26T10:41:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.391+0000 7fb686000640 1 Processor -- start 2024-02-26T10:41:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.391+0000 7fb686000640 1 -- start start 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb686000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6600039b0 0x7fb6800ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb686000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 0x7fb6801ede90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb686000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6801eac30 0x7fb68014ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb686000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800affb0 con 0x7fb6800c1770 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb686000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680155cc0 con 0x7fb6801eac30 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb686000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800b0130 con 0x7fb6600039b0 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 0x7fb6801ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 0x7fb6801ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52076/0 (socket says 172.21.15.67:52076) 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 -- 172.21.15.67:0/1781272075 learned_addr learned my addr 172.21.15.67:0/1781272075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb684c00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6600039b0 0x7fb6800ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 -- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6600039b0 msgr2=0x7fb6800ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb685600640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6801eac30 0x7fb68014ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6600039b0 0x7fb6800ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 -- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6801eac30 msgr2=0x7fb68014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6801eac30 0x7fb68014ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb67fe00640 1 -- 172.21.15.67:0/1781272075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6801eb160 con 0x7fb6800c1770 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.392+0000 7fb684c00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6600039b0 0x7fb6800ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.393+0000 7fb67fe00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 0x7fb6801ede90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb66800f350 tx=0x7fb66800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.393+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668003990 con 0x7fb6800c1770 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.393+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668003b30 con 0x7fb6800c1770 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.393+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668004020 con 0x7fb6800c1770 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.393+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800c9570 con 0x7fb6800c1770 2024-02-26T10:41:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.393+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6801ed970 con 0x7fb6800c1770 2024-02-26T10:41:45.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.394+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb680074a30 con 0x7fb6800c1770 2024-02-26T10:41:45.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.395+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6680041c0 con 0x7fb6800c1770 2024-02-26T10:41:45.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.397+0000 7fb67cc00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb65c06b500 0x7fb65c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:45.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.397+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb66808a470 con 0x7fb6800c1770 2024-02-26T10:41:45.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.397+0000 7fb684c00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb65c06b500 0x7fb65c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:45.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.398+0000 7fb684c00640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb65c06b500 0x7fb65c06d9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb670035000 tx=0x7fb670039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:45.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.401+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb668012070 con 0x7fb6800c1770 2024-02-26T10:41:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.591+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb68006d950 con 0x7fb6800c1770 2024-02-26T10:41:45.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.596+0000 7fb67cc00640 1 -- 172.21.15.67:0/1781272075 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb668059590 con 0x7fb6800c1770 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb65c06b500 msgr2=0x7fb65c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb65c06b500 0x7fb65c06d9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb670035000 tx=0x7fb670039040 comp rx=0 tx=0).stop 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 msgr2=0x7fb6801ede90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 0x7fb6801ede90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb66800f350 tx=0x7fb66800f810 comp rx=0 tx=0).stop 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 shutdown_connections 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb65c06b500 0x7fb65c06d9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6801eac30 0x7fb68014ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6800c1770 0x7fb6801ede90 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 --2- 172.21.15.67:0/1781272075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6600039b0 0x7fb6800ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:45.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 >> 172.21.15.67:0/1781272075 conn(0x7fb6800ac230 msgr2=0x7fb680152c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:45.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 shutdown_connections 2024-02-26T10:41:45.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:45.599+0000 7fb686000640 1 -- 172.21.15.67:0/1781272075 wait complete. 2024-02-26T10:41:45.617 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-26T10:41:45.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:41:45.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:41:45.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-26T10:41:45.847 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:41:45.847 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:45.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:41:45.907 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:41:45.907 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:41:45.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:41:45.962 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-02-26T10:41:45.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:41:46.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.196+0000 7f099ca00640 1 Processor -- start 2024-02-26T10:41:46.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.196+0000 7f099ca00640 1 -- start start 2024-02-26T10:41:46.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f099ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09980a94a0 0x7f099806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f099ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 0x7f09980abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f099ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099806df90 con 0x7f099806ac20 2024-02-26T10:41:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f099ca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09980a9a80 con 0x7f09980a94a0 2024-02-26T10:41:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f099ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09980a9d60 con 0x7f099806db00 2024-02-26T10:41:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f0996800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f099806ac20 0x7f0998153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54790/0 (socket says 172.21.15.67:54790) 2024-02-26T10:41:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f0996800640 1 -- 172.21.15.67:0/1560880196 learned_addr learned my addr 172.21.15.67:0/1560880196 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f0995e00640 1 --2- 172.21.15.67:0/1560880196 >> v2:172.21.15.82:3300/0 conn(0x7f09980a94a0 0x7f099806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.198+0000 7f0997200640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 0x7f09980abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096915163 0 0) 0x7f099806df90 con 0x7f099806ac20 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097c003520 con 0x7f099806ac20 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 146599248 0 0) 0x7f097c003520 con 0x7f099806ac20 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 msgr2=0x7f09980abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 0x7f09980abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 >> v2:172.21.15.82:3300/0 conn(0x7f09980a94a0 msgr2=0x7f099806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 --2- 172.21.15.67:0/1560880196 >> v2:172.21.15.82:3300/0 conn(0x7f09980a94a0 0x7f099806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980ab750 con 0x7f099806ac20 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f0988001670 con 0x7f099806ac20 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 >> v1:172.21.15.67:6789/0 conn(0x7f099806ac20 legacy=0x7f0998153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 0x7f097c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f097c003f30 0x7f097c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09980a9a80 con 0x7f097c004d40 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09980a9d60 con 0x7f097c003f30 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.199+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c00b9c0 con 0x7f099806db00 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0997200640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0996800640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 0x7f097c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0995e00640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f097c003f30 0x7f097c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0997200640 1 -- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 msgr2=0x7f097c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0997200640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f099806db00 0x7f097c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0997200640 1 -- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f097c003f30 msgr2=0x7f097c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0997200640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f097c003f30 0x7f097c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.200+0000 7f0997200640 1 -- 172.21.15.67:0/1560880196 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c003520 con 0x7f097c004d40 2024-02-26T10:41:46.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.201+0000 7f0997200640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0990009fc0 tx=0x7f097c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d3adbfb51ce760 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:46.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.201+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990039070 con 0x7f097c004d40 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.201+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099002cc00 con 0x7f097c004d40 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.201+0000 7f0995400640 1 -- 172.21.15.67:0/1560880196 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990009c10 con 0x7f097c004d40 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f099ca00640 1 -- 172.21.15.67:0/1560880196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f097c004d40 msgr2=0x7f097c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f099ca00640 1 --2- 172.21.15.67:0/1560880196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0990009fc0 tx=0x7f097c005830 comp rx=0 tx=0).stop 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f0997200640 1 -- 172.21.15.67:0/1560880196 reap_dead start 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f099ca00640 1 -- 172.21.15.67:0/1560880196 shutdown_connections 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f099ca00640 1 -- 172.21.15.67:0/1560880196 >> 172.21.15.67:0/1560880196 conn(0x7f09980c63f0 msgr2=0x7f09980c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f099ca00640 1 -- 172.21.15.67:0/1560880196 shutdown_connections 2024-02-26T10:41:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.202+0000 7f099ca00640 1 -- 172.21.15.67:0/1560880196 wait complete. 2024-02-26T10:41:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.204+0000 7f099ca00640 1 Processor -- start 2024-02-26T10:41:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.204+0000 7f099ca00640 1 -- start start 2024-02-26T10:41:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f099ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09981568e0 0x7f0998147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f099ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0998156cc0 0x7f09981e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f099ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 0x7f09980c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f099ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998150590 con 0x7f09981e8ae0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f099ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998150410 con 0x7f0998156cc0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f099ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09980abf50 con 0x7f09981568e0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f0997200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 0x7f09980c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f0997200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 0x7f09980c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52096/0 (socket says 172.21.15.67:52096) 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f0997200640 1 -- 172.21.15.67:0/155363539 learned_addr learned my addr 172.21.15.67:0/155363539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.205+0000 7f0996800640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09981568e0 0x7f0998147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0995e00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0998156cc0 0x7f09981e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0997200640 1 -- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09981568e0 msgr2=0x7f0998147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0997200640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09981568e0 0x7f0998147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0997200640 1 -- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0998156cc0 msgr2=0x7f09981e85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0997200640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0998156cc0 0x7f09981e85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0997200640 1 -- 172.21.15.67:0/155363539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09981e9370 con 0x7f09981e8ae0 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0995e00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0998156cc0 0x7f09981e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f0997200640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 0x7f09980c5540 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f099000e230 tx=0x7f0990009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990039030 con 0x7f09981e8ae0 2024-02-26T10:41:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099003a040 con 0x7f09981e8ae0 2024-02-26T10:41:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990009c10 con 0x7f09981e8ae0 2024-02-26T10:41:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.206+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09981dd570 con 0x7f09981e8ae0 2024-02-26T10:41:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.207+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981dc760 con 0x7f09981e8ae0 2024-02-26T10:41:46.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.208+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09981dd360 con 0x7f09981e8ae0 2024-02-26T10:41:46.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.209+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0990045030 con 0x7f09981e8ae0 2024-02-26T10:41:46.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.210+0000 7f098ea00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f097806b4d0 0x7f097806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:46.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.210+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f09900abbc0 con 0x7f09981e8ae0 2024-02-26T10:41:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.210+0000 7f0996800640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f097806b4d0 0x7f097806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.212+0000 7f0996800640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f097806b4d0 0x7f097806d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0988004840 tx=0x7f0988009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:46.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.214+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f099004c060 con 0x7f09981e8ae0 2024-02-26T10:41:46.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.455+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0998074a30 con 0x7f09981e8ae0 2024-02-26T10:41:46.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.456+0000 7f098ea00640 1 -- 172.21.15.67:0/155363539 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f099007ad70 con 0x7f09981e8ae0 2024-02-26T10:41:46.455 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:46.455 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3811532830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3811532830},{"type":"v1","addr":"172.21.15.82:6835","nonce":3811532830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1837342478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1837342478},{"type":"v1","addr":"172.21.15.67:6835","nonce":1837342478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5542,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/496392104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":496392104},{"type":"v1","addr":"172.21.15.82:6837","nonce":496392104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/2899333510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2899333510},{"type":"v1","addr":"172.21.15.67:6834","nonce":2899333510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[]} 2024-02-26T10:41:46.455 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f097806b4d0 msgr2=0x7f097806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f097806b4d0 0x7f097806d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0988004840 tx=0x7f0988009040 comp rx=0 tx=0).stop 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 msgr2=0x7f09980c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 0x7f09980c5540 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f099000e230 tx=0x7f0990009750 comp rx=0 tx=0).stop 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 shutdown_connections 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09981e8ae0 0x7f09980c5540 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f097806b4d0 0x7f097806d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0998156cc0 0x7f09981e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 --2- 172.21.15.67:0/155363539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09981568e0 0x7f0998147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:46.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 >> 172.21.15.67:0/155363539 conn(0x7f09980c63f0 msgr2=0x7f09980c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:46.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.459+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 shutdown_connections 2024-02-26T10:41:46.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:46.460+0000 7f099ca00640 1 -- 172.21.15.67:0/155363539 wait complete. 2024-02-26T10:41:46.476 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-26T10:41:46.476 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-26T10:41:46.476 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-26T10:41:46.476 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:41:46.476 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-26T10:41:46.476 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-26T10:41:46.477 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-26T10:41:46.477 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:41:46.477 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-26T10:41:46.477 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-26T10:41:46.477 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-26T10:41:46.477 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:41:46.477 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-26T10:41:46.477 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-26T10:41:46.478 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-26T10:41:46.478 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:41:46.495 INFO:tasks.ceph.mds.c:Stopped 2024-02-26T10:41:46.495 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-26T10:41:46.496 INFO:tasks.ceph.mds.a:Stopped 2024-02-26T10:41:46.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-26T10:41:46.499 INFO:tasks.ceph.mds.d:Stopped 2024-02-26T10:41:46.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-26T10:41:46.543 INFO:tasks.ceph.mds.c:Started 2024-02-26T10:41:46.543 DEBUG:teuthology.parallel:result is None 2024-02-26T10:41:46.547 INFO:tasks.ceph.mds.d:Started 2024-02-26T10:41:46.547 DEBUG:teuthology.parallel:result is None 2024-02-26T10:41:46.549 INFO:tasks.ceph.mds.a:Started 2024-02-26T10:41:46.550 DEBUG:teuthology.parallel:result is None 2024-02-26T10:41:46.698 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:41:46.699 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-02-26T10:41:46.717 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-02-26T10:41:48.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-02-26T10:41:52.500 INFO:tasks.ceph.mds.b:Stopped 2024-02-26T10:41:52.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-26T10:41:52.508 INFO:tasks.ceph.mds.b:Started 2024-02-26T10:41:52.508 DEBUG:teuthology.parallel:result is None 2024-02-26T10:41:52.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-26T10:41:52.687 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-02-26T10:41:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.740+0000 7f24abe00640 1 Processor -- start 2024-02-26T10:41:52.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.741+0000 7f24abe00640 1 -- start start 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.742+0000 7f24abe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f24ac06ac20 0x7f24ac0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.742+0000 7f24abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06b000 0x7f24ac06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.742+0000 7f24abe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac06b460 con 0x7f24ac153610 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.742+0000 7f24abe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0a9a80 con 0x7f24ac06ac20 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.742+0000 7f24abe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0a9d60 con 0x7f24ac06b000 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24ab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f24ac153610 0x7f24ac0abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59764/0 (socket says 172.21.15.67:59764) 2024-02-26T10:41:52.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24ab400640 1 -- 172.21.15.67:0/3333421630 learned_addr learned my addr 172.21.15.67:0/3333421630 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:52.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24aa000640 1 --2- 172.21.15.67:0/3333421630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06b000 0x7f24ac06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:52.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24aaa00640 1 --2- 172.21.15.67:0/3333421630 >> v2:172.21.15.82:3300/0 conn(0x7f24ac06ac20 0x7f24ac0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:52.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24a9600640 1 -- 172.21.15.67:0/3333421630 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241057037 0 0) 0x7f24ac06b460 con 0x7f24ac153610 2024-02-26T10:41:52.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24a9600640 1 -- 172.21.15.67:0/3333421630 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2490003520 con 0x7f24ac153610 2024-02-26T10:41:52.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24aa000640 1 -- 172.21.15.67:0/3333421630 >> v2:172.21.15.82:3300/0 conn(0x7f24ac06ac20 msgr2=0x7f24ac0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:52.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24aa000640 1 --2- 172.21.15.67:0/3333421630 >> v2:172.21.15.82:3300/0 conn(0x7f24ac06ac20 0x7f24ac0a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24aa000640 1 -- 172.21.15.67:0/3333421630 >> v1:172.21.15.67:6789/0 conn(0x7f24ac153610 legacy=0x7f24ac0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.743+0000 7f24aa000640 1 -- 172.21.15.67:0/3333421630 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0ab750 con 0x7f24ac06b000 2024-02-26T10:41:52.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.745+0000 7f24aa000640 1 --2- 172.21.15.67:0/3333421630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06b000 0x7f24ac06db00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e637d4d71e1a432 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.745+0000 7f24a9600640 1 -- 172.21.15.67:0/3333421630 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494035020 con 0x7f24ac06b000 2024-02-26T10:41:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.745+0000 7f24a9600640 1 -- 172.21.15.67:0/3333421630 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494031830 con 0x7f24ac06b000 2024-02-26T10:41:52.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.745+0000 7f24a9600640 1 -- 172.21.15.67:0/3333421630 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494031be0 con 0x7f24ac06b000 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 -- 172.21.15.67:0/3333421630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06b000 msgr2=0x7f24ac06db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 --2- 172.21.15.67:0/3333421630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06b000 0x7f24ac06db00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402b8d0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 -- 172.21.15.67:0/3333421630 shutdown_connections 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 --2- 172.21.15.67:0/3333421630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06b000 0x7f24ac06db00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 --2- 172.21.15.67:0/3333421630 >> v2:172.21.15.82:3300/0 conn(0x7f24ac06ac20 0x7f24ac0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 -- 172.21.15.67:0/3333421630 >> 172.21.15.67:0/3333421630 conn(0x7f24ac0c63f0 msgr2=0x7f24ac0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 -- 172.21.15.67:0/3333421630 shutdown_connections 2024-02-26T10:41:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.746+0000 7f24abe00640 1 -- 172.21.15.67:0/3333421630 wait complete. 2024-02-26T10:41:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.748+0000 7f24abe00640 1 Processor -- start 2024-02-26T10:41:52.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.748+0000 7f24abe00640 1 -- start start 2024-02-26T10:41:52.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06ac20 0x7f24ac1edb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24abe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac06b000 0x7f24ac155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24abe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 0x7f24ac155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24abe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac150410 con 0x7f24ac153610 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24abe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac153aa0 con 0x7f24ac06b000 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24abe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac150590 con 0x7f24ac06ac20 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 0x7f24ac155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 0x7f24ac155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33246/0 (socket says 172.21.15.67:33246) 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24ab400640 1 -- 172.21.15.67:0/985277488 learned_addr learned my addr 172.21.15.67:0/985277488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24aa000640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac06b000 0x7f24ac155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:52.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.749+0000 7f24aaa00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06ac20 0x7f24ac1edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:52.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24ab400640 1 -- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06ac20 msgr2=0x7f24ac1edb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:52.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24ab400640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06ac20 0x7f24ac1edb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24ab400640 1 -- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac06b000 msgr2=0x7f24ac155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24ab400640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac06b000 0x7f24ac155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24ab400640 1 -- 172.21.15.67:0/985277488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac1ee440 con 0x7f24ac153610 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24aa000640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac06b000 0x7f24ac155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24ab400640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 0x7f24ac155a50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f24a400caa0 tx=0x7f24a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4017070 con 0x7f24ac153610 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400ac00 con 0x7f24ac153610 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4012400 con 0x7f24ac153610 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac1544a0 con 0x7f24ac153610 2024-02-26T10:41:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.750+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac155f90 con 0x7f24ac153610 2024-02-26T10:41:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.752+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac074a30 con 0x7f24ac153610 2024-02-26T10:41:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.753+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24a401a070 con 0x7f24ac153610 2024-02-26T10:41:52.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.754+0000 7f24a2a00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f248806b4d0 0x7f248806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:52.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.754+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f24a4089c80 con 0x7f24ac153610 2024-02-26T10:41:52.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.754+0000 7f24aaa00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f248806b4d0 0x7f248806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:52.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.756+0000 7f24aaa00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f248806b4d0 0x7f248806d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0728a0 tx=0x7f249c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:52.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.759+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24a40587e0 con 0x7f24ac153610 2024-02-26T10:41:52.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:52.972+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24ac06b9b0 con 0x7f24ac153610 2024-02-26T10:41:54.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.196+0000 7f24a2a00640 1 -- 172.21.15.67:0/985277488 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24a405c880 con 0x7f24ac153610 2024-02-26T10:41:54.195 INFO:teuthology.orchestra.run.smithi067.stderr: removed all blocklist entries 2024-02-26T10:41:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.199+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f248806b4d0 msgr2=0x7f248806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.199+0000 7f24abe00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f248806b4d0 0x7f248806d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0728a0 tx=0x7f249c009190 comp rx=0 tx=0).stop 2024-02-26T10:41:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.199+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 msgr2=0x7f24ac155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.199+0000 7f24abe00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 0x7f24ac155a50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f24a400caa0 tx=0x7f24a400cf60 comp rx=0 tx=0).stop 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 shutdown_connections 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24ac153610 0x7f24ac155a50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f248806b4d0 0x7f248806d990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac06b000 0x7f24ac155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 --2- 172.21.15.67:0/985277488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24ac06ac20 0x7f24ac1edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 >> 172.21.15.67:0/985277488 conn(0x7f24ac0c63f0 msgr2=0x7f24ac0abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 shutdown_connections 2024-02-26T10:41:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.200+0000 7f24abe00640 1 -- 172.21.15.67:0/985277488 wait complete. 2024-02-26T10:41:54.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-26T10:41:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.444+0000 7f1923200640 1 Processor -- start 2024-02-26T10:41:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.444+0000 7f1923200640 1 -- start start 2024-02-26T10:41:54.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.446+0000 7f1923200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 0x7f191c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.446+0000 7f1923200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c06df90 con 0x7f191c06ac20 2024-02-26T10:41:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.446+0000 7f1923200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c0a9a80 con 0x7f191c06db00 2024-02-26T10:41:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.446+0000 7f1923200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0a9d60 con 0x7f191c0a94a0 2024-02-26T10:41:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.446+0000 7f1921e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f191c06ac20 0x7f191c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59768/0 (socket says 172.21.15.67:59768) 2024-02-26T10:41:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.446+0000 7f1921e00640 1 -- 172.21.15.67:0/3585864077 learned_addr learned my addr 172.21.15.67:0/3585864077 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1921400640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 0x7f191c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539389355 0 0) 0x7f191c06df90 con 0x7f191c06ac20 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1904003520 con 0x7f191c06ac20 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276922580 0 0) 0x7f1904003520 con 0x7f191c06ac20 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 msgr2=0x7f191c06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 0x7f191c06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 >> v1:172.21.15.82:6789/0 conn(0x7f191c06db00 legacy=0x7f191c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0ab750 con 0x7f191c06ac20 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1910001670 con 0x7f191c06ac20 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 >> v1:172.21.15.67:6789/0 conn(0x7f191c06ac20 legacy=0x7f191c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 0x7f19040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904003f60 0x7f1904004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f19040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0a9d60 con 0x7f1904004d40 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190400baa0 con 0x7f1904003f60 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.447+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190400bca0 con 0x7f191c0a94a0 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1922800640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f19040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1921e00640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 0x7f19040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1921400640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904003f60 0x7f1904004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1922800640 1 -- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 msgr2=0x7f19040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1922800640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c0a94a0 0x7f19040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1922800640 1 -- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904003f60 msgr2=0x7f1904004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1922800640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904003f60 0x7f1904004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.448+0000 7f1922800640 1 -- 172.21.15.67:0/3585864077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904003520 con 0x7f1904004d40 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.449+0000 7f1922800640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f19040051c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1918001e50 tx=0x7f1904005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1fa1e106fe5f4b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918001a70 con 0x7f1904004d40 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918003b70 con 0x7f1904004d40 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1920a00640 1 -- 172.21.15.67:0/3585864077 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180050d0 con 0x7f1904004d40 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1923200640 1 -- 172.21.15.67:0/3585864077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 msgr2=0x7f19040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1923200640 1 --2- 172.21.15.67:0/3585864077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f19040051c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1918001e50 tx=0x7f1904005830 comp rx=0 tx=0).stop 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1922800640 1 -- 172.21.15.67:0/3585864077 reap_dead start 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1923200640 1 -- 172.21.15.67:0/3585864077 shutdown_connections 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.450+0000 7f1923200640 1 -- 172.21.15.67:0/3585864077 >> 172.21.15.67:0/3585864077 conn(0x7f191c0c63f0 msgr2=0x7f191c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.451+0000 7f1923200640 1 -- 172.21.15.67:0/3585864077 shutdown_connections 2024-02-26T10:41:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.451+0000 7f1923200640 1 -- 172.21.15.67:0/3585864077 wait complete. 2024-02-26T10:41:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.452+0000 7f1923200640 1 Processor -- start 2024-02-26T10:41:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.452+0000 7f1923200640 1 -- start start 2024-02-26T10:41:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1923200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f191c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1923200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f191c070c50 0x7f191c14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1923200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c147bd0 0x7f191c148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1923200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c150590 con 0x7f1904004d40 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1923200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c150410 con 0x7f191c070c50 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1923200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c06b000 con 0x7f191c147bd0 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.453+0000 7f1921e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f191c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f191c14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33256/0 (socket says 172.21.15.67:33256) 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 -- 172.21.15.67:0/2489997159 learned_addr learned my addr 172.21.15.67:0/2489997159 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1922800640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c147bd0 0x7f191c148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 -- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c147bd0 msgr2=0x7f191c148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c147bd0 0x7f191c148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 -- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f191c070c50 msgr2=0x7f191c14ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f191c070c50 0x7f191c14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 -- 172.21.15.67:0/2489997159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c071d10 con 0x7f1904004d40 2024-02-26T10:41:54.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.454+0000 7f1921e00640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f191c14e770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f191000e8d0 tx=0x7f191000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.455+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000c870 con 0x7f1904004d40 2024-02-26T10:41:54.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.455+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910018070 con 0x7f1904004d40 2024-02-26T10:41:54.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.455+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c071ff0 con 0x7f1904004d40 2024-02-26T10:41:54.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.455+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000cc50 con 0x7f1904004d40 2024-02-26T10:41:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.455+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c0722d0 con 0x7f1904004d40 2024-02-26T10:41:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.456+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c06b3d0 con 0x7f1904004d40 2024-02-26T10:41:54.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.457+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1910013420 con 0x7f1904004d40 2024-02-26T10:41:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.458+0000 7f1916000640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f190006b500 0x7f190006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.458+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1910089ba0 con 0x7f1904004d40 2024-02-26T10:41:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.458+0000 7f1921400640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f190006b500 0x7f190006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.459+0000 7f1921400640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f190006b500 0x7f190006d9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f191c14a660 tx=0x7f190c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.462+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f191c06b3d0 con 0x7f1904004d40 2024-02-26T10:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.730+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f191c074a30 con 0x7f1904004d40 2024-02-26T10:41:54.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.731+0000 7f1916000640 1 -- 172.21.15.67:0/2489997159 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5081 (secure 0 0 0) 0x7f1910058cd0 con 0x7f1904004d40 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_dump": [ 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.a", 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlrPwnMBAADC1UqiHSleXCZZ8LRLN/MQ==", 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.b", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlXEpIGBAANii7fywaaL0+vjyqLX6Atw==", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.c", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxl2Z1dBhAAT9aRD1Te0TwHE93ZvNdInA==", 2024-02-26T10:41:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mds.d", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlDx1gJxAAYGDhukySlrxjHaHWn+707g==", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:41:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.0", 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDzadxlki0qBBAAWmQAFvxfJJ5kYMrk0ChwLQ==", 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.1", 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD2adxl4X5OHRAAHtaf00M+kXgvYOn1/ddr7A==", 2024-02-26T10:41:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.2", 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD5adxlQqxnNxAAWzDQ/1DzO3r1ztTrociQzQ==", 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.3", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQD9adxlgCXOFRAAHdACX7JyyLLY0KPVHOycGw==", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.4", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAEatxlJ8FqFhAA2XFN8d8NUYW1rITUu3tQnA==", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.735 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.5", 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAHatxlO9YRMBAA6z8hYbTwHnU0BV3OMxKSjA==", 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.6", 2024-02-26T10:41:54.736 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAKatxlzGhIORAANuWy2QCgtFcJRO4vQV4EkQ==", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "osd.7", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQAOatxlUwuJGBAAYZlIHvgD3suXzh5kyeUWqQ==", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow profile osd", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile osd", 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.0", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw==", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.1", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw==", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow r", 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw pool=cephfs2_data" 2024-02-26T10:41:54.738 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.admin", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDsadxlvPgjIxAAClmsDQ2hetAdBGNLgj50lg==", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": "allow *", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow *", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "client.testuser", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDJatxldmarBhAA/d7MI1T3eae+VhCeSsGkFQ==", 2024-02-26T10:41:54.739 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow rw fsname=cephfs", 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow rw fsname=cephfs, allow rw fsname=cephfs2", 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.x", 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlQMXpIxAA0W1caz5FAL2wDMrNG3j5Mg==", 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: "entity": "mgr.y", 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "AQDtadxlMa/TFxAAAa7erEK+tXM+k5KOfx2JgA==", 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": { 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": "allow *", 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": "allow profile mgr", 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": "allow *" 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-02-26T10:41:54.741 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.733+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f190006b500 msgr2=0x7f190006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.733+0000 7f1923200640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f190006b500 0x7f190006d9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f191c14a660 tx=0x7f190c008040 comp rx=0 tx=0).stop 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.733+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 msgr2=0x7f191c14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.733+0000 7f1923200640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f191c14e770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f191000e8d0 tx=0x7f191000ed90 comp rx=0 tx=0).stop 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 shutdown_connections 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191c147bd0 0x7f191c148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f190006b500 0x7f190006d9c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f191c070c50 0x7f191c14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 --2- 172.21.15.67:0/2489997159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1904004d40 0x7f191c14e770 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 >> 172.21.15.67:0/2489997159 conn(0x7f191c0c63f0 msgr2=0x7f191c0c1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 shutdown_connections 2024-02-26T10:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.734+0000 7f1923200640 1 -- 172.21.15.67:0/2489997159 wait complete. 2024-02-26T10:41:54.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.testuser 2024-02-26T10:41:54.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-02-26T10:41:54.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.973+0000 7f2341a00640 1 Processor -- start 2024-02-26T10:41:54.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.974+0000 7f2341a00640 1 -- start start 2024-02-26T10:41:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.975+0000 7f2341a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 0x7f233c0c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.975+0000 7f2341a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c152e50 con 0x7f233c156230 2024-02-26T10:41:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.975+0000 7f2341a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c153130 con 0x7f233c06e410 2024-02-26T10:41:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.975+0000 7f2341a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c1506e0 con 0x7f233c150300 2024-02-26T10:41:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f233b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f233c156230 0x7f233c0ab2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59778/0 (socket says 172.21.15.67:59778) 2024-02-26T10:41:54.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f233b400640 1 -- 172.21.15.67:0/3754932452 learned_addr learned my addr 172.21.15.67:0/3754932452 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f233aa00640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 0x7f233c0c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770174383 0 0) 0x7f233c152e50 con 0x7f233c156230 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2320003520 con 0x7f233c156230 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666201789 0 0) 0x7f2320003520 con 0x7f233c156230 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 msgr2=0x7f233c0c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 0x7f233c0c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 >> v1:172.21.15.82:6789/0 conn(0x7f233c06e410 legacy=0x7f233c0aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c153410 con 0x7f233c156230 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f23340019f0 con 0x7f233c156230 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.976+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 >> v1:172.21.15.67:6789/0 conn(0x7f233c156230 legacy=0x7f233c0ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f2339600640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23200039e0 0x7f2320003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f2339600640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 0x7f2320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f2339600640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f23200051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c1506e0 con 0x7f2320004d40 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232000bbd0 con 0x7f23200039e0 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232000bdd0 con 0x7f233c150300 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233b400640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f23200051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233aa00640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23200039e0 0x7f2320003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233a000640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 0x7f2320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233b400640 1 -- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 msgr2=0x7f2320004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233b400640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c150300 0x7f2320004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233b400640 1 -- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23200039e0 msgr2=0x7f2320003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233b400640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23200039e0 0x7f2320003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.977+0000 7f233b400640 1 -- 172.21.15.67:0/3754932452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2320003520 con 0x7f2320004d40 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f233b400640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f23200051a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f233400a790 tx=0x7f2320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc298bbe26b1bd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400a3b0 con 0x7f2320004d40 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334034650 con 0x7f2320004d40 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2339600640 1 -- 172.21.15.67:0/3754932452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334034a40 con 0x7f2320004d40 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2341a00640 1 -- 172.21.15.67:0/3754932452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 msgr2=0x7f23200051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2341a00640 1 --2- 172.21.15.67:0/3754932452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f23200051a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f233400a790 tx=0x7f2320005830 comp rx=0 tx=0).stop 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f233b400640 1 -- 172.21.15.67:0/3754932452 reap_dead start 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2341a00640 1 -- 172.21.15.67:0/3754932452 shutdown_connections 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.979+0000 7f2341a00640 1 -- 172.21.15.67:0/3754932452 >> 172.21.15.67:0/3754932452 conn(0x7f233c0a84a0 msgr2=0x7f233c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.980+0000 7f2341a00640 1 -- 172.21.15.67:0/3754932452 shutdown_connections 2024-02-26T10:41:54.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.980+0000 7f2341a00640 1 -- 172.21.15.67:0/3754932452 wait complete. 2024-02-26T10:41:54.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.981+0000 7f2341a00640 1 Processor -- start 2024-02-26T10:41:54.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.981+0000 7f2341a00640 1 -- start start 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f2341a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f233c0bb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f2341a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c0c43b0 0x7f233c14e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f2341a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f233c13db90 0x7f233c13e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f2341a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c06dd80 con 0x7f2320004d40 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f2341a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c153a60 con 0x7f233c13db90 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f2341a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c0c2ff0 con 0x7f233c0c43b0 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f233aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f233c0bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f233aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f233c0bb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33274/0 (socket says 172.21.15.67:33274) 2024-02-26T10:41:54.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.982+0000 7f233aa00640 1 -- 172.21.15.67:0/1986011962 learned_addr learned my addr 172.21.15.67:0/1986011962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:54.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233a000640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c0c43b0 0x7f233c14e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233b400640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f233c13db90 0x7f233c13e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233aa00640 1 -- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c0c43b0 msgr2=0x7f233c14e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233aa00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c0c43b0 0x7f233c14e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233aa00640 1 -- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f233c13db90 msgr2=0x7f233c13e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233aa00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f233c13db90 0x7f233c13e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233aa00640 1 -- 172.21.15.67:0/1986011962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c1e92e0 con 0x7f2320004d40 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f233aa00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f233c0bb620 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f232c0119d0 tx=0x7f232c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00fb50 con 0x7f2320004d40 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00fcf0 con 0x7f2320004d40 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00fea0 con 0x7f2320004d40 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.983+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c1e95c0 con 0x7f2320004d40 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.984+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c152910 con 0x7f2320004d40 2024-02-26T10:41:54.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.985+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c074a30 con 0x7f2320004d40 2024-02-26T10:41:54.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.986+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f232c012040 con 0x7f2320004d40 2024-02-26T10:41:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.987+0000 7f2332a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231806b4d0 0x7f231806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.987+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f232c08a570 con 0x7f2320004d40 2024-02-26T10:41:54.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.987+0000 7f233a000640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231806b4d0 0x7f231806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:54.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.988+0000 7f233a000640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231806b4d0 0x7f231806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f2324009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:54.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:54.991+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f232c019030 con 0x7f2320004d40 2024-02-26T10:41:55.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.261+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.testuser"} v 0) v1 -- 0x7f233c06e0d0 con 0x7f2320004d40 2024-02-26T10:41:55.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.266+0000 7f2332a00640 1 -- 172.21.15.67:0/1986011962 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.testuser"}]=0 v16) v1 ==== 85+0+0 (secure 0 0 0) 0x7f232c059730 con 0x7f2320004d40 2024-02-26T10:41:55.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231806b4d0 msgr2=0x7f231806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231806b4d0 0x7f231806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f2324009040 comp rx=0 tx=0).stop 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 msgr2=0x7f233c0bb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f233c0bb620 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f232c0119d0 tx=0x7f232c011e90 comp rx=0 tx=0).stop 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 shutdown_connections 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f231806b4d0 0x7f231806d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f233c13db90 0x7f233c13e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f233c0c43b0 0x7f233c14e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 --2- 172.21.15.67:0/1986011962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320004d40 0x7f233c0bb620 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 >> 172.21.15.67:0/1986011962 conn(0x7f233c0a84a0 msgr2=0x7f233c06b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 shutdown_connections 2024-02-26T10:41:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.269+0000 7f2341a00640 1 -- 172.21.15.67:0/1986011962 wait complete. 2024-02-26T10:41:55.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:55.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.502+0000 7f8f95600640 1 Processor -- start 2024-02-26T10:41:55.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.502+0000 7f8f95600640 1 -- start start 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f95600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f9006ac20 0x7f8f900a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f95600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f9006b000 0x7f8f9006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f95600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9006b460 con 0x7f8f90153610 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f95600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f900a9a80 con 0x7f8f9006b000 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f95600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900a9d60 con 0x7f8f9006ac20 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f8f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f9006b000 0x7f8f9006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f8f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f9006b000 0x7f8f9006db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33284/0 (socket says 172.21.15.67:33284) 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f8f400640 1 -- 172.21.15.67:0/2443112610 learned_addr learned my addr 172.21.15.67:0/2443112610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f94c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8f90153610 0x7f8f900abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59780/0 (socket says 172.21.15.67:59780) 2024-02-26T10:41:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f8fe00640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f9006ac20 0x7f8f900a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.504+0000 7f8f8f400640 1 -- 172.21.15.67:0/2443112610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f9006ac20 msgr2=0x7f8f900a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.505+0000 7f8f8f400640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f9006ac20 0x7f8f900a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.505+0000 7f8f8f400640 1 -- 172.21.15.67:0/2443112610 >> v1:172.21.15.67:6789/0 conn(0x7f8f90153610 legacy=0x7f8f900abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.505+0000 7f8f8f400640 1 -- 172.21.15.67:0/2443112610 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900ab750 con 0x7f8f9006b000 2024-02-26T10:41:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.505+0000 7f8f8fe00640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f9006ac20 0x7f8f900a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.506+0000 7f8f8f400640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9006db00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009870 tx=0x7f8f7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2845fd33f4cd79a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.506+0000 7f8f8ea00640 1 -- 172.21.15.67:0/2443112610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c034070 con 0x7f8f9006b000 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.506+0000 7f8f8ea00640 1 -- 172.21.15.67:0/2443112610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c030840 con 0x7f8f9006b000 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.506+0000 7f8f8ea00640 1 -- 172.21.15.67:0/2443112610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c030c10 con 0x7f8f9006b000 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 -- 172.21.15.67:0/2443112610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 msgr2=0x7f8f9006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9006db00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009870 tx=0x7f8f7c02b980 comp rx=0 tx=0).stop 2024-02-26T10:41:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 -- 172.21.15.67:0/2443112610 shutdown_connections 2024-02-26T10:41:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9006db00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 --2- 172.21.15.67:0/2443112610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f9006ac20 0x7f8f900a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 -- 172.21.15.67:0/2443112610 >> 172.21.15.67:0/2443112610 conn(0x7f8f900c63f0 msgr2=0x7f8f900c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 -- 172.21.15.67:0/2443112610 shutdown_connections 2024-02-26T10:41:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.507+0000 7f8f95600640 1 -- 172.21.15.67:0/2443112610 wait complete. 2024-02-26T10:41:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.508+0000 7f8f95600640 1 Processor -- start 2024-02-26T10:41:55.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 -- start start 2024-02-26T10:41:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f9006ac20 0x7f8f900bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f90153610 0x7f8f90155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90150410 con 0x7f8f9006b000 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90153aa0 con 0x7f8f9006ac20 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.509+0000 7f8f95600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90150590 con 0x7f8f90153610 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9014c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33290/0 (socket says 172.21.15.67:33290) 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 -- 172.21.15.67:0/1226175782 learned_addr learned my addr 172.21.15.67:0/1226175782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 -- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f90153610 msgr2=0x7f8f90155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f94c00640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f90153610 0x7f8f90155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f90153610 0x7f8f90155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8fe00640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f9006ac20 0x7f8f900bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 -- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f9006ac20 msgr2=0x7f8f900bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f9006ac20 0x7f8f900bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8f400640 1 -- 172.21.15.67:0/1226175782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900c07b0 con 0x7f8f9006b000 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.510+0000 7f8f8fe00640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f9006ac20 0x7f8f900bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.511+0000 7f8f8f400640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9014c1d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009870 tx=0x7f8f7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.511+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c048070 con 0x7f8f9006b000 2024-02-26T10:41:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.511+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c030640 con 0x7f8f9006b000 2024-02-26T10:41:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.511+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c034040 con 0x7f8f9006b000 2024-02-26T10:41:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.511+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f901eebd0 con 0x7f8f9006b000 2024-02-26T10:41:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.511+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f901557c0 con 0x7f8f9006b000 2024-02-26T10:41:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.512+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f90074a30 con 0x7f8f9006b000 2024-02-26T10:41:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.513+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f7c037070 con 0x7f8f9006b000 2024-02-26T10:41:55.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.514+0000 7f8f87e00640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f7006b4d0 0x7f8f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.514+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f7c0abd70 con 0x7f8f9006b000 2024-02-26T10:41:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.515+0000 7f8f8fe00640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f7006b4d0 0x7f8f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.516+0000 7f8f8fe00640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f7006b4d0 0x7f8f7006d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8f900c0320 tx=0x7f8f80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.519+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f900c07b0 con 0x7f8f9006b000 2024-02-26T10:41:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.717+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f901e0010 con 0x7f8f9006b000 2024-02-26T10:41:55.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.719+0000 7f8f87e00640 1 -- 172.21.15.67:0/1226175782 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+12820 (secure 0 0 0) 0x7f8f7c07af30 con 0x7f8f9006b000 2024-02-26T10:41:55.718 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:55.718 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":72,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:54.185313+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:55.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.721+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f7006b4d0 msgr2=0x7f8f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.721+0000 7f8f95600640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f7006b4d0 0x7f8f7006d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8f900c0320 tx=0x7f8f80008040 comp rx=0 tx=0).stop 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 msgr2=0x7f8f9014c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9014c1d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009870 tx=0x7f8f7c004730 comp rx=0 tx=0).stop 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 shutdown_connections 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f90153610 0x7f8f90155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f9006b000 0x7f8f9014c1d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f7006b4d0 0x7f8f7006d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 --2- 172.21.15.67:0/1226175782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f9006ac20 0x7f8f900bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 >> 172.21.15.67:0/1226175782 conn(0x7f8f900c63f0 msgr2=0x7f8f9014f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 shutdown_connections 2024-02-26T10:41:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.722+0000 7f8f95600640 1 -- 172.21.15.67:0/1226175782 wait complete. 2024-02-26T10:41:55.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:41:55.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:41:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.954+0000 7fb9d7200640 1 Processor -- start 2024-02-26T10:41:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.954+0000 7fb9d7200640 1 -- start start 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9d80c5290 0x7fb9d81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d8155880 0x7fb9d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb9d806ce30 0x7fb9d80af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d806c110 con 0x7fb9d80c5290 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d80c72c0 con 0x7fb9d806ce30 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80c75a0 con 0x7fb9d8155880 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9d80c5290 0x7fb9d81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9d80c5290 0x7fb9d81562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33302/0 (socket says 172.21.15.67:33302) 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d5e00640 1 -- 172.21.15.67:0/953550526 learned_addr learned my addr 172.21.15.67:0/953550526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9cfe00640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d8155880 0x7fb9d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.956+0000 7fb9d6800640 1 --2- 172.21.15.67:0/953550526 >> v2:172.21.15.82:3300/0 conn(0x7fb9d806ce30 0x7fb9d80af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.957+0000 7fb9d5e00640 1 -- 172.21.15.67:0/953550526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d8155880 msgr2=0x7fb9d806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.957+0000 7fb9d5e00640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d8155880 0x7fb9d806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.957+0000 7fb9d5e00640 1 -- 172.21.15.67:0/953550526 >> v2:172.21.15.82:3300/0 conn(0x7fb9d806ce30 msgr2=0x7fb9d80af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.957+0000 7fb9d5e00640 1 --2- 172.21.15.67:0/953550526 >> v2:172.21.15.82:3300/0 conn(0x7fb9d806ce30 0x7fb9d80af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.957+0000 7fb9d5e00640 1 -- 172.21.15.67:0/953550526 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80c7880 con 0x7fb9d80c5290 2024-02-26T10:41:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.957+0000 7fb9cfe00640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d8155880 0x7fb9d806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.958+0000 7fb9d5e00640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d80c5290 0x7fb9d81562f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80c64d0 tx=0x7fb9c802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1cb9b6c7597629 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.958+0000 7fb9d5400640 1 -- 172.21.15.67:0/953550526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c802e070 con 0x7fb9d80c5290 2024-02-26T10:41:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.958+0000 7fb9d5400640 1 -- 172.21.15.67:0/953550526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8037070 con 0x7fb9d80c5290 2024-02-26T10:41:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.958+0000 7fb9d5400640 1 -- 172.21.15.67:0/953550526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8031a00 con 0x7fb9d80c5290 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 -- 172.21.15.67:0/953550526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d80c5290 msgr2=0x7fb9d81562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d80c5290 0x7fb9d81562f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80c64d0 tx=0x7fb9c802b880 comp rx=0 tx=0).stop 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 -- 172.21.15.67:0/953550526 shutdown_connections 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 --2- 172.21.15.67:0/953550526 >> v2:172.21.15.82:3300/0 conn(0x7fb9d806ce30 0x7fb9d80af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d8155880 0x7fb9d806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 --2- 172.21.15.67:0/953550526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d80c5290 0x7fb9d81562f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 -- 172.21.15.67:0/953550526 >> 172.21.15.67:0/953550526 conn(0x7fb9d80ac230 msgr2=0x7fb9d80ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 -- 172.21.15.67:0/953550526 shutdown_connections 2024-02-26T10:41:55.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.959+0000 7fb9d7200640 1 -- 172.21.15.67:0/953550526 wait complete. 2024-02-26T10:41:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.961+0000 7fb9d7200640 1 Processor -- start 2024-02-26T10:41:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.961+0000 7fb9d7200640 1 -- start start 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d80c5290 0x7fb9d8154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 0x7fb9d80c1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d80c4400 0x7fb9d80c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80c5670 con 0x7fb9d8155880 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8155cc0 con 0x7fb9d80c4400 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80c57f0 con 0x7fb9d80c5290 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 0x7fb9d80c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 0x7fb9d80c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33314/0 (socket says 172.21.15.67:33314) 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9cfe00640 1 -- 172.21.15.67:0/1919509966 learned_addr learned my addr 172.21.15.67:0/1919509966 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.962+0000 7fb9d5e00640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d80c5290 0x7fb9d8154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9d6800640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d80c4400 0x7fb9d80c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9cfe00640 1 -- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d80c5290 msgr2=0x7fb9d8154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9cfe00640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d80c5290 0x7fb9d8154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9cfe00640 1 -- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d80c4400 msgr2=0x7fb9d80c1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9cfe00640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d80c4400 0x7fb9d80c1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9cfe00640 1 -- 172.21.15.67:0/1919509966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d81e2250 con 0x7fb9d8155880 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9cfe00640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 0x7fb9d80c1770 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000d7d0 tx=0x7fb9c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0004030 con 0x7fb9d8155880 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c00041d0 con 0x7fb9d8155880 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00096a0 con 0x7fb9d8155880 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.963+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d81517b0 con 0x7fb9d8155880 2024-02-26T10:41:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.964+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80c3280 con 0x7fb9d8155880 2024-02-26T10:41:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.965+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d8074a30 con 0x7fb9d8155880 2024-02-26T10:41:55.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.965+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9c0009840 con 0x7fb9d8155880 2024-02-26T10:41:55.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.967+0000 7fb9ce000640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb9b406b500 0x7fb9b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:55.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.967+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb9c008a0b0 con 0x7fb9d8155880 2024-02-26T10:41:55.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.967+0000 7fb9d5e00640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb9b406b500 0x7fb9b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:55.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.968+0000 7fb9d5e00640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb9b406b500 0x7fb9b406d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8036000 tx=0x7fb9c803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:55.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:55.971+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9c00592f0 con 0x7fb9d8155880 2024-02-26T10:41:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.170+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9d8067f50 con 0x7fb9d8155880 2024-02-26T10:41:56.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.171+0000 7fb9ce000640 1 -- 172.21.15.67:0/1919509966 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9c0058bf0 con 0x7fb9d8155880 2024-02-26T10:41:56.169 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:41:56.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb9b406b500 msgr2=0x7fb9b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb9b406b500 0x7fb9b406d9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8036000 tx=0x7fb9c803d040 comp rx=0 tx=0).stop 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 msgr2=0x7fb9d80c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 0x7fb9d80c1770 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000d7d0 tx=0x7fb9c000dc90 comp rx=0 tx=0).stop 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 shutdown_connections 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb9b406b500 0x7fb9b406d9c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d80c4400 0x7fb9d80c1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9d8155880 0x7fb9d80c1770 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 --2- 172.21.15.67:0/1919509966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb9d80c5290 0x7fb9d8154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 >> 172.21.15.67:0/1919509966 conn(0x7fb9d80ac230 msgr2=0x7fb9d8155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 shutdown_connections 2024-02-26T10:41:56.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.174+0000 7fb9d7200640 1 -- 172.21.15.67:0/1919509966 wait complete. 2024-02-26T10:41:56.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:41:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.410+0000 7f6f9f600640 1 Processor -- start 2024-02-26T10:41:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.410+0000 7f6f9f600640 1 -- start start 2024-02-26T10:41:56.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f980c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9f600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f980c6b50 0x7f6f980ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9f600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f98158b20 con 0x7f6f980c4be0 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9f600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f980c5c60 con 0x7f6f980c6b50 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980c5f40 con 0x7f6f98152670 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f980c6b50 0x7f6f980ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f980c6b50 0x7f6f980ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33322/0 (socket says 172.21.15.67:33322) 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9d600640 1 -- 172.21.15.67:0/2115788924 learned_addr learned my addr 172.21.15.67:0/2115788924 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:56.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.412+0000 7f6f9cc00640 1 --2- 172.21.15.67:0/2115788924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f980c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.413+0000 7f6f9d600640 1 -- 172.21.15.67:0/2115788924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 msgr2=0x7f6f980c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:56.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.413+0000 7f6f9d600640 1 --2- 172.21.15.67:0/2115788924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f980c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.413+0000 7f6f9d600640 1 -- 172.21.15.67:0/2115788924 >> v1:172.21.15.82:6789/0 conn(0x7f6f980c4be0 legacy=0x7f6f98158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.413+0000 7f6f9d600640 1 -- 172.21.15.67:0/2115788924 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f980c07d0 con 0x7f6f980c6b50 2024-02-26T10:41:56.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.414+0000 7f6f9d600640 1 --2- 172.21.15.67:0/2115788924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c6b50 0x7f6f980ca710 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009e50 tx=0x7f6f9002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8676003bc03cd92a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.414+0000 7f6f97400640 1 -- 172.21.15.67:0/2115788924 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9002ca70 con 0x7f6f980c6b50 2024-02-26T10:41:56.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f97400640 1 -- 172.21.15.67:0/2115788924 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9002cc10 con 0x7f6f980c6b50 2024-02-26T10:41:56.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f97400640 1 -- 172.21.15.67:0/2115788924 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f900343f0 con 0x7f6f980c6b50 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 -- 172.21.15.67:0/2115788924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c6b50 msgr2=0x7f6f980ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 --2- 172.21.15.67:0/2115788924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c6b50 0x7f6f980ca710 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009e50 tx=0x7f6f9002bed0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 -- 172.21.15.67:0/2115788924 shutdown_connections 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 --2- 172.21.15.67:0/2115788924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c6b50 0x7f6f980ca710 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 --2- 172.21.15.67:0/2115788924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f980c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 -- 172.21.15.67:0/2115788924 >> 172.21.15.67:0/2115788924 conn(0x7f6f98072350 msgr2=0x7f6f98072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.415+0000 7f6f9f600640 1 -- 172.21.15.67:0/2115788924 shutdown_connections 2024-02-26T10:41:56.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.416+0000 7f6f9f600640 1 -- 172.21.15.67:0/2115788924 wait complete. 2024-02-26T10:41:56.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.417+0000 7f6f9f600640 1 Processor -- start 2024-02-26T10:41:56.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.417+0000 7f6f9f600640 1 -- start start 2024-02-26T10:41:56.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9f600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 0x7f6f98154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f980c6b50 0x7f6f98143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f981440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9f600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980c0480 con 0x7f6f980c4be0 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9f600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980c0d90 con 0x7f6f980c6b50 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980cac50 con 0x7f6f98152670 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.418+0000 7f6f9cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 0x7f6f98154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 0x7f6f98154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33334/0 (socket says 172.21.15.67:33334) 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 -- 172.21.15.67:0/176245571 learned_addr learned my addr 172.21.15.67:0/176245571 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f97e00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f980c6b50 0x7f6f98143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9d600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f981440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 -- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 msgr2=0x7f6f981440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f981440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 -- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f980c6b50 msgr2=0x7f6f98143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f980c6b50 0x7f6f98143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 -- 172.21.15.67:0/176245571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98150990 con 0x7f6f980c4be0 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f97e00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f980c6b50 0x7f6f98143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:56.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f9cc00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 0x7f6f98154080 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00d620 tx=0x7f6f8c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.419+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00b620 con 0x7f6f980c4be0 2024-02-26T10:41:56.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.420+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c019070 con 0x7f6f980c4be0 2024-02-26T10:41:56.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.420+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00ba00 con 0x7f6f980c4be0 2024-02-26T10:41:56.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.420+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f98150c70 con 0x7f6f980c4be0 2024-02-26T10:41:56.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.420+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f981545c0 con 0x7f6f980c4be0 2024-02-26T10:41:56.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.421+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f981ee990 con 0x7f6f980c4be0 2024-02-26T10:41:56.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.422+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f8c00bba0 con 0x7f6f980c4be0 2024-02-26T10:41:56.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.423+0000 7f6f94c00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f7806b500 0x7f6f7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.423+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f8c08a210 con 0x7f6f980c4be0 2024-02-26T10:41:56.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.424+0000 7f6f97e00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f7806b500 0x7f6f7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.425+0000 7f6f97e00640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f7806b500 0x7f6f7806d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6f9806cc40 tx=0x7f6f84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.427+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f981ee990 con 0x7f6f980c4be0 2024-02-26T10:41:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.627+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6f98074a30 con 0x7f6f980c4be0 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.628+0000 7f6f94c00640 1 -- 172.21.15.67:0/176245571 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6f8c059470 con 0x7f6f980c4be0 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:41:56.626 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:41:56.627 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f7806b500 msgr2=0x7f6f7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f7806b500 0x7f6f7806d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6f9806cc40 tx=0x7f6f84009040 comp rx=0 tx=0).stop 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 msgr2=0x7f6f98154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 0x7f6f98154080 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00d620 tx=0x7f6f8c00dae0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 shutdown_connections 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f98152670 0x7f6f981440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f7806b500 0x7f6f7806d9c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f980c6b50 0x7f6f98143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 --2- 172.21.15.67:0/176245571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f980c4be0 0x7f6f98154080 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 >> 172.21.15.67:0/176245571 conn(0x7f6f98072350 msgr2=0x7f6f9806d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.630+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 shutdown_connections 2024-02-26T10:41:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.631+0000 7f6f9f600640 1 -- 172.21.15.67:0/176245571 wait complete. 2024-02-26T10:41:56.648 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-26T10:41:56.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-26T10:41:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.864+0000 7f7872400640 1 Processor -- start 2024-02-26T10:41:56.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.864+0000 7f7872400640 1 -- start start 2024-02-26T10:41:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f7872400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f786c0c2950 0x7f786c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f7872400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f786c153920 0x7f786c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f7872400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0a6eb0 con 0x7f786c1503d0 2024-02-26T10:41:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f7872400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c153d50 con 0x7f786c153920 2024-02-26T10:41:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f7872400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0a7190 con 0x7f786c0c2950 2024-02-26T10:41:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f786c153920 0x7f786c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f786c153920 0x7f786c15d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33336/0 (socket says 172.21.15.67:33336) 2024-02-26T10:41:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 -- 172.21.15.67:0/991413308 learned_addr learned my addr 172.21.15.67:0/991413308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f7863e00640 1 --2- 172.21.15.67:0/991413308 >> v2:172.21.15.82:3300/0 conn(0x7f786c0c2950 0x7f786c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 -- 172.21.15.67:0/991413308 >> v2:172.21.15.82:3300/0 conn(0x7f786c0c2950 msgr2=0x7f786c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 --2- 172.21.15.67:0/991413308 >> v2:172.21.15.82:3300/0 conn(0x7f786c0c2950 0x7f786c1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 -- 172.21.15.67:0/991413308 >> v1:172.21.15.82:6789/0 conn(0x7f786c1503d0 legacy=0x7f786c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.866+0000 7f786be00640 1 -- 172.21.15.67:0/991413308 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0a7470 con 0x7f786c153920 2024-02-26T10:41:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.867+0000 7f7863e00640 1 --2- 172.21.15.67:0/991413308 >> v2:172.21.15.82:3300/0 conn(0x7f786c0c2950 0x7f786c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.868+0000 7f786be00640 1 --2- 172.21.15.67:0/991413308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c153920 0x7f786c15d650 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f786c0734b0 tx=0x7f786402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122c1281ce3633df server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.868+0000 7f786aa00640 1 -- 172.21.15.67:0/991413308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786402c960 con 0x7f786c153920 2024-02-26T10:41:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.868+0000 7f786aa00640 1 -- 172.21.15.67:0/991413308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786402cb00 con 0x7f786c153920 2024-02-26T10:41:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.868+0000 7f786aa00640 1 -- 172.21.15.67:0/991413308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640343f0 con 0x7f786c153920 2024-02-26T10:41:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 -- 172.21.15.67:0/991413308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c153920 msgr2=0x7f786c15d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 --2- 172.21.15.67:0/991413308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c153920 0x7f786c15d650 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f786c0734b0 tx=0x7f786402bdc0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 -- 172.21.15.67:0/991413308 shutdown_connections 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 --2- 172.21.15.67:0/991413308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c153920 0x7f786c15d650 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 --2- 172.21.15.67:0/991413308 >> v2:172.21.15.82:3300/0 conn(0x7f786c0c2950 0x7f786c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 -- 172.21.15.67:0/991413308 >> 172.21.15.67:0/991413308 conn(0x7f786c0c6da0 msgr2=0x7f786c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 -- 172.21.15.67:0/991413308 shutdown_connections 2024-02-26T10:41:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.869+0000 7f7872400640 1 -- 172.21.15.67:0/991413308 wait complete. 2024-02-26T10:41:56.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.871+0000 7f7872400640 1 Processor -- start 2024-02-26T10:41:56.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.871+0000 7f7872400640 1 -- start start 2024-02-26T10:41:56.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f7872400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 0x7f786c072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f7872400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f786c1503d0 0x7f786c141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f7872400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786c153920 0x7f786c141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f7872400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c06ef00 con 0x7f786c0c2950 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f7872400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c152ec0 con 0x7f786c1503d0 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f7872400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c1507b0 con 0x7f786c153920 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f786b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 0x7f786c072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f786b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 0x7f786c072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33348/0 (socket says 172.21.15.67:33348) 2024-02-26T10:41:56.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.872+0000 7f786b400640 1 -- 172.21.15.67:0/1568836577 learned_addr learned my addr 172.21.15.67:0/1568836577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:56.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f7863e00640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f786c1503d0 0x7f786c141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786be00640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786c153920 0x7f786c141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786b400640 1 -- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786c153920 msgr2=0x7f786c141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786b400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786c153920 0x7f786c141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786b400640 1 -- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f786c1503d0 msgr2=0x7f786c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786b400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f786c1503d0 0x7f786c141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786b400640 1 -- 172.21.15.67:0/1568836577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c151b80 con 0x7f786c0c2950 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f7863e00640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f786c1503d0 0x7f786c141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f786b400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 0x7f786c072190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c430 tx=0x7f785c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c011790 con 0x7f786c0c2950 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c017070 con 0x7f786c0c2950 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c011bb0 con 0x7f786c0c2950 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.873+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c150c20 con 0x7f786c0c2950 2024-02-26T10:41:56.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.874+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c150ec0 con 0x7f786c0c2950 2024-02-26T10:41:56.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.875+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c074a30 con 0x7f786c0c2950 2024-02-26T10:41:56.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.876+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f785c01c460 con 0x7f786c0c2950 2024-02-26T10:41:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.876+0000 7f7863400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.877+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f785c089da0 con 0x7f786c0c2950 2024-02-26T10:41:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.877+0000 7f7863e00640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.878+0000 7f7863e00640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f784806b4d0 0x7f784806d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7854006fd0 tx=0x7f7854008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:56.881+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f785c058f60 con 0x7f786c0c2950 2024-02-26T10:41:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.071+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f786c06ec50 con 0x7f786c0c2950 2024-02-26T10:41:57.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.224+0000 7f7863400640 1 -- 172.21.15.67:0/1568836577 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) v1 ==== 138+0+0 (secure 0 0 0) 0x7f785c058860 con 0x7f786c0c2950 2024-02-26T10:41:57.223 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-02-26T10:41:57.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.227+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f784806b4d0 msgr2=0x7f784806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.227+0000 7f7872400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f784806b4d0 0x7f784806d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7854006fd0 tx=0x7f7854008040 comp rx=0 tx=0).stop 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.227+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 msgr2=0x7f786c072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.227+0000 7f7872400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 0x7f786c072190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c430 tx=0x7f785c00c8f0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 shutdown_connections 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786c153920 0x7f786c141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f786c1503d0 0x7f786c141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 --2- 172.21.15.67:0/1568836577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f786c0c2950 0x7f786c072190 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 >> 172.21.15.67:0/1568836577 conn(0x7f786c0c6da0 msgr2=0x7f786c0c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 shutdown_connections 2024-02-26T10:41:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.228+0000 7f7872400640 1 -- 172.21.15.67:0/1568836577 wait complete. 2024-02-26T10:41:57.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:41:57.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.467+0000 7f0d0f200640 1 Processor -- start 2024-02-26T10:41:57.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.468+0000 7f0d0f200640 1 -- start start 2024-02-26T10:41:57.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.469+0000 7f0d0f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d1013c000 0x7f0d10072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.469+0000 7f0d0f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d1006e460 0x7f0d100c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.469+0000 7f0d0f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d100c3670 0x7f0d10152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.469+0000 7f0d0f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1006e8c0 con 0x7f0d100c3670 2024-02-26T10:41:57.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.469+0000 7f0d0f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1006eb60 con 0x7f0d1006e460 2024-02-26T10:41:57.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.469+0000 7f0d0f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d101564b0 con 0x7f0d1013c000 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.470+0000 7f0d0e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d100c3670 0x7f0d10152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.470+0000 7f0d05400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d1006e460 0x7f0d100c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.470+0000 7f0d0de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d1013c000 0x7f0d10072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.470+0000 7f0d0e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0d100c3670 0x7f0d10152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33350/0 (socket says 172.21.15.67:33350) 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.470+0000 7f0d0e800640 1 -- 172.21.15.67:0/876749737 learned_addr learned my addr 172.21.15.67:0/876749737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.470+0000 7f0d0e800640 1 -- 172.21.15.67:0/876749737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d1013c000 msgr2=0x7f0d10072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:57.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.471+0000 7f0d0e800640 1 --2- 172.21.15.67:0/876749737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d1013c000 0x7f0d10072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.471+0000 7f0d0e800640 1 -- 172.21.15.67:0/876749737 >> v2:172.21.15.82:3300/0 conn(0x7f0d1006e460 msgr2=0x7f0d100c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:57.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.471+0000 7f0d0e800640 1 --2- 172.21.15.67:0/876749737 >> v2:172.21.15.82:3300/0 conn(0x7f0d1006e460 0x7f0d100c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.471+0000 7f0d0e800640 1 -- 172.21.15.67:0/876749737 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10156790 con 0x7f0d100c3670 2024-02-26T10:41:57.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.471+0000 7f0d0de00640 1 --2- 172.21.15.67:0/876749737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d1013c000 0x7f0d10072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:57.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.472+0000 7f0d0e800640 1 --2- 172.21.15.67:0/876749737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d100c3670 0x7f0d10152ea0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0d100c3fb0 tx=0x7f0d0802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c1bf42d0e356eb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:57.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.472+0000 7f0d0d400640 1 -- 172.21.15.67:0/876749737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0802c9e0 con 0x7f0d100c3670 2024-02-26T10:41:57.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.472+0000 7f0d0d400640 1 -- 172.21.15.67:0/876749737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0802cb80 con 0x7f0d100c3670 2024-02-26T10:41:57.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.472+0000 7f0d0d400640 1 -- 172.21.15.67:0/876749737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080343f0 con 0x7f0d100c3670 2024-02-26T10:41:57.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 -- 172.21.15.67:0/876749737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d100c3670 msgr2=0x7f0d10152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 --2- 172.21.15.67:0/876749737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d100c3670 0x7f0d10152ea0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0d100c3fb0 tx=0x7f0d0802be40 comp rx=0 tx=0).stop 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 -- 172.21.15.67:0/876749737 shutdown_connections 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 --2- 172.21.15.67:0/876749737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d100c3670 0x7f0d10152ea0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 --2- 172.21.15.67:0/876749737 >> v2:172.21.15.82:3300/0 conn(0x7f0d1006e460 0x7f0d100c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 --2- 172.21.15.67:0/876749737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d1013c000 0x7f0d10072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 -- 172.21.15.67:0/876749737 >> 172.21.15.67:0/876749737 conn(0x7f0d100a84a0 msgr2=0x7f0d100a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 -- 172.21.15.67:0/876749737 shutdown_connections 2024-02-26T10:41:57.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.473+0000 7f0d0f200640 1 -- 172.21.15.67:0/876749737 wait complete. 2024-02-26T10:41:57.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.475+0000 7f0d0f200640 1 Processor -- start 2024-02-26T10:41:57.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.475+0000 7f0d0f200640 1 -- start start 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1006e460 0x7f0d10139040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d100c3670 0x7f0d101eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 0x7f0d101eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d10157270 con 0x7f0d1013c000 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d10156d50 con 0x7f0d1006e460 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1013c3e0 con 0x7f0d100c3670 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 0x7f0d101eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 0x7f0d101eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33360/0 (socket says 172.21.15.67:33360) 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0e800640 1 -- 172.21.15.67:0/1770803423 learned_addr learned my addr 172.21.15.67:0/1770803423 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d0de00640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1006e460 0x7f0d10139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.476+0000 7f0d05400640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d100c3670 0x7f0d101eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d0e800640 1 -- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d100c3670 msgr2=0x7f0d101eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d0e800640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d100c3670 0x7f0d101eac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d0e800640 1 -- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1006e460 msgr2=0x7f0d10139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d0e800640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1006e460 0x7f0d10139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:57.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d0e800640 1 -- 172.21.15.67:0/1770803423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10137b80 con 0x7f0d1013c000 2024-02-26T10:41:57.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d0e800640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 0x7f0d101eb190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d08036040 tx=0x7f0d08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:57.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08043020 con 0x7f0d1013c000 2024-02-26T10:41:57.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0803e450 con 0x7f0d1013c000 2024-02-26T10:41:57.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.477+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08009510 con 0x7f0d1013c000 2024-02-26T10:41:57.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.478+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d10136440 con 0x7f0d1013c000 2024-02-26T10:41:57.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.478+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d101380e0 con 0x7f0d1013c000 2024-02-26T10:41:57.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.479+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10075020 con 0x7f0d1013c000 2024-02-26T10:41:57.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.480+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d08046020 con 0x7f0d1013c000 2024-02-26T10:41:57.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.481+0000 7f0d06a00640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0cec06b4d0 0x7f0cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:57.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.481+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0d080ac810 con 0x7f0d1013c000 2024-02-26T10:41:57.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.482+0000 7f0d0de00640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0cec06b4d0 0x7f0cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:57.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.483+0000 7f0d0de00640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0cec06b4d0 0x7f0cec06d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0d00006fd0 tx=0x7f0d00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:57.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.487+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d10075020 con 0x7f0d1013c000 2024-02-26T10:41:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:57.691+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0d10074a30 con 0x7f0d1013c000 2024-02-26T10:41:58.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.231+0000 7f0d06a00640 1 -- 172.21.15.67:0/1770803423 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0d0804d030 con 0x7f0d1013c000 2024-02-26T10:41:58.229 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.234+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0cec06b4d0 msgr2=0x7f0cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.234+0000 7f0d0f200640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0cec06b4d0 0x7f0cec06d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0d00006fd0 tx=0x7f0d00008040 comp rx=0 tx=0).stop 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.234+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 msgr2=0x7f0d101eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.234+0000 7f0d0f200640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 0x7f0d101eb190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d08036040 tx=0x7f0d08004a40 comp rx=0 tx=0).stop 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 shutdown_connections 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d1013c000 0x7f0d101eb190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d100c3670 0x7f0d101eac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f0cec06b4d0 0x7f0cec06d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 --2- 172.21.15.67:0/1770803423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1006e460 0x7f0d10139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 >> 172.21.15.67:0/1770803423 conn(0x7f0d100a84a0 msgr2=0x7f0d100a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:58.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.235+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 shutdown_connections 2024-02-26T10:41:58.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.236+0000 7f0d0f200640 1 -- 172.21.15.67:0/1770803423 wait complete. 2024-02-26T10:41:58.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-26T10:41:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.461+0000 7f7643200640 1 Processor -- start 2024-02-26T10:41:58.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.461+0000 7f7643200640 1 -- start start 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.462+0000 7f7643200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f763c1503a0 0x7f763c0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.462+0000 7f7643200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c0a9430 0x7f763c1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7643200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c153ae0 con 0x7f763c06e2d0 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7643200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c150780 con 0x7f763c1503a0 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7643200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0a98c0 con 0x7f763c0a9430 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f763c1503a0 0x7f763c0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f763c1503a0 0x7f763c0ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33372/0 (socket says 172.21.15.67:33372) 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 -- 172.21.15.67:0/1603596394 learned_addr learned my addr 172.21.15.67:0/1603596394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641400640 1 --2- 172.21.15.67:0/1603596394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c0a9430 0x7f763c1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 -- 172.21.15.67:0/1603596394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c0a9430 msgr2=0x7f763c1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:58.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 --2- 172.21.15.67:0/1603596394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c0a9430 0x7f763c1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 -- 172.21.15.67:0/1603596394 >> v1:172.21.15.82:6789/0 conn(0x7f763c06e2d0 legacy=0x7f763c0c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:58.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.463+0000 7f7641e00640 1 -- 172.21.15.67:0/1603596394 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0a9ba0 con 0x7f763c1503a0 2024-02-26T10:41:58.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.465+0000 7f7641e00640 1 --2- 172.21.15.67:0/1603596394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0ab480 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f763c073030 tx=0x7f763002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438173854189bf49 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:58.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.465+0000 7f7640a00640 1 -- 172.21.15.67:0/1603596394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630035020 con 0x7f763c1503a0 2024-02-26T10:41:58.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.465+0000 7f7640a00640 1 -- 172.21.15.67:0/1603596394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7630031830 con 0x7f763c1503a0 2024-02-26T10:41:58.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.465+0000 7f7640a00640 1 -- 172.21.15.67:0/1603596394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630031c00 con 0x7f763c1503a0 2024-02-26T10:41:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 -- 172.21.15.67:0/1603596394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 msgr2=0x7f763c0ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:58.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 --2- 172.21.15.67:0/1603596394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0ab480 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f763c073030 tx=0x7f763002b8d0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 -- 172.21.15.67:0/1603596394 shutdown_connections 2024-02-26T10:41:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 --2- 172.21.15.67:0/1603596394 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c0a9430 0x7f763c1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 --2- 172.21.15.67:0/1603596394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0ab480 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 -- 172.21.15.67:0/1603596394 >> 172.21.15.67:0/1603596394 conn(0x7f763c0c6380 msgr2=0x7f763c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 -- 172.21.15.67:0/1603596394 shutdown_connections 2024-02-26T10:41:58.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.466+0000 7f7643200640 1 -- 172.21.15.67:0/1603596394 wait complete. 2024-02-26T10:41:58.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.468+0000 7f7643200640 1 Processor -- start 2024-02-26T10:41:58.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.468+0000 7f7643200640 1 -- start start 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7643200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c06e2d0 0x7f763c070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7643200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7643200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f763c072740 0x7f763c147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7643200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c06ddb0 con 0x7f763c1503a0 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7643200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c06dad0 con 0x7f763c072740 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7643200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0c0f80 con 0x7f763c06e2d0 2024-02-26T10:41:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7641400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7641400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33382/0 (socket says 172.21.15.67:33382) 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7641400640 1 -- 172.21.15.67:0/3629165721 learned_addr learned my addr 172.21.15.67:0/3629165721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7642800640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f763c072740 0x7f763c147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.469+0000 7f7641e00640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c06e2d0 0x7f763c070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7641400640 1 -- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c06e2d0 msgr2=0x7f763c070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7641400640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c06e2d0 0x7f763c070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7641400640 1 -- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f763c072740 msgr2=0x7f763c147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7641400640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f763c072740 0x7f763c147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7641400640 1 -- 172.21.15.67:0/3629165721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c072c60 con 0x7f763c1503a0 2024-02-26T10:41:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7641400640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0c1ae0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f762c00d800 tx=0x7f762c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00b6c0 con 0x7f763c1503a0 2024-02-26T10:41:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c019070 con 0x7f763c1503a0 2024-02-26T10:41:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00ba80 con 0x7f763c1503a0 2024-02-26T10:41:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c1ee770 con 0x7f763c1503a0 2024-02-26T10:41:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.470+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c1eea10 con 0x7f763c1503a0 2024-02-26T10:41:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.472+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c0678e0 con 0x7f763c1503a0 2024-02-26T10:41:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.473+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f762c00bc20 con 0x7f763c1503a0 2024-02-26T10:41:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.473+0000 7f7636000640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.474+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f762c08a690 con 0x7f763c1503a0 2024-02-26T10:41:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.474+0000 7f7641e00640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:58.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.475+0000 7f7641e00640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f762006b4d0 0x7f762006d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7630036000 tx=0x7f763003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:58.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.479+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f762c059560 con 0x7f763c1503a0 2024-02-26T10:41:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:58.712+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f763c0c1650 con 0x7f763c1503a0 2024-02-26T10:41:59.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.272+0000 7f7636000640 1 -- 172.21.15.67:0/3629165721 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7f762c058e60 con 0x7f763c1503a0 2024-02-26T10:41:59.272 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 12 and data pool 13 2024-02-26T10:41:59.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.284+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f762006b4d0 msgr2=0x7f762006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.284+0000 7f7643200640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f762006b4d0 0x7f762006d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7630036000 tx=0x7f763003c040 comp rx=0 tx=0).stop 2024-02-26T10:41:59.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.284+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 msgr2=0x7f763c0c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.284+0000 7f7643200640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0c1ae0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f762c00d800 tx=0x7f762c00dcc0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 shutdown_connections 2024-02-26T10:41:59.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f763c072740 0x7f763c147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f763c1503a0 0x7f763c0c1ae0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 --2- 172.21.15.67:0/3629165721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f763c06e2d0 0x7f763c070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 >> 172.21.15.67:0/3629165721 conn(0x7f763c0c6380 msgr2=0x7f763c14b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:59.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.286+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 shutdown_connections 2024-02-26T10:41:59.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.287+0000 7f7643200640 1 -- 172.21.15.67:0/3629165721 wait complete. 2024-02-26T10:41:59.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.499+0000 7fdeeac00640 1 Processor -- start 2024-02-26T10:41:59.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.499+0000 7fdeeac00640 1 -- start start 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.500+0000 7fdeeac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdee414f3d0 0x7fdee406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.500+0000 7fdeeac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 0x7fdee4155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.500+0000 7fdeeac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee406ed10 con 0x7fdee41556a0 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.500+0000 7fdeeac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee406dd80 con 0x7fdee414f3d0 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.500+0000 7fdeeac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee4155c00 con 0x7fdee40c2870 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.500+0000 7fdee8c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdee41556a0 0x7fdee4071700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56092/0 (socket says 172.21.15.67:56092) 2024-02-26T10:41:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee8c00640 1 -- 172.21.15.67:0/68918427 learned_addr learned my addr 172.21.15.67:0/68918427 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3e00640 1 --2- 172.21.15.67:0/68918427 >> v2:172.21.15.82:3300/0 conn(0x7fdee414f3d0 0x7fdee406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdedb400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 0x7fdee4155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439913436 0 0) 0x7fdee406ed10 con 0x7fdee41556a0 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec8003520 con 0x7fdee41556a0 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051348180 0 0) 0x7fdec8003520 con 0x7fdee41556a0 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 msgr2=0x7fdee4155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 0x7fdee4155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 >> v2:172.21.15.82:3300/0 conn(0x7fdee414f3d0 msgr2=0x7fdee406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 --2- 172.21.15.67:0/68918427 >> v2:172.21.15.82:3300/0 conn(0x7fdee414f3d0 0x7fdee406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4155ee0 con 0x7fdee41556a0 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fdedc0019f0 con 0x7fdee41556a0 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.501+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 >> v1:172.21.15.67:6789/0 conn(0x7fdee41556a0 legacy=0x7fdee4071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 0x7fdec80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8004d40 0x7fdec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406dd80 con 0x7fdec8003f80 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee4155c00 con 0x7fdec8004d40 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec800dc80 con 0x7fdee40c2870 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdedb400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdee3e00640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 0x7fdec80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdedb400640 1 -- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 msgr2=0x7fdec80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdedb400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40c2870 0x7fdec80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdedb400640 1 -- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8004d40 msgr2=0x7fdec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:59.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdedb400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8004d40 0x7fdec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.502+0000 7fdedb400640 1 -- 172.21.15.67:0/68918427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8003520 con 0x7fdec8003f80 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.503+0000 7fdedb400640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdec8004800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009ac0 tx=0x7fdec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599bf514357fb7f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc039070 con 0x7fdec8003f80 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc02c6b0 con 0x7fdec8003f80 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdee3400640 1 -- 172.21.15.67:0/68918427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc009690 con 0x7fdec8003f80 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdeeac00640 1 -- 172.21.15.67:0/68918427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 msgr2=0x7fdec8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdeeac00640 1 --2- 172.21.15.67:0/68918427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdec8004800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009ac0 tx=0x7fdec8005830 comp rx=0 tx=0).stop 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdee8c00640 1 -- 172.21.15.67:0/68918427 reap_dead start 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdeeac00640 1 -- 172.21.15.67:0/68918427 shutdown_connections 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdeeac00640 1 -- 172.21.15.67:0/68918427 >> 172.21.15.67:0/68918427 conn(0x7fdee40a84a0 msgr2=0x7fdee40a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdeeac00640 1 -- 172.21.15.67:0/68918427 shutdown_connections 2024-02-26T10:41:59.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.504+0000 7fdeeac00640 1 -- 172.21.15.67:0/68918427 wait complete. 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.506+0000 7fdeeac00640 1 Processor -- start 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.506+0000 7fdeeac00640 1 -- start start 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdeeac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdee40bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdeeac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee4147c20 0x7fdee4148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdeeac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee41487d0 0x7fdee4145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdeeac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40c1810 con 0x7fdec8003f80 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdeeac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406e2b0 con 0x7fdee41487d0 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdeeac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee4071e10 con 0x7fdee4147c20 2024-02-26T10:41:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdee40bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdee40bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43338/0 (socket says 172.21.15.67:43338) 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.507+0000 7fdee3e00640 1 -- 172.21.15.67:0/72172999 learned_addr learned my addr 172.21.15.67:0/72172999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee3e00640 1 -- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee4147c20 msgr2=0x7fdee4148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee3e00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee4147c20 0x7fdee4148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee3e00640 1 -- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee41487d0 msgr2=0x7fdee4145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee3e00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee41487d0 0x7fdee4145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee3e00640 1 -- 172.21.15.67:0/72172999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee41eb310 con 0x7fdec8003f80 2024-02-26T10:41:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee3e00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdee40bfca0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fded40108e0 tx=0x7fded4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400f040 con 0x7fdec8003f80 2024-02-26T10:41:59.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400eb60 con 0x7fdec8003f80 2024-02-26T10:41:59.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded401c400 con 0x7fdec8003f80 2024-02-26T10:41:59.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee41eb5f0 con 0x7fdec8003f80 2024-02-26T10:41:59.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.508+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee4152960 con 0x7fdec8003f80 2024-02-26T10:41:59.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.510+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee40a9d70 con 0x7fdec8003f80 2024-02-26T10:41:59.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.510+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fded401c5a0 con 0x7fdec8003f80 2024-02-26T10:41:59.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.511+0000 7fdee0c00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdec006b4d0 0x7fdec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.511+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fded408aa40 con 0x7fdec8003f80 2024-02-26T10:41:59.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.512+0000 7fdedb400640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdec006b4d0 0x7fdec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.513+0000 7fdedb400640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdec006b4d0 0x7fdec006d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdecc036000 tx=0x7fdecc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.516+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdee40a9d70 con 0x7fdec8003f80 2024-02-26T10:41:59.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.717+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdee4152470 con 0x7fdec8003f80 2024-02-26T10:41:59.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.721+0000 7fdee0c00640 1 -- 172.21.15.67:0/72172999 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fded4018030 con 0x7fdec8003f80 2024-02-26T10:41:59.719 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:41:59.720 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:58.714792+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:41:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdec006b4d0 msgr2=0x7fdec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdec006b4d0 0x7fdec006d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdecc036000 tx=0x7fdecc02e000 comp rx=0 tx=0).stop 2024-02-26T10:41:59.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 msgr2=0x7fdee40bfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdee40bfca0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fded40108e0 tx=0x7fded4010da0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 shutdown_connections 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdec006b4d0 0x7fdec006d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee41487d0 0x7fdee4145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee4147c20 0x7fdee4148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 --2- 172.21.15.67:0/72172999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdec8003f80 0x7fdee40bfca0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 >> 172.21.15.67:0/72172999 conn(0x7fdee40a84a0 msgr2=0x7fdee40c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 shutdown_connections 2024-02-26T10:41:59.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.724+0000 7fdeeac00640 1 -- 172.21.15.67:0/72172999 wait complete. 2024-02-26T10:41:59.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:41:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.959+0000 7f7ed7e00640 1 Processor -- start 2024-02-26T10:41:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.959+0000 7f7ed7e00640 1 -- start start 2024-02-26T10:41:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.960+0000 7f7ed7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ed806ac20 0x7f7ed80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7ed806b000 0x7f7ed806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed806b460 con 0x7f7ed8153610 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed80a9a80 con 0x7f7ed806b000 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed7e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed80a9d60 con 0x7f7ed806ac20 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7ed806b000 0x7f7ed806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7ed806b000 0x7f7ed806db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43354/0 (socket says 172.21.15.67:43354) 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed6000640 1 -- 172.21.15.67:0/840759580 learned_addr learned my addr 172.21.15.67:0/840759580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed7400640 1 --1- 172.21.15.67:0/840759580 >> v1:172.21.15.67:6789/0 conn(0x7f7ed8153610 0x7f7ed80abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56094/0 (socket says 172.21.15.67:56094) 2024-02-26T10:41:59.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.961+0000 7f7ed6a00640 1 --2- 172.21.15.67:0/840759580 >> v2:172.21.15.82:3300/0 conn(0x7f7ed806ac20 0x7f7ed80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.962+0000 7f7ed6000640 1 -- 172.21.15.67:0/840759580 >> v2:172.21.15.82:3300/0 conn(0x7f7ed806ac20 msgr2=0x7f7ed80a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:41:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.962+0000 7f7ed6000640 1 --2- 172.21.15.67:0/840759580 >> v2:172.21.15.82:3300/0 conn(0x7f7ed806ac20 0x7f7ed80a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.962+0000 7f7ed6000640 1 -- 172.21.15.67:0/840759580 >> v1:172.21.15.67:6789/0 conn(0x7f7ed8153610 legacy=0x7f7ed80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.962+0000 7f7ed6000640 1 -- 172.21.15.67:0/840759580 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80ab750 con 0x7f7ed806b000 2024-02-26T10:41:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.962+0000 7f7ed6a00640 1 --2- 172.21.15.67:0/840759580 >> v2:172.21.15.82:3300/0 conn(0x7f7ed806ac20 0x7f7ed80a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:41:59.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.963+0000 7f7ed6000640 1 --2- 172.21.15.67:0/840759580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806b000 0x7f7ed806db00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7ed806ecb0 tx=0x7f7ec402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cf6d221ace8f9a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.963+0000 7f7ed5600640 1 -- 172.21.15.67:0/840759580 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4034070 con 0x7f7ed806b000 2024-02-26T10:41:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.963+0000 7f7ed5600640 1 -- 172.21.15.67:0/840759580 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec4030870 con 0x7f7ed806b000 2024-02-26T10:41:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.963+0000 7f7ed5600640 1 -- 172.21.15.67:0/840759580 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4030c40 con 0x7f7ed806b000 2024-02-26T10:41:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 -- 172.21.15.67:0/840759580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806b000 msgr2=0x7f7ed806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/840759580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806b000 0x7f7ed806db00 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7ed806ecb0 tx=0x7f7ec402b9b0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 -- 172.21.15.67:0/840759580 shutdown_connections 2024-02-26T10:41:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/840759580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806b000 0x7f7ed806db00 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/840759580 >> v2:172.21.15.82:3300/0 conn(0x7f7ed806ac20 0x7f7ed80a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 -- 172.21.15.67:0/840759580 >> 172.21.15.67:0/840759580 conn(0x7f7ed80c63f0 msgr2=0x7f7ed80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:41:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 -- 172.21.15.67:0/840759580 shutdown_connections 2024-02-26T10:41:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.964+0000 7f7ed7e00640 1 -- 172.21.15.67:0/840759580 wait complete. 2024-02-26T10:41:59.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.965+0000 7f7ed7e00640 1 Processor -- start 2024-02-26T10:41:59.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 -- start start 2024-02-26T10:41:59.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 0x7f7ed8155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed806b000 0x7f7ed8151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ed8153610 0x7f7ed81ec090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8150410 con 0x7f7ed806ac20 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8153a60 con 0x7f7ed806b000 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.966+0000 7f7ed7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8150590 con 0x7f7ed8153610 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 0x7f7ed8155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 0x7f7ed8155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43356/0 (socket says 172.21.15.67:43356) 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 -- 172.21.15.67:0/3342417014 learned_addr learned my addr 172.21.15.67:0/3342417014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 -- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ed8153610 msgr2=0x7f7ed81ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ed8153610 0x7f7ed81ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 -- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed806b000 msgr2=0x7f7ed8151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6000640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed806b000 0x7f7ed8151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed806b000 0x7f7ed8151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.967+0000 7f7ed6a00640 1 -- 172.21.15.67:0/3342417014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80c2770 con 0x7f7ed806ac20 2024-02-26T10:41:59.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.968+0000 7f7ed6a00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 0x7f7ed8155280 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e7e0 tx=0x7f7ec800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.968+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800f040 con 0x7f7ed806ac20 2024-02-26T10:41:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.968+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8004030 con 0x7f7ed806ac20 2024-02-26T10:41:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.968+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8004400 con 0x7f7ed806ac20 2024-02-26T10:41:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.968+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed81521f0 con 0x7f7ed806ac20 2024-02-26T10:41:59.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.968+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed81514b0 con 0x7f7ed806ac20 2024-02-26T10:41:59.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.969+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed8074a30 con 0x7f7ed806ac20 2024-02-26T10:41:59.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.970+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ec800c6d0 con 0x7f7ed806ac20 2024-02-26T10:41:59.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.971+0000 7f7ecea00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eb806b4d0 0x7f7eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:41:59.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.971+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ec8089f30 con 0x7f7ed806ac20 2024-02-26T10:41:59.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.972+0000 7f7ed6000640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eb806b4d0 0x7f7eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:41:59.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.973+0000 7f7ed6000640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eb806b4d0 0x7f7eb806d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4036000 tx=0x7f7ec403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:41:59.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:41:59.975+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ec8017020 con 0x7f7ed806ac20 2024-02-26T10:42:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.179+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed81540a0 con 0x7f7ed806ac20 2024-02-26T10:42:00.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.181+0000 7f7ecea00640 1 -- 172.21.15.67:0/3342417014 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f7ec8058e40 con 0x7f7ed806ac20 2024-02-26T10:42:00.180 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:00.180 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:41:58.714792+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:00.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eb806b4d0 msgr2=0x7f7eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eb806b4d0 0x7f7eb806d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4036000 tx=0x7f7ec403b040 comp rx=0 tx=0).stop 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 msgr2=0x7f7ed8155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 0x7f7ed8155280 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e7e0 tx=0x7f7ec800eca0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 shutdown_connections 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ed8153610 0x7f7ed81ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7eb806b4d0 0x7f7eb806d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed806b000 0x7f7ed8151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 --2- 172.21.15.67:0/3342417014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ed806ac20 0x7f7ed8155280 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 >> 172.21.15.67:0/3342417014 conn(0x7f7ed80c63f0 msgr2=0x7f7ed80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 shutdown_connections 2024-02-26T10:42:00.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.184+0000 7f7ed7e00640 1 -- 172.21.15.67:0/3342417014 wait complete. 2024-02-26T10:42:00.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-26T10:42:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.431+0000 7f5be3e00640 1 Processor -- start 2024-02-26T10:42:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.431+0000 7f5be3e00640 1 -- start start 2024-02-26T10:42:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5be40a9430 0x7f5be40ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5be406e2d0 0x7f5be40c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be4150780 con 0x7f5be41503a0 2024-02-26T10:42:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be40a98c0 con 0x7f5be406e2d0 2024-02-26T10:42:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be40a9ba0 con 0x7f5be40a9430 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5be406e2d0 0x7f5be40c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5be406e2d0 0x7f5be40c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43370/0 (socket says 172.21.15.67:43370) 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 -- 172.21.15.67:0/3758437864 learned_addr learned my addr 172.21.15.67:0/3758437864 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 -- 172.21.15.67:0/3758437864 >> v2:172.21.15.82:3300/0 conn(0x7f5be40a9430 msgr2=0x7f5be40ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 --2- 172.21.15.67:0/3758437864 >> v2:172.21.15.82:3300/0 conn(0x7f5be40a9430 0x7f5be40ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 -- 172.21.15.67:0/3758437864 >> v1:172.21.15.82:6789/0 conn(0x7f5be41503a0 legacy=0x7f5be41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:00.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.433+0000 7f5be3400640 1 -- 172.21.15.67:0/3758437864 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be40a9e80 con 0x7f5be406e2d0 2024-02-26T10:42:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.435+0000 7f5be3400640 1 --2- 172.21.15.67:0/3758437864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be40c2e10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5be406f1c0 tx=0x7f5bdc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9228106b3ffe230a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.435+0000 7f5be1600640 1 -- 172.21.15.67:0/3758437864 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc02ca80 con 0x7f5be406e2d0 2024-02-26T10:42:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.435+0000 7f5be1600640 1 -- 172.21.15.67:0/3758437864 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc02cc20 con 0x7f5be406e2d0 2024-02-26T10:42:00.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.435+0000 7f5be1600640 1 -- 172.21.15.67:0/3758437864 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0343f0 con 0x7f5be406e2d0 2024-02-26T10:42:00.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.435+0000 7f5be3e00640 1 -- 172.21.15.67:0/3758437864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 msgr2=0x7f5be40c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.435+0000 7f5be3e00640 1 --2- 172.21.15.67:0/3758437864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be40c2e10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5be406f1c0 tx=0x7f5bdc02bee0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.436+0000 7f5be3e00640 1 -- 172.21.15.67:0/3758437864 shutdown_connections 2024-02-26T10:42:00.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.436+0000 7f5be3e00640 1 --2- 172.21.15.67:0/3758437864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be40c2e10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.436+0000 7f5be3e00640 1 --2- 172.21.15.67:0/3758437864 >> v2:172.21.15.82:3300/0 conn(0x7f5be40a9430 0x7f5be40ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.436+0000 7f5be3e00640 1 -- 172.21.15.67:0/3758437864 >> 172.21.15.67:0/3758437864 conn(0x7f5be40c6380 msgr2=0x7f5be40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.436+0000 7f5be3e00640 1 -- 172.21.15.67:0/3758437864 shutdown_connections 2024-02-26T10:42:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.436+0000 7f5be3e00640 1 -- 172.21.15.67:0/3758437864 wait complete. 2024-02-26T10:42:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.437+0000 7f5be3e00640 1 Processor -- start 2024-02-26T10:42:00.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.437+0000 7f5be3e00640 1 -- start start 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.438+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be41de620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.438+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5be41503a0 0x7f5be40715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.438+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5be4071b20 0x7f5be41edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.438+0000 7f5be3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be40c0f80 con 0x7f5be406e2d0 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.438+0000 7f5be3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be406ddb0 con 0x7f5be41503a0 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.438+0000 7f5be3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be406e720 con 0x7f5be4071b20 2024-02-26T10:42:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be41de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be41de620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43386/0 (socket says 172.21.15.67:43386) 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 -- 172.21.15.67:0/4055512611 learned_addr learned my addr 172.21.15.67:0/4055512611 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 -- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5be4071b20 msgr2=0x7f5be41edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5be4071b20 0x7f5be41edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 -- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5be41503a0 msgr2=0x7f5be40715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5be41503a0 0x7f5be40715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 -- 172.21.15.67:0/4055512611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be40c55c0 con 0x7f5be406e2d0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.439+0000 7f5be2a00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be41de620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b3f0 tx=0x7f5bd400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.440+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4015020 con 0x7f5be406e2d0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.440+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd4011830 con 0x7f5be406e2d0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.440+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4011c20 con 0x7f5be406e2d0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.440+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be41ee9e0 con 0x7f5be406e2d0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.440+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be40c3f70 con 0x7f5be406e2d0 2024-02-26T10:42:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.441+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be40678e0 con 0x7f5be406e2d0 2024-02-26T10:42:00.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.442+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bd4018020 con 0x7f5be406e2d0 2024-02-26T10:42:00.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.443+0000 7f5bdaa00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5bc406b4d0 0x7f5bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:00.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.443+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bd4089e50 con 0x7f5be406e2d0 2024-02-26T10:42:00.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.443+0000 7f5be2000640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5bc406b4d0 0x7f5bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:00.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.444+0000 7f5be2000640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5bc406b4d0 0x7f5bc406d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5be41570d0 tx=0x7f5bd0006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:00.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.448+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bd401f030 con 0x7f5be406e2d0 2024-02-26T10:42:00.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:00.687+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5be4074a30 con 0x7f5be406e2d0 2024-02-26T10:42:00.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:42:00.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:42:01.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.289+0000 7f5bdaa00640 1 -- 172.21.15.67:0/4055512611 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5bd4058cc0 con 0x7f5be406e2d0 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5bc406b4d0 msgr2=0x7f5bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5bc406b4d0 0x7f5bc406d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5be41570d0 tx=0x7f5bd0006c90 comp rx=0 tx=0).stop 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 msgr2=0x7f5be41de620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be41de620 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b3f0 tx=0x7f5bd400b8b0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 shutdown_connections 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5be4071b20 0x7f5be41edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5bc406b4d0 0x7f5bc406d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5be41503a0 0x7f5be40715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 --2- 172.21.15.67:0/4055512611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5be406e2d0 0x7f5be41de620 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.294+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 >> 172.21.15.67:0/4055512611 conn(0x7f5be40c6380 msgr2=0x7f5be40c24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:01.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.295+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 shutdown_connections 2024-02-26T10:42:01.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.295+0000 7f5be3e00640 1 -- 172.21.15.67:0/4055512611 wait complete. 2024-02-26T10:42:01.317 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:42:01.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-26T10:42:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.546+0000 7f4c28a00640 1 Processor -- start 2024-02-26T10:42:01.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.547+0000 7f4c28a00640 1 -- start start 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.548+0000 7f4c28a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c241523b0 0x7f4c240c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.548+0000 7f4c28a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c24154cf0 0x7f4c240c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.548+0000 7f4c28a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2406dc80 con 0x7f4c24071fd0 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.548+0000 7f4c28a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c24152790 con 0x7f4c24154cf0 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c28a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2406da30 con 0x7f4c241523b0 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c24154cf0 0x7f4c240c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4c24154cf0 0x7f4c240c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43392/0 (socket says 172.21.15.67:43392) 2024-02-26T10:42:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 -- 172.21.15.67:0/25054126 learned_addr learned my addr 172.21.15.67:0/25054126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c23400640 1 --2- 172.21.15.67:0/25054126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c241523b0 0x7f4c240c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 -- 172.21.15.67:0/25054126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c241523b0 msgr2=0x7f4c240c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 --2- 172.21.15.67:0/25054126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c241523b0 0x7f4c240c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 -- 172.21.15.67:0/25054126 >> v1:172.21.15.82:6789/0 conn(0x7f4c24071fd0 legacy=0x7f4c24072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.549+0000 7f4c1aa00640 1 -- 172.21.15.67:0/25054126 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2406df60 con 0x7f4c24154cf0 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.550+0000 7f4c1aa00640 1 --2- 172.21.15.67:0/25054126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c24154cf0 0x7f4c240c3130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009870 tx=0x7f4c0c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e98935f4b85cf53 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.551+0000 7f4c22a00640 1 -- 172.21.15.67:0/25054126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0317c0 con 0x7f4c24154cf0 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.551+0000 7f4c22a00640 1 -- 172.21.15.67:0/25054126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c037070 con 0x7f4c24154cf0 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.551+0000 7f4c22a00640 1 -- 172.21.15.67:0/25054126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c031b80 con 0x7f4c24154cf0 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.551+0000 7f4c28a00640 1 -- 172.21.15.67:0/25054126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c24154cf0 msgr2=0x7f4c240c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.551+0000 7f4c28a00640 1 --2- 172.21.15.67:0/25054126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c24154cf0 0x7f4c240c3130 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009870 tx=0x7f4c0c02b860 comp rx=0 tx=0).stop 2024-02-26T10:42:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.552+0000 7f4c28a00640 1 -- 172.21.15.67:0/25054126 shutdown_connections 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.552+0000 7f4c28a00640 1 --2- 172.21.15.67:0/25054126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c24154cf0 0x7f4c240c3130 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.552+0000 7f4c28a00640 1 --2- 172.21.15.67:0/25054126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c241523b0 0x7f4c240c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.552+0000 7f4c28a00640 1 -- 172.21.15.67:0/25054126 >> 172.21.15.67:0/25054126 conn(0x7f4c240c6380 msgr2=0x7f4c240c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.552+0000 7f4c28a00640 1 -- 172.21.15.67:0/25054126 shutdown_connections 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.552+0000 7f4c28a00640 1 -- 172.21.15.67:0/25054126 wait complete. 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.553+0000 7f4c28a00640 1 Processor -- start 2024-02-26T10:42:01.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.553+0000 7f4c28a00640 1 -- start start 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.554+0000 7f4c28a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c24071fd0 0x7f4c240c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.554+0000 7f4c28a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 0x7f4c241de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.554+0000 7f4c28a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c24154cf0 0x7f4c241ee220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.554+0000 7f4c28a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2406e800 con 0x7f4c241523b0 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.554+0000 7f4c28a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2406e520 con 0x7f4c24154cf0 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.554+0000 7f4c28a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240c08e0 con 0x7f4c24071fd0 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 0x7f4c241de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:01.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 0x7f4c241de4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43404/0 (socket says 172.21.15.67:43404) 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 -- 172.21.15.67:0/269886675 learned_addr learned my addr 172.21.15.67:0/269886675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c23400640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c24071fd0 0x7f4c240c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c23e00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c24154cf0 0x7f4c241ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 -- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c24071fd0 msgr2=0x7f4c240c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c24071fd0 0x7f4c240c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 -- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c24154cf0 msgr2=0x7f4c241ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c24154cf0 0x7f4c241ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 -- 172.21.15.67:0/269886675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240c53f0 con 0x7f4c241523b0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c23e00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c24154cf0 0x7f4c241ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.555+0000 7f4c1aa00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 0x7f4c241de4e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c02b8a0 tx=0x7f4c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.556+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c032860 con 0x7f4c241523b0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.556+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c032a20 con 0x7f4c241523b0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.556+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c032e10 con 0x7f4c241523b0 2024-02-26T10:42:01.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.556+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c241eda20 con 0x7f4c241523b0 2024-02-26T10:42:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.556+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241496d0 con 0x7f4c241523b0 2024-02-26T10:42:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.557+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c240678e0 con 0x7f4c241523b0 2024-02-26T10:42:01.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.558+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c0c04c060 con 0x7f4c241523b0 2024-02-26T10:42:01.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.559+0000 7f4c1be00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c0006b500 0x7f4c0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:01.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.559+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c0c0ab870 con 0x7f4c241523b0 2024-02-26T10:42:01.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.559+0000 7f4c23400640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c0006b500 0x7f4c0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:01.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.561+0000 7f4c23400640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c0006b500 0x7f4c0006d9c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4c14006fd0 tx=0x7f4c14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:01.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.564+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c0c040070 con 0x7f4c241523b0 2024-02-26T10:42:01.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:01.801+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4c24074a30 con 0x7f4c241523b0 2024-02-26T10:42:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.308+0000 7f4c1be00640 1 -- 172.21.15.67:0/269886675 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c0c07a780 con 0x7f4c241523b0 2024-02-26T10:42:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c0006b500 msgr2=0x7f4c0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c0006b500 0x7f4c0006d9c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4c14006fd0 tx=0x7f4c14008040 comp rx=0 tx=0).stop 2024-02-26T10:42:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 msgr2=0x7f4c241de4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 0x7f4c241de4e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c02b8a0 tx=0x7f4c0c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 shutdown_connections 2024-02-26T10:42:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4c0006b500 0x7f4c0006d9c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c24154cf0 0x7f4c241ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c241523b0 0x7f4c241de4e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 --2- 172.21.15.67:0/269886675 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c24071fd0 0x7f4c240c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.313+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 >> 172.21.15.67:0/269886675 conn(0x7f4c240c6380 msgr2=0x7f4c240c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.314+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 shutdown_connections 2024-02-26T10:42:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.314+0000 7f4c28a00640 1 -- 172.21.15.67:0/269886675 wait complete. 2024-02-26T10:42:02.335 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-26T10:42:02.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.578+0000 7f21ee600640 1 Processor -- start 2024-02-26T10:42:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.578+0000 7f21ee600640 1 -- start start 2024-02-26T10:42:02.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.580+0000 7f21ee600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21e80714d0 0x7f21e806bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.580+0000 7f21ee600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e8155970 con 0x7f21e80a61d0 2024-02-26T10:42:02.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.580+0000 7f21ee600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e8155c50 con 0x7f21e8072670 2024-02-26T10:42:02.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.580+0000 7f21ee600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e8071900 con 0x7f21e80714d0 2024-02-26T10:42:02.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.580+0000 7f21ed200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f21e80a61d0 0x7f21e80c5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56100/0 (socket says 172.21.15.67:56100) 2024-02-26T10:42:02.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.580+0000 7f21ed200640 1 -- 172.21.15.67:0/2387504016 learned_addr learned my addr 172.21.15.67:0/2387504016 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:02.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21edc00640 1 --2- 172.21.15.67:0/2387504016 >> v2:172.21.15.82:3300/0 conn(0x7f21e80714d0 0x7f21e806bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083545799 0 0) 0x7f21e8155970 con 0x7f21e80a61d0 2024-02-26T10:42:02.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d0003520 con 0x7f21e80a61d0 2024-02-26T10:42:02.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623412664 0 0) 0x7f21d0003520 con 0x7f21e80a61d0 2024-02-26T10:42:02.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 >> v2:172.21.15.82:3300/0 conn(0x7f21e80714d0 msgr2=0x7f21e806bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:02.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 --2- 172.21.15.67:0/2387504016 >> v2:172.21.15.82:3300/0 conn(0x7f21e80714d0 0x7f21e806bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 >> v1:172.21.15.82:6789/0 conn(0x7f21e8072670 legacy=0x7f21e80bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80c4d00 con 0x7f21e80a61d0 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f21dc001670 con 0x7f21e80a61d0 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.581+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 >> v1:172.21.15.67:6789/0 conn(0x7f21e80a61d0 legacy=0x7f21e80c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7400640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21d0003a00 0x7f21d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7400640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7400640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21d00052e0 0x7f21d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8071900 con 0x7f21d0004800 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d000c1e0 con 0x7f21d00052e0 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d000c3e0 con 0x7f21d0003a00 2024-02-26T10:42:02.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7e00640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21ed200640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21d0003a00 0x7f21d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21edc00640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21d00052e0 0x7f21d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7e00640 1 -- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21d0003a00 msgr2=0x7f21d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7e00640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21d0003a00 0x7f21d0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7e00640 1 -- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21d00052e0 msgr2=0x7f21d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7e00640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21d00052e0 0x7f21d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.582+0000 7f21e7e00640 1 -- 172.21.15.67:0/2387504016 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d0003520 con 0x7f21d0004800 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.583+0000 7f21edc00640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21d00052e0 0x7f21d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.583+0000 7f21edc00640 1 -- 172.21.15.67:0/2387504016 reap_dead start 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.583+0000 7f21ed200640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21d0003a00 0x7f21d0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.584+0000 7f21e7e00640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21d0004c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f21d4001750 tx=0x7f21d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5077f4a724df8669 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.584+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4001370 con 0x7f21d0004800 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.584+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d40052d0 con 0x7f21d0004800 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.584+0000 7f21e7400640 1 -- 172.21.15.67:0/2387504016 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4003830 con 0x7f21d0004800 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.584+0000 7f21ee600640 1 -- 172.21.15.67:0/2387504016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 msgr2=0x7f21d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.584+0000 7f21ee600640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21d0004c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f21d4001750 tx=0x7f21d0005830 comp rx=0 tx=0).stop 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.585+0000 7f21ee600640 1 -- 172.21.15.67:0/2387504016 shutdown_connections 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.585+0000 7f21ee600640 1 --2- 172.21.15.67:0/2387504016 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21d0004c80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.585+0000 7f21ee600640 1 -- 172.21.15.67:0/2387504016 >> 172.21.15.67:0/2387504016 conn(0x7f21e80c36b0 msgr2=0x7f21e80c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.585+0000 7f21ee600640 1 -- 172.21.15.67:0/2387504016 shutdown_connections 2024-02-26T10:42:02.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.585+0000 7f21ee600640 1 -- 172.21.15.67:0/2387504016 wait complete. 2024-02-26T10:42:02.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.587+0000 7f21ee600640 1 Processor -- start 2024-02-26T10:42:02.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.587+0000 7f21ee600640 1 -- start start 2024-02-26T10:42:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.588+0000 7f21ee600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21e81ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.588+0000 7f21ee600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e81dafd0 0x7f21e80aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.588+0000 7f21ee600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21e80ab0b0 0x7f21e814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.588+0000 7f21ee600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80721e0 con 0x7f21d0004800 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.588+0000 7f21ee600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80c4ab0 con 0x7f21e81dafd0 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.588+0000 7f21ee600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80a65b0 con 0x7f21e80ab0b0 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21e81ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21e81ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43418/0 (socket says 172.21.15.67:43418) 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 -- 172.21.15.67:0/1343838502 learned_addr learned my addr 172.21.15.67:0/1343838502 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21edc00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21e80ab0b0 0x7f21e814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21e7e00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e81dafd0 0x7f21e80aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 -- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21e80ab0b0 msgr2=0x7f21e814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21e80ab0b0 0x7f21e814df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 -- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e81dafd0 msgr2=0x7f21e80aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e81dafd0 0x7f21e80aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21ed200640 1 -- 172.21.15.67:0/1343838502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8153b70 con 0x7f21d0004800 2024-02-26T10:42:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21e7e00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e81dafd0 0x7f21e80aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.589+0000 7f21edc00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21e80ab0b0 0x7f21e814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.590+0000 7f21ed200640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21e81ee0d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00d7d0 tx=0x7f21dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.590+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc019070 con 0x7f21d0004800 2024-02-26T10:42:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.590+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc0072d0 con 0x7f21d0004800 2024-02-26T10:42:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.590+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0047e0 con 0x7f21d0004800 2024-02-26T10:42:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.590+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e81dba30 con 0x7f21d0004800 2024-02-26T10:42:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.590+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e81eb650 con 0x7f21d0004800 2024-02-26T10:42:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.591+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e80a7560 con 0x7f21d0004800 2024-02-26T10:42:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.592+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21dc004980 con 0x7f21d0004800 2024-02-26T10:42:02.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.594+0000 7f21e4c00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f21c806b4d0 0x7f21c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:02.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.594+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21dc089930 con 0x7f21d0004800 2024-02-26T10:42:02.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.594+0000 7f21e7e00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f21c806b4d0 0x7f21c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:02.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.595+0000 7f21e7e00640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f21c806b4d0 0x7f21c806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f21d4033000 tx=0x7f21d4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:02.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.598+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21e80a7560 con 0x7f21d0004800 2024-02-26T10:42:02.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.836+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e8071e50 con 0x7f21d0004800 2024-02-26T10:42:02.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.838+0000 7f21e4c00640 1 -- 172.21.15.67:0/1343838502 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f21e8071e50 con 0x7f21d0004800 2024-02-26T10:42:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:02.836 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6447,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-26T10:42:02.837 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 48 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f21c806b4d0 msgr2=0x7f21c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f21c806b4d0 0x7f21c806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f21d4033000 tx=0x7f21d4009030 comp rx=0 tx=0).stop 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 msgr2=0x7f21e81ee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21e81ee0d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00d7d0 tx=0x7f21dc00dc90 comp rx=0 tx=0).stop 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 shutdown_connections 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21e80ab0b0 0x7f21e814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f21c806b4d0 0x7f21c806d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e81dafd0 0x7f21e80aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 --2- 172.21.15.67:0/1343838502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21d0004800 0x7f21e81ee0d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.840+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 >> 172.21.15.67:0/1343838502 conn(0x7f21e80c36b0 msgr2=0x7f21e80a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.841+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 shutdown_connections 2024-02-26T10:42:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:02.841+0000 7f21ee600640 1 -- 172.21.15.67:0/1343838502 wait complete. 2024-02-26T10:42:02.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:03.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.079+0000 7fb53aa00640 1 Processor -- start 2024-02-26T10:42:03.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.079+0000 7fb53aa00640 1 -- start start 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.081+0000 7fb53aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb53406b000 0x7fb53406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.081+0000 7fb53aa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5340a9a80 con 0x7fb5340a94a0 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.081+0000 7fb53aa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5340a9d60 con 0x7fb53406ac20 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.081+0000 7fb53aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53406b490 con 0x7fb53406b000 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.082+0000 7fb538c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb5340a94a0 0x7fb5340abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56110/0 (socket says 172.21.15.67:56110) 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.082+0000 7fb538c00640 1 -- 172.21.15.67:0/3385583288 learned_addr learned my addr 172.21.15.67:0/3385583288 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:03.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.082+0000 7fb53a000640 1 --2- 172.21.15.67:0/3385583288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb53406b000 0x7fb53406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.082+0000 7fb533e00640 1 -- 172.21.15.67:0/3385583288 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662759884 0 0) 0x7fb5340a9a80 con 0x7fb5340a94a0 2024-02-26T10:42:03.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.082+0000 7fb533e00640 1 -- 172.21.15.67:0/3385583288 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb51c003520 con 0x7fb5340a94a0 2024-02-26T10:42:03.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.083+0000 7fb53a000640 1 -- 172.21.15.67:0/3385583288 >> v1:172.21.15.82:6789/0 conn(0x7fb53406ac20 legacy=0x7fb534153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.083+0000 7fb53a000640 1 -- 172.21.15.67:0/3385583288 >> v1:172.21.15.67:6789/0 conn(0x7fb5340a94a0 legacy=0x7fb5340abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.083+0000 7fb53a000640 1 -- 172.21.15.67:0/3385583288 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340ab750 con 0x7fb53406b000 2024-02-26T10:42:03.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.084+0000 7fb53a000640 1 --2- 172.21.15.67:0/3385583288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb53406b000 0x7fb53406db00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009dc0 tx=0x7fb52c02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b7f9dfee0deabb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:03.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb533e00640 1 -- 172.21.15.67:0/3385583288 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c039070 con 0x7fb53406b000 2024-02-26T10:42:03.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb533e00640 1 -- 172.21.15.67:0/3385583288 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c02ccc0 con 0x7fb53406b000 2024-02-26T10:42:03.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb533e00640 1 -- 172.21.15.67:0/3385583288 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c034640 con 0x7fb53406b000 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb53aa00640 1 -- 172.21.15.67:0/3385583288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb53406b000 msgr2=0x7fb53406db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb53aa00640 1 --2- 172.21.15.67:0/3385583288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb53406b000 0x7fb53406db00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009dc0 tx=0x7fb52c02bf10 comp rx=0 tx=0).stop 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb53aa00640 1 -- 172.21.15.67:0/3385583288 shutdown_connections 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb53aa00640 1 --2- 172.21.15.67:0/3385583288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb53406b000 0x7fb53406db00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb53aa00640 1 -- 172.21.15.67:0/3385583288 >> 172.21.15.67:0/3385583288 conn(0x7fb5340c63f0 msgr2=0x7fb5340c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.085+0000 7fb53aa00640 1 -- 172.21.15.67:0/3385583288 shutdown_connections 2024-02-26T10:42:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.086+0000 7fb53aa00640 1 -- 172.21.15.67:0/3385583288 wait complete. 2024-02-26T10:42:03.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.087+0000 7fb53aa00640 1 Processor -- start 2024-02-26T10:42:03.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.087+0000 7fb53aa00640 1 -- start start 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb53aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb53406ac20 0x7fb53414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb53aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 0x7fb5341edce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb53aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5340a94a0 0x7fb5341ee220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb53aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb534150590 con 0x7fb53406b000 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb53aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb534150410 con 0x7fb53406ac20 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb53aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53415e520 con 0x7fb5340a94a0 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb538c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 0x7fb5341edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb538c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 0x7fb5341edce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43426/0 (socket says 172.21.15.67:43426) 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.088+0000 7fb538c00640 1 -- 172.21.15.67:0/2319488727 learned_addr learned my addr 172.21.15.67:0/2319488727 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:03.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb539600640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb53406ac20 0x7fb53414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb53a000640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5340a94a0 0x7fb5341ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb538c00640 1 -- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5340a94a0 msgr2=0x7fb5341ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb538c00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5340a94a0 0x7fb5341ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb538c00640 1 -- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb53406ac20 msgr2=0x7fb53414bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb538c00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb53406ac20 0x7fb53414bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb538c00640 1 -- 172.21.15.67:0/2319488727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340726d0 con 0x7fb53406b000 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.089+0000 7fb538c00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 0x7fb5341edce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c440 tx=0x7fb52000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.090+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520017070 con 0x7fb53406b000 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.090+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb520005ce0 con 0x7fb53406b000 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.090+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5340729b0 con 0x7fb53406b000 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.090+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200072b0 con 0x7fb53406b000 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.090+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53414c510 con 0x7fb53406b000 2024-02-26T10:42:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.091+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb534074a30 con 0x7fb53406b000 2024-02-26T10:42:03.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.092+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb52001a070 con 0x7fb53406b000 2024-02-26T10:42:03.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.093+0000 7fb531600640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb51406b4d0 0x7fb51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.093+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb520089540 con 0x7fb53406b000 2024-02-26T10:42:03.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.094+0000 7fb539600640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb51406b4d0 0x7fb51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.095+0000 7fb539600640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb51406b4d0 0x7fb51406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb528007fd0 tx=0x7fb528009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:03.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.098+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5200584f0 con 0x7fb53406b000 2024-02-26T10:42:03.169 INFO:tasks.ceph.mds.a.smithi067.stderr:2024-02-26T10:42:03.171+0000 7fa307000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:42:03.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.289+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb53406b9b0 con 0x7fb53406b000 2024-02-26T10:42:03.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.291+0000 7fb531600640 1 -- 172.21.15.67:0/2319488727 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16588 (secure 0 0 0) 0x7fb520057df0 con 0x7fb53406b000 2024-02-26T10:42:03.290 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:03.290 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":76,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:00.271058+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb51406b4d0 msgr2=0x7fb51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb51406b4d0 0x7fb51406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb528007fd0 tx=0x7fb528009040 comp rx=0 tx=0).stop 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 msgr2=0x7fb5341edce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 0x7fb5341edce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c440 tx=0x7fb52000c900 comp rx=0 tx=0).stop 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 shutdown_connections 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5340a94a0 0x7fb5341ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb53406b000 0x7fb5341edce0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb51406b4d0 0x7fb51406d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 --2- 172.21.15.67:0/2319488727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb53406ac20 0x7fb53414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 >> 172.21.15.67:0/2319488727 conn(0x7fb5340c63f0 msgr2=0x7fb5340c2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 shutdown_connections 2024-02-26T10:42:03.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.294+0000 7fb53aa00640 1 -- 172.21.15.67:0/2319488727 wait complete. 2024-02-26T10:42:03.313 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:42:03.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.509+0000 7fd756e00640 1 Processor -- start 2024-02-26T10:42:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.509+0000 7fd756e00640 1 -- start start 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd75006ac20 0x7fd7500a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd75006db00 0x7fd7500abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75006b460 con 0x7fd75006b000 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7500a9a80 con 0x7fd75006db00 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7500a9d60 con 0x7fd75006ac20 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd75006db00 0x7fd7500abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd75006db00 0x7fd7500abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43440/0 (socket says 172.21.15.67:43440) 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd756400640 1 -- 172.21.15.67:0/2566569040 learned_addr learned my addr 172.21.15.67:0/2566569040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd755000640 1 --1- 172.21.15.67:0/2566569040 >> v1:172.21.15.67:6789/0 conn(0x7fd75006b000 0x7fd750153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56114/0 (socket says 172.21.15.67:56114) 2024-02-26T10:42:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.511+0000 7fd755a00640 1 --2- 172.21.15.67:0/2566569040 >> v2:172.21.15.82:3300/0 conn(0x7fd75006ac20 0x7fd7500a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.512+0000 7fd756400640 1 -- 172.21.15.67:0/2566569040 >> v2:172.21.15.82:3300/0 conn(0x7fd75006ac20 msgr2=0x7fd7500a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.512+0000 7fd756400640 1 --2- 172.21.15.67:0/2566569040 >> v2:172.21.15.82:3300/0 conn(0x7fd75006ac20 0x7fd7500a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.512+0000 7fd756400640 1 -- 172.21.15.67:0/2566569040 >> v1:172.21.15.67:6789/0 conn(0x7fd75006b000 legacy=0x7fd750153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.512+0000 7fd756400640 1 -- 172.21.15.67:0/2566569040 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75006df90 con 0x7fd75006db00 2024-02-26T10:42:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.513+0000 7fd756400640 1 --2- 172.21.15.67:0/2566569040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006db00 0x7fd7500abf50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd75006ecb0 tx=0x7fd74802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28737190c4b20aa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.513+0000 7fd74fe00640 1 -- 172.21.15.67:0/2566569040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748034070 con 0x7fd75006db00 2024-02-26T10:42:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.513+0000 7fd74fe00640 1 -- 172.21.15.67:0/2566569040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748030860 con 0x7fd75006db00 2024-02-26T10:42:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.513+0000 7fd74fe00640 1 -- 172.21.15.67:0/2566569040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748030c30 con 0x7fd75006db00 2024-02-26T10:42:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 -- 172.21.15.67:0/2566569040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006db00 msgr2=0x7fd7500abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 --2- 172.21.15.67:0/2566569040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006db00 0x7fd7500abf50 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd75006ecb0 tx=0x7fd74802b9a0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 -- 172.21.15.67:0/2566569040 shutdown_connections 2024-02-26T10:42:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 --2- 172.21.15.67:0/2566569040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006db00 0x7fd7500abf50 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 --2- 172.21.15.67:0/2566569040 >> v2:172.21.15.82:3300/0 conn(0x7fd75006ac20 0x7fd7500a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 -- 172.21.15.67:0/2566569040 >> 172.21.15.67:0/2566569040 conn(0x7fd7500c63f0 msgr2=0x7fd7500c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 -- 172.21.15.67:0/2566569040 shutdown_connections 2024-02-26T10:42:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.514+0000 7fd756e00640 1 -- 172.21.15.67:0/2566569040 wait complete. 2024-02-26T10:42:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.515+0000 7fd756e00640 1 Processor -- start 2024-02-26T10:42:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.516+0000 7fd756e00640 1 -- start start 2024-02-26T10:42:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.516+0000 7fd756e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 0x7fd750152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.516+0000 7fd756e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd75006b000 0x7fd7501ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd756e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75006db00 0x7fd7501ea870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd756e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7500ac490 con 0x7fd75006ac20 2024-02-26T10:42:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd756e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7500aba30 con 0x7fd75006db00 2024-02-26T10:42:03.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd756e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd750150410 con 0x7fd75006b000 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 0x7fd750152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 0x7fd750152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43454/0 (socket says 172.21.15.67:43454) 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 -- 172.21.15.67:0/1823093154 learned_addr learned my addr 172.21.15.67:0/1823093154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 -- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd75006b000 msgr2=0x7fd7501ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd75006b000 0x7fd7501ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 -- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75006db00 msgr2=0x7fd7501ea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75006db00 0x7fd7501ea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.517+0000 7fd755a00640 1 -- 172.21.15.67:0/1823093154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7501eed00 con 0x7fd75006ac20 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.518+0000 7fd755a00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 0x7fd750152c60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d830 tx=0x7fd74000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.518+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000b6c0 con 0x7fd75006ac20 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.518+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740019070 con 0x7fd75006ac20 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.518+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000ba80 con 0x7fd75006ac20 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.518+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd750072790 con 0x7fd75006ac20 2024-02-26T10:42:03.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.518+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7501edfc0 con 0x7fd75006ac20 2024-02-26T10:42:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.519+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd750074a30 con 0x7fd75006ac20 2024-02-26T10:42:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.520+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd74001e460 con 0x7fd75006ac20 2024-02-26T10:42:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.521+0000 7fd74d600640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd73006b4d0 0x7fd73006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.521+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd74008a8d0 con 0x7fd75006ac20 2024-02-26T10:42:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.522+0000 7fd755000640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd73006b4d0 0x7fd73006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:03.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.523+0000 7fd755000640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd73006b4d0 0x7fd73006d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd73c007fd0 tx=0x7fd73c007d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:03.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.526+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd740059740 con 0x7fd75006ac20 2024-02-26T10:42:03.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.720+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd750072cb0 con 0x7fd73006b4d0 2024-02-26T10:42:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.738+0000 7fd74d600640 1 -- 172.21.15.67:0/1823093154 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246057 (secure 0 0 0) 0x7fd750072cb0 con 0x7fd73006b4d0 2024-02-26T10:42:03.741 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:03.743 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd73006b4d0 msgr2=0x7fd73006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd73006b4d0 0x7fd73006d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd73c007fd0 tx=0x7fd73c007d80 comp rx=0 tx=0).stop 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 msgr2=0x7fd750152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 0x7fd750152c60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd74000d830 tx=0x7fd74000dcf0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 shutdown_connections 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd73006b4d0 0x7fd73006d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75006db00 0x7fd7501ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd75006b000 0x7fd7501ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 --2- 172.21.15.67:0/1823093154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd75006ac20 0x7fd750152c60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.744+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 >> 172.21.15.67:0/1823093154 conn(0x7fd7500c63f0 msgr2=0x7fd7500c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.745+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 shutdown_connections 2024-02-26T10:42:03.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:03.745+0000 7fd756e00640 1 -- 172.21.15.67:0/1823093154 wait complete. 2024-02-26T10:42:03.763 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-02-26T10:42:02.402903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245100,"kb_used_data":8136,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729620,"statfs":{"total":773094113280,"available":772843130880,"internally_reserved":0,"allocated":8331264,"data_stored":4338338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.175523"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.253763+0000","last_change":"2024-02-26T10:41:58.253763+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.253763+0000","last_undegraded":"2024-02-26T10:41:58.253763+0000","last_fullsized":"2024-02-26T10:41:58.253763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.251428+0000","last_change":"2024-02-26T10:41:58.251428+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.251428+0000","last_undegraded":"2024-02-26T10:41:58.251428+0000","last_fullsized":"2024-02-26T10:41:58.251428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259934+0000","last_change":"2024-02-26T10:41:58.259934+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259934+0000","last_undegraded":"2024-02-26T10:41:58.259934+0000","last_fullsized":"2024-02-26T10:41:58.259934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.261802+0000","last_change":"2024-02-26T10:41:58.261802+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.261802+0000","last_undegraded":"2024-02-26T10:41:58.261802+0000","last_fullsized":"2024-02-26T10:41:58.261802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.243320+0000","last_change":"2024-02-26T10:41:58.243320+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.243320+0000","last_undegraded":"2024-02-26T10:41:58.243320+0000","last_fullsized":"2024-02-26T10:41:58.243320+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240428+0000","last_change":"2024-02-26T10:41:58.240428+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240428+0000","last_undegraded":"2024-02-26T10:41:58.240428+0000","last_fullsized":"2024-02-26T10:41:58.240428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.245720+0000","last_change":"2024-02-26T10:41:58.245720+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.245720+0000","last_undegraded":"2024-02-26T10:41:58.245720+0000","last_fullsized":"2024-02-26T10:41:58.245720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.249820+0000","last_change":"2024-02-26T10:41:58.249820+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.249820+0000","last_undegraded":"2024-02-26T10:41:58.249820+0000","last_fullsized":"2024-02-26T10:41:58.249820+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250555+0000","last_change":"2024-02-26T10:41:58.250555+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250555+0000","last_undegraded":"2024-02-26T10:41:58.250555+0000","last_fullsized":"2024-02-26T10:41:58.250555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.254123+0000","last_change":"2024-02-26T10:41:58.254123+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.254123+0000","last_undegraded":"2024-02-26T10:41:58.254123+0000","last_fullsized":"2024-02-26T10:41:58.254123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259763+0000","last_change":"2024-02-26T10:41:58.259763+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259763+0000","last_undegraded":"2024-02-26T10:41:58.259763+0000","last_fullsized":"2024-02-26T10:41:58.259763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252106+0000","last_change":"2024-02-26T10:41:58.252106+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252106+0000","last_undegraded":"2024-02-26T10:41:58.252106+0000","last_fullsized":"2024-02-26T10:41:58.252106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252016+0000","last_change":"2024-02-26T10:41:58.252016+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252016+0000","last_undegraded":"2024-02-26T10:41:58.252016+0000","last_fullsized":"2024-02-26T10:41:58.252016+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259892+0000","last_change":"2024-02-26T10:41:58.259892+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259892+0000","last_undegraded":"2024-02-26T10:41:58.259892+0000","last_fullsized":"2024-02-26T10:41:58.259892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.239700+0000","last_change":"2024-02-26T10:41:58.239700+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.239700+0000","last_undegraded":"2024-02-26T10:41:58.239700+0000","last_fullsized":"2024-02-26T10:41:58.239700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.255189+0000","last_change":"2024-02-26T10:41:58.255189+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.255189+0000","last_undegraded":"2024-02-26T10:41:58.255189+0000","last_fullsized":"2024-02-26T10:41:58.255189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252328+0000","last_change":"2024-02-26T10:41:58.252328+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252328+0000","last_undegraded":"2024-02-26T10:41:58.252328+0000","last_fullsized":"2024-02-26T10:41:58.252328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250084+0000","last_change":"2024-02-26T10:41:58.250084+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250084+0000","last_undegraded":"2024-02-26T10:41:58.250084+0000","last_fullsized":"2024-02-26T10:41:58.250084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.257717+0000","last_change":"2024-02-26T10:41:58.257717+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.257717+0000","last_undegraded":"2024-02-26T10:41:58.257717+0000","last_fullsized":"2024-02-26T10:41:58.257717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.263260+0000","last_change":"2024-02-26T10:41:58.263260+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.263260+0000","last_undegraded":"2024-02-26T10:41:58.263260+0000","last_fullsized":"2024-02-26T10:41:58.263260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.251233+0000","last_change":"2024-02-26T10:41:58.251233+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.251233+0000","last_undegraded":"2024-02-26T10:41:58.251233+0000","last_fullsized":"2024-02-26T10:41:58.251233+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248998+0000","last_change":"2024-02-26T10:41:58.248998+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248998+0000","last_undegraded":"2024-02-26T10:41:58.248998+0000","last_fullsized":"2024-02-26T10:41:58.248998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.253518+0000","last_change":"2024-02-26T10:41:58.253518+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.253518+0000","last_undegraded":"2024-02-26T10:41:58.253518+0000","last_fullsized":"2024-02-26T10:41:58.253518+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.262162+0000","last_change":"2024-02-26T10:41:58.262162+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.262162+0000","last_undegraded":"2024-02-26T10:41:58.262162+0000","last_fullsized":"2024-02-26T10:41:58.262162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.253532+0000","last_change":"2024-02-26T10:41:58.253532+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.253532+0000","last_undegraded":"2024-02-26T10:41:58.253532+0000","last_fullsized":"2024-02-26T10:41:58.253532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.265419+0000","last_change":"2024-02-26T10:41:58.265419+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.265419+0000","last_undegraded":"2024-02-26T10:41:58.265419+0000","last_fullsized":"2024-02-26T10:41:58.265419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275701+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:41:58.275701+0000","last_peered":"2024-02-26T10:41:58.275701+0000","last_clean":"2024-02-26T10:41:58.275701+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:41:58.275701+0000","last_undegraded":"2024-02-26T10:41:58.275701+0000","last_fullsized":"2024-02-26T10:41:58.275701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:06.257992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252415+0000","last_change":"2024-02-26T10:41:58.252415+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252415+0000","last_undegraded":"2024-02-26T10:41:58.252415+0000","last_fullsized":"2024-02-26T10:41:58.252415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.272930+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:41:58.272930+0000","last_peered":"2024-02-26T10:41:58.272930+0000","last_clean":"2024-02-26T10:41:58.272930+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:41:58.272930+0000","last_undegraded":"2024-02-26T10:41:58.272930+0000","last_fullsized":"2024-02-26T10:41:58.272930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:13:21.433177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250084+0000","last_change":"2024-02-26T10:41:58.250084+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250084+0000","last_undegraded":"2024-02-26T10:41:58.250084+0000","last_fullsized":"2024-02-26T10:41:58.250084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.279118+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:41:58.279118+0000","last_peered":"2024-02-26T10:41:58.279118+0000","last_clean":"2024-02-26T10:41:58.279118+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:41:58.279118+0000","last_undegraded":"2024-02-26T10:41:58.279118+0000","last_fullsized":"2024-02-26T10:41:58.279118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:29.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259368+0000","last_change":"2024-02-26T10:41:58.259368+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259368+0000","last_undegraded":"2024-02-26T10:41:58.259368+0000","last_fullsized":"2024-02-26T10:41:58.259368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.267920+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:41:58.267920+0000","last_peered":"2024-02-26T10:41:58.267920+0000","last_clean":"2024-02-26T10:41:58.267920+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:41:58.267920+0000","last_undegraded":"2024-02-26T10:41:58.267920+0000","last_fullsized":"2024-02-26T10:41:58.267920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252720+0000","last_change":"2024-02-26T10:41:58.252720+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252720+0000","last_undegraded":"2024-02-26T10:41:58.252720+0000","last_fullsized":"2024-02-26T10:41:58.252720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.274664+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:41:58.274664+0000","last_peered":"2024-02-26T10:41:58.274664+0000","last_clean":"2024-02-26T10:41:58.274664+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:41:58.274664+0000","last_undegraded":"2024-02-26T10:41:58.274664+0000","last_fullsized":"2024-02-26T10:41:58.274664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:07.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.258754+0000","last_change":"2024-02-26T10:41:58.258754+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.258754+0000","last_undegraded":"2024-02-26T10:41:58.258754+0000","last_fullsized":"2024-02-26T10:41:58.258754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.274360+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:41:58.274360+0000","last_peered":"2024-02-26T10:41:58.274360+0000","last_clean":"2024-02-26T10:41:58.274360+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:41:58.274360+0000","last_undegraded":"2024-02-26T10:41:58.274360+0000","last_fullsized":"2024-02-26T10:41:58.274360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:05.753103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.264915+0000","last_change":"2024-02-26T10:41:58.264915+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.264915+0000","last_undegraded":"2024-02-26T10:41:58.264915+0000","last_fullsized":"2024-02-26T10:41:58.264915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.257318+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:41:58.257318+0000","last_peered":"2024-02-26T10:41:58.257318+0000","last_clean":"2024-02-26T10:41:58.257318+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:41:58.257318+0000","last_undegraded":"2024-02-26T10:41:58.257318+0000","last_fullsized":"2024-02-26T10:41:58.257318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.246031+0000","last_change":"2024-02-26T10:41:58.246031+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.246031+0000","last_undegraded":"2024-02-26T10:41:58.246031+0000","last_fullsized":"2024-02-26T10:41:58.246031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.269260+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:41:58.269260+0000","last_peered":"2024-02-26T10:41:58.269260+0000","last_clean":"2024-02-26T10:41:58.269260+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:41:58.269260+0000","last_undegraded":"2024-02-26T10:41:58.269260+0000","last_fullsized":"2024-02-26T10:41:58.269260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:28:46.080492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248409+0000","last_change":"2024-02-26T10:41:58.248409+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248409+0000","last_undegraded":"2024-02-26T10:41:58.248409+0000","last_fullsized":"2024-02-26T10:41:58.248409+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.274711+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:41:58.274711+0000","last_peered":"2024-02-26T10:41:58.274711+0000","last_clean":"2024-02-26T10:41:58.274711+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:41:58.274711+0000","last_undegraded":"2024-02-26T10:41:58.274711+0000","last_fullsized":"2024-02-26T10:41:58.274711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:20.751081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.241094+0000","last_change":"2024-02-26T10:41:58.241094+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.241094+0000","last_undegraded":"2024-02-26T10:41:58.241094+0000","last_fullsized":"2024-02-26T10:41:58.241094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276127+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:41:58.276127+0000","last_peered":"2024-02-26T10:41:58.276127+0000","last_clean":"2024-02-26T10:41:58.276127+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:41:58.276127+0000","last_undegraded":"2024-02-26T10:41:58.276127+0000","last_fullsized":"2024-02-26T10:41:58.276127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:17.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.241880+0000","last_change":"2024-02-26T10:41:58.241880+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.241880+0000","last_undegraded":"2024-02-26T10:41:58.241880+0000","last_fullsized":"2024-02-26T10:41:58.241880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276143+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:41:58.276143+0000","last_peered":"2024-02-26T10:41:58.276143+0000","last_clean":"2024-02-26T10:41:58.276143+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:41:58.276143+0000","last_undegraded":"2024-02-26T10:41:58.276143+0000","last_fullsized":"2024-02-26T10:41:58.276143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:36.807502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.262725+0000","last_change":"2024-02-26T10:41:58.262725+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.262725+0000","last_undegraded":"2024-02-26T10:41:58.262725+0000","last_fullsized":"2024-02-26T10:41:58.262725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.265808+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:41:58.265808+0000","last_peered":"2024-02-26T10:41:58.265808+0000","last_clean":"2024-02-26T10:41:58.265808+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:41:58.265808+0000","last_undegraded":"2024-02-26T10:41:58.265808+0000","last_fullsized":"2024-02-26T10:41:58.265808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:52.549647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240762+0000","last_change":"2024-02-26T10:41:58.240762+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240762+0000","last_undegraded":"2024-02-26T10:41:58.240762+0000","last_fullsized":"2024-02-26T10:41:58.240762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.262504+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:41:58.262504+0000","last_peered":"2024-02-26T10:41:58.262504+0000","last_clean":"2024-02-26T10:41:58.262504+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:41:58.262504+0000","last_undegraded":"2024-02-26T10:41:58.262504+0000","last_fullsized":"2024-02-26T10:41:58.262504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:10.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.256927+0000","last_change":"2024-02-26T10:41:58.256927+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.256927+0000","last_undegraded":"2024-02-26T10:41:58.256927+0000","last_fullsized":"2024-02-26T10:41:58.256927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.268048+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:41:58.268048+0000","last_peered":"2024-02-26T10:41:58.268048+0000","last_clean":"2024-02-26T10:41:58.268048+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:41:58.268048+0000","last_undegraded":"2024-02-26T10:41:58.268048+0000","last_fullsized":"2024-02-26T10:41:58.268048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:35.521987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248891+0000","last_change":"2024-02-26T10:41:58.248891+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248891+0000","last_undegraded":"2024-02-26T10:41:58.248891+0000","last_fullsized":"2024-02-26T10:41:58.248891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.271079+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:41:58.271079+0000","last_peered":"2024-02-26T10:41:58.271079+0000","last_clean":"2024-02-26T10:41:58.271079+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:41:58.271079+0000","last_undegraded":"2024-02-26T10:41:58.271079+0000","last_fullsized":"2024-02-26T10:41:58.271079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:29.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248770+0000","last_change":"2024-02-26T10:41:58.248770+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248770+0000","last_undegraded":"2024-02-26T10:41:58.248770+0000","last_fullsized":"2024-02-26T10:41:58.248770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.278785+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:41:58.278785+0000","last_peered":"2024-02-26T10:41:58.278785+0000","last_clean":"2024-02-26T10:41:58.278785+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:41:58.278785+0000","last_undegraded":"2024-02-26T10:41:58.278785+0000","last_fullsized":"2024-02-26T10:41:58.278785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:52.549647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.255749+0000","last_change":"2024-02-26T10:41:58.255749+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.255749+0000","last_undegraded":"2024-02-26T10:41:58.255749+0000","last_fullsized":"2024-02-26T10:41:58.255749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.269405+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:41:58.269405+0000","last_peered":"2024-02-26T10:41:58.269405+0000","last_clean":"2024-02-26T10:41:58.269405+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:41:58.269405+0000","last_undegraded":"2024-02-26T10:41:58.269405+0000","last_fullsized":"2024-02-26T10:41:58.269405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:09.674799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250318+0000","last_change":"2024-02-26T10:41:58.250318+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250318+0000","last_undegraded":"2024-02-26T10:41:58.250318+0000","last_fullsized":"2024-02-26T10:41:58.250318+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275638+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:41:58.275638+0000","last_peered":"2024-02-26T10:41:58.275638+0000","last_clean":"2024-02-26T10:41:58.275638+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:41:58.275638+0000","last_undegraded":"2024-02-26T10:41:58.275638+0000","last_fullsized":"2024-02-26T10:41:58.275638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.262070+0000","last_change":"2024-02-26T10:41:58.262070+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.262070+0000","last_undegraded":"2024-02-26T10:41:58.262070+0000","last_fullsized":"2024-02-26T10:41:58.262070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.267839+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:41:58.267839+0000","last_peered":"2024-02-26T10:41:58.267839+0000","last_clean":"2024-02-26T10:41:58.267839+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:41:58.267839+0000","last_undegraded":"2024-02-26T10:41:58.267839+0000","last_fullsized":"2024-02-26T10:41:58.267839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.279305+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:41:58.279305+0000","last_peered":"2024-02-26T10:41:58.279305+0000","last_clean":"2024-02-26T10:41:58.279305+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:41:58.279305+0000","last_undegraded":"2024-02-26T10:41:58.279305+0000","last_fullsized":"2024-02-26T10:41:58.279305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:09.156750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240461+0000","last_change":"2024-02-26T10:41:58.240461+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240461+0000","last_undegraded":"2024-02-26T10:41:58.240461+0000","last_fullsized":"2024-02-26T10:41:58.240461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-02-26T10:41:57.235808+0000","last_change":"2024-02-26T10:41:57.235808+0000","last_active":"2024-02-26T10:41:57.210128+0000","last_peered":"2024-02-26T10:41:57.210128+0000","last_clean":"2024-02-26T10:41:57.210128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:57.235808+0000","last_undegraded":"2024-02-26T10:41:57.235808+0000","last_fullsized":"2024-02-26T10:41:57.235808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-02-26T10:41:57.235824+0000","last_change":"2024-02-26T10:41:57.235824+0000","last_active":"2024-02-26T10:41:57.210128+0000","last_peered":"2024-02-26T10:41:57.210128+0000","last_clean":"2024-02-26T10:41:57.210128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:57.235824+0000","last_undegraded":"2024-02-26T10:41:57.235824+0000","last_fullsized":"2024-02-26T10:41:57.235824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.241929+0000","last_change":"2024-02-26T10:41:58.241929+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.241929+0000","last_undegraded":"2024-02-26T10:41:58.241929+0000","last_fullsized":"2024-02-26T10:41:58.241929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275663+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:41:58.275663+0000","last_peered":"2024-02-26T10:41:58.275663+0000","last_clean":"2024-02-26T10:41:58.275663+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:41:58.275663+0000","last_undegraded":"2024-02-26T10:41:58.275663+0000","last_fullsized":"2024-02-26T10:41:58.275663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:26.142556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.257874+0000","last_change":"2024-02-26T10:41:58.257874+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.257874+0000","last_undegraded":"2024-02-26T10:41:58.257874+0000","last_fullsized":"2024-02-26T10:41:58.257874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276247+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:41:58.276247+0000","last_peered":"2024-02-26T10:41:58.276247+0000","last_clean":"2024-02-26T10:41:58.276247+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:41:58.276247+0000","last_undegraded":"2024-02-26T10:41:58.276247+0000","last_fullsized":"2024-02-26T10:41:58.276247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:04.024496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.256592+0000","last_change":"2024-02-26T10:41:58.256592+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.256592+0000","last_undegraded":"2024-02-26T10:41:58.256592+0000","last_fullsized":"2024-02-26T10:41:58.256592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.256856+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:41:58.256856+0000","last_peered":"2024-02-26T10:41:58.256856+0000","last_clean":"2024-02-26T10:41:58.256856+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:41:58.256856+0000","last_undegraded":"2024-02-26T10:41:58.256856+0000","last_fullsized":"2024-02-26T10:41:58.256856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:24:48.243827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248650+0000","last_change":"2024-02-26T10:41:58.248650+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248650+0000","last_undegraded":"2024-02-26T10:41:58.248650+0000","last_fullsized":"2024-02-26T10:41:58.248650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275629+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:41:58.275629+0000","last_peered":"2024-02-26T10:41:58.275629+0000","last_clean":"2024-02-26T10:41:58.275629+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:41:58.275629+0000","last_undegraded":"2024-02-26T10:41:58.275629+0000","last_fullsized":"2024-02-26T10:41:58.275629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:55.395006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.257910+0000","last_change":"2024-02-26T10:41:58.257910+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.257910+0000","last_undegraded":"2024-02-26T10:41:58.257910+0000","last_fullsized":"2024-02-26T10:41:58.257910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.269326+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:41:58.269326+0000","last_peered":"2024-02-26T10:41:58.269326+0000","last_clean":"2024-02-26T10:41:58.269326+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:41:58.269326+0000","last_undegraded":"2024-02-26T10:41:58.269326+0000","last_fullsized":"2024-02-26T10:41:58.269326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:37.043645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240528+0000","last_change":"2024-02-26T10:41:58.240528+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240528+0000","last_undegraded":"2024-02-26T10:41:58.240528+0000","last_fullsized":"2024-02-26T10:41:58.240528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.278743+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:41:58.278743+0000","last_peered":"2024-02-26T10:41:58.278743+0000","last_clean":"2024-02-26T10:41:58.278743+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:41:58.278743+0000","last_undegraded":"2024-02-26T10:41:58.278743+0000","last_fullsized":"2024-02-26T10:41:58.278743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.239566+0000","last_change":"2024-02-26T10:41:58.239566+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.239566+0000","last_undegraded":"2024-02-26T10:41:58.239566+0000","last_fullsized":"2024-02-26T10:41:58.239566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.271122+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:41:58.271122+0000","last_peered":"2024-02-26T10:41:58.271122+0000","last_clean":"2024-02-26T10:41:58.271122+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:41:58.271122+0000","last_undegraded":"2024-02-26T10:41:58.271122+0000","last_fullsized":"2024-02-26T10:41:58.271122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.263699+0000","last_change":"2024-02-26T10:41:58.263699+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.263699+0000","last_undegraded":"2024-02-26T10:41:58.263699+0000","last_fullsized":"2024-02-26T10:41:58.263699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.267511+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:41:58.267511+0000","last_peered":"2024-02-26T10:41:58.267511+0000","last_clean":"2024-02-26T10:41:58.267511+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:41:58.267511+0000","last_undegraded":"2024-02-26T10:41:58.267511+0000","last_fullsized":"2024-02-26T10:41:58.267511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:12:16.519035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.251246+0000","last_change":"2024-02-26T10:41:58.251246+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.251246+0000","last_undegraded":"2024-02-26T10:41:58.251246+0000","last_fullsized":"2024-02-26T10:41:58.251246+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276875+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:41:58.276875+0000","last_peered":"2024-02-26T10:41:58.276875+0000","last_clean":"2024-02-26T10:41:58.276875+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:41:58.276875+0000","last_undegraded":"2024-02-26T10:41:58.276875+0000","last_fullsized":"2024-02-26T10:41:58.276875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:54.837264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248807+0000","last_change":"2024-02-26T10:41:58.248807+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248807+0000","last_undegraded":"2024-02-26T10:41:58.248807+0000","last_fullsized":"2024-02-26T10:41:58.248807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275381+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:41:58.275381+0000","last_peered":"2024-02-26T10:41:58.275381+0000","last_clean":"2024-02-26T10:41:58.275381+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:41:58.275381+0000","last_undegraded":"2024-02-26T10:41:58.275381+0000","last_fullsized":"2024-02-26T10:41:58.275381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":270,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.266978+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:41:58.266978+0000","last_peered":"2024-02-26T10:41:58.266978+0000","last_clean":"2024-02-26T10:41:58.266978+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:41:58.266978+0000","last_undegraded":"2024-02-26T10:41:58.266978+0000","last_fullsized":"2024-02-26T10:41:58.266978+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":985011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":905216,"data_stored":400918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":388826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1486848,"data_stored":985011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:03.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.019+0000 7f8cb7000640 1 Processor -- start 2024-02-26T10:42:04.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.019+0000 7f8cb7000640 1 -- start start 2024-02-26T10:42:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.020+0000 7f8cb7000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 0x7f8cb00c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.020+0000 7f8cb7000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb0152760 con 0x7f8cb006d290 2024-02-26T10:42:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.020+0000 7f8cb7000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb00c7550 con 0x7f8cb0152380 2024-02-26T10:42:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.020+0000 7f8cb7000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00c7830 con 0x7f8cb00c7020 2024-02-26T10:42:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.021+0000 7f8cb6600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8cb006d290 0x7f8cb00c4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56116/0 (socket says 172.21.15.67:56116) 2024-02-26T10:42:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.021+0000 7f8cb6600640 1 -- 172.21.15.67:0/4016670019 learned_addr learned my addr 172.21.15.67:0/4016670019 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:04.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.021+0000 7f8cb5200640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 0x7f8cb00c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.021+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169803386 0 0) 0x7f8cb0152760 con 0x7f8cb006d290 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.021+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c98003520 con 0x7f8cb006d290 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730159724 0 0) 0x7f8cb00c7550 con 0x7f8cb0152380 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb0152760 con 0x7f8cb0152380 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594037573 0 0) 0x7f8c98003520 con 0x7f8cb006d290 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 msgr2=0x7f8cb00c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 0x7f8cb00c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 >> v1:172.21.15.82:6789/0 conn(0x7f8cb0152380 legacy=0x7f8cb0155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb006ca90 con 0x7f8cb006d290 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8ca8001670 con 0x7f8cb006d290 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 >> v1:172.21.15.67:6789/0 conn(0x7f8cb006d290 legacy=0x7f8cb00c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8c98003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c98004800 0x7f8c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 0x7f8c98005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00c7830 con 0x7f8c98003b70 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800bb50 con 0x7f8c98004800 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800bd20 con 0x7f8cb00c7020 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.022+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8c98003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb6600640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 0x7f8c98005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb5200640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c98004800 0x7f8c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb5c00640 1 -- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 msgr2=0x7f8c98005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00c7020 0x7f8c98005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb5c00640 1 -- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c98004800 msgr2=0x7f8c98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c98004800 0x7f8c98004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.023+0000 7f8cb5c00640 1 -- 172.21.15.67:0/4016670019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98003520 con 0x7f8c98003b70 2024-02-26T10:42:04.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.024+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8c98003f70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4002000 tx=0x7f8c98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda0478b54f4e71e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.024+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4001bf0 con 0x7f8c98003b70 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.024+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4003b70 con 0x7f8c98003b70 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.024+0000 7f8cafe00640 1 -- 172.21.15.67:0/4016670019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40050d0 con 0x7f8c98003b70 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb7000640 1 -- 172.21.15.67:0/4016670019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 msgr2=0x7f8c98003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb7000640 1 --2- 172.21.15.67:0/4016670019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8c98003f70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4002000 tx=0x7f8c98005830 comp rx=0 tx=0).stop 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb6600640 1 -- 172.21.15.67:0/4016670019 reap_dead start 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb7000640 1 -- 172.21.15.67:0/4016670019 shutdown_connections 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb7000640 1 -- 172.21.15.67:0/4016670019 >> 172.21.15.67:0/4016670019 conn(0x7f8cb00ac1a0 msgr2=0x7f8cb00ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb7000640 1 -- 172.21.15.67:0/4016670019 shutdown_connections 2024-02-26T10:42:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.025+0000 7f8cb7000640 1 -- 172.21.15.67:0/4016670019 wait complete. 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.027+0000 7f8cb7000640 1 Processor -- start 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.027+0000 7f8cb7000640 1 -- start start 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb7000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8cb00c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb7000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00bf4f0 0x7f8cb01eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb7000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb00aaed0 0x7f8cb0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb7000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00c5820 con 0x7f8c98003b70 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb7000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00c56a0 con 0x7f8cb00aaed0 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb7000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb006d720 con 0x7f8cb00bf4f0 2024-02-26T10:42:04.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8cb00c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8cb00c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43468/0 (socket says 172.21.15.67:43468) 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.028+0000 7f8cb5c00640 1 -- 172.21.15.67:0/3156015515 learned_addr learned my addr 172.21.15.67:0/3156015515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5c00640 1 -- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00bf4f0 msgr2=0x7f8cb01eaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5200640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00bf4f0 0x7f8cb01eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb6600640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb00aaed0 0x7f8cb0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00bf4f0 0x7f8cb01eaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5c00640 1 -- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb00aaed0 msgr2=0x7f8cb0157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb00aaed0 0x7f8cb0157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5c00640 1 -- 172.21.15.67:0/3156015515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0158b50 con 0x7f8c98003b70 2024-02-26T10:42:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb5c00640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8cb00c0d50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4002000 tx=0x7f8ca4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4039040 con 0x7f8c98003b70 2024-02-26T10:42:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca4042070 con 0x7f8c98003b70 2024-02-26T10:42:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb0158e30 con 0x7f8c98003b70 2024-02-26T10:42:04.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.029+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4001bf0 con 0x7f8c98003b70 2024-02-26T10:42:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.030+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb00be670 con 0x7f8c98003b70 2024-02-26T10:42:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.031+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb0074a30 con 0x7f8c98003b70 2024-02-26T10:42:04.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.032+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ca402c570 con 0x7f8c98003b70 2024-02-26T10:42:04.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.032+0000 7f8cad600640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8c9006b4d0 0x7f8c9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.033+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ca40abad0 con 0x7f8c98003b70 2024-02-26T10:42:04.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.033+0000 7f8cb5200640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8c9006b4d0 0x7f8c9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.034+0000 7f8cb5200640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8c9006b4d0 0x7f8c9006d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00c4bf0 tx=0x7f8c9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:04.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.037+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ca407a940 con 0x7f8c98003b70 2024-02-26T10:42:04.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.233+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cb0155ee0 con 0x7f8c9006b4d0 2024-02-26T10:42:04.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.251+0000 7f8cad600640 1 -- 172.21.15.67:0/3156015515 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246057 (secure 0 0 0) 0x7f8cb0155ee0 con 0x7f8c9006b4d0 2024-02-26T10:42:04.254 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:04.255 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8c9006b4d0 msgr2=0x7f8c9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8c9006b4d0 0x7f8c9006d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00c4bf0 tx=0x7f8c9c008040 comp rx=0 tx=0).stop 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 msgr2=0x7f8cb00c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8cb00c0d50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4002000 tx=0x7f8ca4001650 comp rx=0 tx=0).stop 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 shutdown_connections 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8c9006b4d0 0x7f8c9006d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb00aaed0 0x7f8cb0157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cb00bf4f0 0x7f8cb01eaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 --2- 172.21.15.67:0/3156015515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c98003b70 0x7f8cb00c0d50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 >> 172.21.15.67:0/3156015515 conn(0x7f8cb00ac1a0 msgr2=0x7f8cb00c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.257+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 shutdown_connections 2024-02-26T10:42:04.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.258+0000 7f8cb7000640 1 -- 172.21.15.67:0/3156015515 wait complete. 2024-02-26T10:42:04.274 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-02-26T10:42:02.402903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245100,"kb_used_data":8136,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729620,"statfs":{"total":773094113280,"available":772843130880,"internally_reserved":0,"allocated":8331264,"data_stored":4338338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.175523"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.253763+0000","last_change":"2024-02-26T10:41:58.253763+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.253763+0000","last_undegraded":"2024-02-26T10:41:58.253763+0000","last_fullsized":"2024-02-26T10:41:58.253763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.251428+0000","last_change":"2024-02-26T10:41:58.251428+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.251428+0000","last_undegraded":"2024-02-26T10:41:58.251428+0000","last_fullsized":"2024-02-26T10:41:58.251428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259934+0000","last_change":"2024-02-26T10:41:58.259934+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259934+0000","last_undegraded":"2024-02-26T10:41:58.259934+0000","last_fullsized":"2024-02-26T10:41:58.259934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.261802+0000","last_change":"2024-02-26T10:41:58.261802+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.261802+0000","last_undegraded":"2024-02-26T10:41:58.261802+0000","last_fullsized":"2024-02-26T10:41:58.261802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.243320+0000","last_change":"2024-02-26T10:41:58.243320+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.243320+0000","last_undegraded":"2024-02-26T10:41:58.243320+0000","last_fullsized":"2024-02-26T10:41:58.243320+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240428+0000","last_change":"2024-02-26T10:41:58.240428+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240428+0000","last_undegraded":"2024-02-26T10:41:58.240428+0000","last_fullsized":"2024-02-26T10:41:58.240428+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.245720+0000","last_change":"2024-02-26T10:41:58.245720+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.245720+0000","last_undegraded":"2024-02-26T10:41:58.245720+0000","last_fullsized":"2024-02-26T10:41:58.245720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.249820+0000","last_change":"2024-02-26T10:41:58.249820+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.249820+0000","last_undegraded":"2024-02-26T10:41:58.249820+0000","last_fullsized":"2024-02-26T10:41:58.249820+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250555+0000","last_change":"2024-02-26T10:41:58.250555+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250555+0000","last_undegraded":"2024-02-26T10:41:58.250555+0000","last_fullsized":"2024-02-26T10:41:58.250555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.254123+0000","last_change":"2024-02-26T10:41:58.254123+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.254123+0000","last_undegraded":"2024-02-26T10:41:58.254123+0000","last_fullsized":"2024-02-26T10:41:58.254123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259763+0000","last_change":"2024-02-26T10:41:58.259763+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259763+0000","last_undegraded":"2024-02-26T10:41:58.259763+0000","last_fullsized":"2024-02-26T10:41:58.259763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252106+0000","last_change":"2024-02-26T10:41:58.252106+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252106+0000","last_undegraded":"2024-02-26T10:41:58.252106+0000","last_fullsized":"2024-02-26T10:41:58.252106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252016+0000","last_change":"2024-02-26T10:41:58.252016+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252016+0000","last_undegraded":"2024-02-26T10:41:58.252016+0000","last_fullsized":"2024-02-26T10:41:58.252016+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259892+0000","last_change":"2024-02-26T10:41:58.259892+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259892+0000","last_undegraded":"2024-02-26T10:41:58.259892+0000","last_fullsized":"2024-02-26T10:41:58.259892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.239700+0000","last_change":"2024-02-26T10:41:58.239700+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.239700+0000","last_undegraded":"2024-02-26T10:41:58.239700+0000","last_fullsized":"2024-02-26T10:41:58.239700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.255189+0000","last_change":"2024-02-26T10:41:58.255189+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.255189+0000","last_undegraded":"2024-02-26T10:41:58.255189+0000","last_fullsized":"2024-02-26T10:41:58.255189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252328+0000","last_change":"2024-02-26T10:41:58.252328+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252328+0000","last_undegraded":"2024-02-26T10:41:58.252328+0000","last_fullsized":"2024-02-26T10:41:58.252328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250084+0000","last_change":"2024-02-26T10:41:58.250084+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250084+0000","last_undegraded":"2024-02-26T10:41:58.250084+0000","last_fullsized":"2024-02-26T10:41:58.250084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.257717+0000","last_change":"2024-02-26T10:41:58.257717+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.257717+0000","last_undegraded":"2024-02-26T10:41:58.257717+0000","last_fullsized":"2024-02-26T10:41:58.257717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.263260+0000","last_change":"2024-02-26T10:41:58.263260+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.263260+0000","last_undegraded":"2024-02-26T10:41:58.263260+0000","last_fullsized":"2024-02-26T10:41:58.263260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.251233+0000","last_change":"2024-02-26T10:41:58.251233+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.251233+0000","last_undegraded":"2024-02-26T10:41:58.251233+0000","last_fullsized":"2024-02-26T10:41:58.251233+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248998+0000","last_change":"2024-02-26T10:41:58.248998+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248998+0000","last_undegraded":"2024-02-26T10:41:58.248998+0000","last_fullsized":"2024-02-26T10:41:58.248998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.253518+0000","last_change":"2024-02-26T10:41:58.253518+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.253518+0000","last_undegraded":"2024-02-26T10:41:58.253518+0000","last_fullsized":"2024-02-26T10:41:58.253518+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.262162+0000","last_change":"2024-02-26T10:41:58.262162+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.262162+0000","last_undegraded":"2024-02-26T10:41:58.262162+0000","last_fullsized":"2024-02-26T10:41:58.262162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.253532+0000","last_change":"2024-02-26T10:41:58.253532+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.253532+0000","last_undegraded":"2024-02-26T10:41:58.253532+0000","last_fullsized":"2024-02-26T10:41:58.253532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.265419+0000","last_change":"2024-02-26T10:41:58.265419+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.265419+0000","last_undegraded":"2024-02-26T10:41:58.265419+0000","last_fullsized":"2024-02-26T10:41:58.265419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275701+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:41:58.275701+0000","last_peered":"2024-02-26T10:41:58.275701+0000","last_clean":"2024-02-26T10:41:58.275701+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:41:58.275701+0000","last_undegraded":"2024-02-26T10:41:58.275701+0000","last_fullsized":"2024-02-26T10:41:58.275701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:06.257992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252415+0000","last_change":"2024-02-26T10:41:58.252415+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252415+0000","last_undegraded":"2024-02-26T10:41:58.252415+0000","last_fullsized":"2024-02-26T10:41:58.252415+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.272930+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:41:58.272930+0000","last_peered":"2024-02-26T10:41:58.272930+0000","last_clean":"2024-02-26T10:41:58.272930+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:41:58.272930+0000","last_undegraded":"2024-02-26T10:41:58.272930+0000","last_fullsized":"2024-02-26T10:41:58.272930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:13:21.433177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250084+0000","last_change":"2024-02-26T10:41:58.250084+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250084+0000","last_undegraded":"2024-02-26T10:41:58.250084+0000","last_fullsized":"2024-02-26T10:41:58.250084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.279118+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:41:58.279118+0000","last_peered":"2024-02-26T10:41:58.279118+0000","last_clean":"2024-02-26T10:41:58.279118+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:41:58.279118+0000","last_undegraded":"2024-02-26T10:41:58.279118+0000","last_fullsized":"2024-02-26T10:41:58.279118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:29.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.259368+0000","last_change":"2024-02-26T10:41:58.259368+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.259368+0000","last_undegraded":"2024-02-26T10:41:58.259368+0000","last_fullsized":"2024-02-26T10:41:58.259368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.267920+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:41:58.267920+0000","last_peered":"2024-02-26T10:41:58.267920+0000","last_clean":"2024-02-26T10:41:58.267920+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:41:58.267920+0000","last_undegraded":"2024-02-26T10:41:58.267920+0000","last_fullsized":"2024-02-26T10:41:58.267920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.252720+0000","last_change":"2024-02-26T10:41:58.252720+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.252720+0000","last_undegraded":"2024-02-26T10:41:58.252720+0000","last_fullsized":"2024-02-26T10:41:58.252720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.274664+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:41:58.274664+0000","last_peered":"2024-02-26T10:41:58.274664+0000","last_clean":"2024-02-26T10:41:58.274664+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:41:58.274664+0000","last_undegraded":"2024-02-26T10:41:58.274664+0000","last_fullsized":"2024-02-26T10:41:58.274664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:20:07.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.258754+0000","last_change":"2024-02-26T10:41:58.258754+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.258754+0000","last_undegraded":"2024-02-26T10:41:58.258754+0000","last_fullsized":"2024-02-26T10:41:58.258754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.274360+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:41:58.274360+0000","last_peered":"2024-02-26T10:41:58.274360+0000","last_clean":"2024-02-26T10:41:58.274360+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:41:58.274360+0000","last_undegraded":"2024-02-26T10:41:58.274360+0000","last_fullsized":"2024-02-26T10:41:58.274360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:05.753103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.264915+0000","last_change":"2024-02-26T10:41:58.264915+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.264915+0000","last_undegraded":"2024-02-26T10:41:58.264915+0000","last_fullsized":"2024-02-26T10:41:58.264915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.257318+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:41:58.257318+0000","last_peered":"2024-02-26T10:41:58.257318+0000","last_clean":"2024-02-26T10:41:58.257318+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:41:58.257318+0000","last_undegraded":"2024-02-26T10:41:58.257318+0000","last_fullsized":"2024-02-26T10:41:58.257318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.246031+0000","last_change":"2024-02-26T10:41:58.246031+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.246031+0000","last_undegraded":"2024-02-26T10:41:58.246031+0000","last_fullsized":"2024-02-26T10:41:58.246031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.269260+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:41:58.269260+0000","last_peered":"2024-02-26T10:41:58.269260+0000","last_clean":"2024-02-26T10:41:58.269260+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:41:58.269260+0000","last_undegraded":"2024-02-26T10:41:58.269260+0000","last_fullsized":"2024-02-26T10:41:58.269260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:28:46.080492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248409+0000","last_change":"2024-02-26T10:41:58.248409+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248409+0000","last_undegraded":"2024-02-26T10:41:58.248409+0000","last_fullsized":"2024-02-26T10:41:58.248409+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.274711+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:41:58.274711+0000","last_peered":"2024-02-26T10:41:58.274711+0000","last_clean":"2024-02-26T10:41:58.274711+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:41:58.274711+0000","last_undegraded":"2024-02-26T10:41:58.274711+0000","last_fullsized":"2024-02-26T10:41:58.274711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:20.751081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.241094+0000","last_change":"2024-02-26T10:41:58.241094+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.241094+0000","last_undegraded":"2024-02-26T10:41:58.241094+0000","last_fullsized":"2024-02-26T10:41:58.241094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276127+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:41:58.276127+0000","last_peered":"2024-02-26T10:41:58.276127+0000","last_clean":"2024-02-26T10:41:58.276127+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:41:58.276127+0000","last_undegraded":"2024-02-26T10:41:58.276127+0000","last_fullsized":"2024-02-26T10:41:58.276127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:17.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.241880+0000","last_change":"2024-02-26T10:41:58.241880+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.241880+0000","last_undegraded":"2024-02-26T10:41:58.241880+0000","last_fullsized":"2024-02-26T10:41:58.241880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276143+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:41:58.276143+0000","last_peered":"2024-02-26T10:41:58.276143+0000","last_clean":"2024-02-26T10:41:58.276143+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:41:58.276143+0000","last_undegraded":"2024-02-26T10:41:58.276143+0000","last_fullsized":"2024-02-26T10:41:58.276143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:36.807502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.262725+0000","last_change":"2024-02-26T10:41:58.262725+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.262725+0000","last_undegraded":"2024-02-26T10:41:58.262725+0000","last_fullsized":"2024-02-26T10:41:58.262725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.265808+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:41:58.265808+0000","last_peered":"2024-02-26T10:41:58.265808+0000","last_clean":"2024-02-26T10:41:58.265808+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:41:58.265808+0000","last_undegraded":"2024-02-26T10:41:58.265808+0000","last_fullsized":"2024-02-26T10:41:58.265808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:52.549647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240762+0000","last_change":"2024-02-26T10:41:58.240762+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240762+0000","last_undegraded":"2024-02-26T10:41:58.240762+0000","last_fullsized":"2024-02-26T10:41:58.240762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.262504+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:41:58.262504+0000","last_peered":"2024-02-26T10:41:58.262504+0000","last_clean":"2024-02-26T10:41:58.262504+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:41:58.262504+0000","last_undegraded":"2024-02-26T10:41:58.262504+0000","last_fullsized":"2024-02-26T10:41:58.262504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:28:10.192048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.256927+0000","last_change":"2024-02-26T10:41:58.256927+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.256927+0000","last_undegraded":"2024-02-26T10:41:58.256927+0000","last_fullsized":"2024-02-26T10:41:58.256927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.268048+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:41:58.268048+0000","last_peered":"2024-02-26T10:41:58.268048+0000","last_clean":"2024-02-26T10:41:58.268048+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:41:58.268048+0000","last_undegraded":"2024-02-26T10:41:58.268048+0000","last_fullsized":"2024-02-26T10:41:58.268048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:35.521987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248891+0000","last_change":"2024-02-26T10:41:58.248891+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248891+0000","last_undegraded":"2024-02-26T10:41:58.248891+0000","last_fullsized":"2024-02-26T10:41:58.248891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.271079+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:41:58.271079+0000","last_peered":"2024-02-26T10:41:58.271079+0000","last_clean":"2024-02-26T10:41:58.271079+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:41:58.271079+0000","last_undegraded":"2024-02-26T10:41:58.271079+0000","last_fullsized":"2024-02-26T10:41:58.271079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:29.864788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248770+0000","last_change":"2024-02-26T10:41:58.248770+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248770+0000","last_undegraded":"2024-02-26T10:41:58.248770+0000","last_fullsized":"2024-02-26T10:41:58.248770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.278785+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:41:58.278785+0000","last_peered":"2024-02-26T10:41:58.278785+0000","last_clean":"2024-02-26T10:41:58.278785+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:41:58.278785+0000","last_undegraded":"2024-02-26T10:41:58.278785+0000","last_fullsized":"2024-02-26T10:41:58.278785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:52.549647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.255749+0000","last_change":"2024-02-26T10:41:58.255749+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.255749+0000","last_undegraded":"2024-02-26T10:41:58.255749+0000","last_fullsized":"2024-02-26T10:41:58.255749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.269405+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:41:58.269405+0000","last_peered":"2024-02-26T10:41:58.269405+0000","last_clean":"2024-02-26T10:41:58.269405+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:41:58.269405+0000","last_undegraded":"2024-02-26T10:41:58.269405+0000","last_fullsized":"2024-02-26T10:41:58.269405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:09.674799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.250318+0000","last_change":"2024-02-26T10:41:58.250318+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.250318+0000","last_undegraded":"2024-02-26T10:41:58.250318+0000","last_fullsized":"2024-02-26T10:41:58.250318+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275638+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:41:58.275638+0000","last_peered":"2024-02-26T10:41:58.275638+0000","last_clean":"2024-02-26T10:41:58.275638+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:41:58.275638+0000","last_undegraded":"2024-02-26T10:41:58.275638+0000","last_fullsized":"2024-02-26T10:41:58.275638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.262070+0000","last_change":"2024-02-26T10:41:58.262070+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.262070+0000","last_undegraded":"2024-02-26T10:41:58.262070+0000","last_fullsized":"2024-02-26T10:41:58.262070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.267839+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:41:58.267839+0000","last_peered":"2024-02-26T10:41:58.267839+0000","last_clean":"2024-02-26T10:41:58.267839+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:41:58.267839+0000","last_undegraded":"2024-02-26T10:41:58.267839+0000","last_fullsized":"2024-02-26T10:41:58.267839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.279305+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:41:58.279305+0000","last_peered":"2024-02-26T10:41:58.279305+0000","last_clean":"2024-02-26T10:41:58.279305+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:41:58.279305+0000","last_undegraded":"2024-02-26T10:41:58.279305+0000","last_fullsized":"2024-02-26T10:41:58.279305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:09.156750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240461+0000","last_change":"2024-02-26T10:41:58.240461+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240461+0000","last_undegraded":"2024-02-26T10:41:58.240461+0000","last_fullsized":"2024-02-26T10:41:58.240461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-02-26T10:41:57.235808+0000","last_change":"2024-02-26T10:41:57.235808+0000","last_active":"2024-02-26T10:41:57.210128+0000","last_peered":"2024-02-26T10:41:57.210128+0000","last_clean":"2024-02-26T10:41:57.210128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:57.235808+0000","last_undegraded":"2024-02-26T10:41:57.235808+0000","last_fullsized":"2024-02-26T10:41:57.235808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-26T10:41:58.225047+0000","last_change":"2024-02-26T10:41:58.225047+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.225047+0000","last_undegraded":"2024-02-26T10:41:58.225047+0000","last_fullsized":"2024-02-26T10:41:58.225047+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-02-26T10:41:57.235824+0000","last_change":"2024-02-26T10:41:57.235824+0000","last_active":"2024-02-26T10:41:57.210128+0000","last_peered":"2024-02-26T10:41:57.210128+0000","last_clean":"2024-02-26T10:41:57.210128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:57.235824+0000","last_undegraded":"2024-02-26T10:41:57.235824+0000","last_fullsized":"2024-02-26T10:41:57.235824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.241929+0000","last_change":"2024-02-26T10:41:58.241929+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.241929+0000","last_undegraded":"2024-02-26T10:41:58.241929+0000","last_fullsized":"2024-02-26T10:41:58.241929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275663+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:41:58.275663+0000","last_peered":"2024-02-26T10:41:58.275663+0000","last_clean":"2024-02-26T10:41:58.275663+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:41:58.275663+0000","last_undegraded":"2024-02-26T10:41:58.275663+0000","last_fullsized":"2024-02-26T10:41:58.275663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:26.142556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.257874+0000","last_change":"2024-02-26T10:41:58.257874+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.257874+0000","last_undegraded":"2024-02-26T10:41:58.257874+0000","last_fullsized":"2024-02-26T10:41:58.257874+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276247+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:41:58.276247+0000","last_peered":"2024-02-26T10:41:58.276247+0000","last_clean":"2024-02-26T10:41:58.276247+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:41:58.276247+0000","last_undegraded":"2024-02-26T10:41:58.276247+0000","last_fullsized":"2024-02-26T10:41:58.276247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:11:04.024496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.256592+0000","last_change":"2024-02-26T10:41:58.256592+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.256592+0000","last_undegraded":"2024-02-26T10:41:58.256592+0000","last_fullsized":"2024-02-26T10:41:58.256592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.256856+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:41:58.256856+0000","last_peered":"2024-02-26T10:41:58.256856+0000","last_clean":"2024-02-26T10:41:58.256856+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:41:58.256856+0000","last_undegraded":"2024-02-26T10:41:58.256856+0000","last_fullsized":"2024-02-26T10:41:58.256856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:24:48.243827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248650+0000","last_change":"2024-02-26T10:41:58.248650+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248650+0000","last_undegraded":"2024-02-26T10:41:58.248650+0000","last_fullsized":"2024-02-26T10:41:58.248650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275629+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:41:58.275629+0000","last_peered":"2024-02-26T10:41:58.275629+0000","last_clean":"2024-02-26T10:41:58.275629+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:41:58.275629+0000","last_undegraded":"2024-02-26T10:41:58.275629+0000","last_fullsized":"2024-02-26T10:41:58.275629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:55.395006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.257910+0000","last_change":"2024-02-26T10:41:58.257910+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.257910+0000","last_undegraded":"2024-02-26T10:41:58.257910+0000","last_fullsized":"2024-02-26T10:41:58.257910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.269326+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:41:58.269326+0000","last_peered":"2024-02-26T10:41:58.269326+0000","last_clean":"2024-02-26T10:41:58.269326+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:41:58.269326+0000","last_undegraded":"2024-02-26T10:41:58.269326+0000","last_fullsized":"2024-02-26T10:41:58.269326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:37.043645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.240528+0000","last_change":"2024-02-26T10:41:58.240528+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.240528+0000","last_undegraded":"2024-02-26T10:41:58.240528+0000","last_fullsized":"2024-02-26T10:41:58.240528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.278743+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:41:58.278743+0000","last_peered":"2024-02-26T10:41:58.278743+0000","last_clean":"2024-02-26T10:41:58.278743+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:41:58.278743+0000","last_undegraded":"2024-02-26T10:41:58.278743+0000","last_fullsized":"2024-02-26T10:41:58.278743+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.239566+0000","last_change":"2024-02-26T10:41:58.239566+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.239566+0000","last_undegraded":"2024-02-26T10:41:58.239566+0000","last_fullsized":"2024-02-26T10:41:58.239566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.271122+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:41:58.271122+0000","last_peered":"2024-02-26T10:41:58.271122+0000","last_clean":"2024-02-26T10:41:58.271122+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:41:58.271122+0000","last_undegraded":"2024-02-26T10:41:58.271122+0000","last_fullsized":"2024-02-26T10:41:58.271122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.263699+0000","last_change":"2024-02-26T10:41:58.263699+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.263699+0000","last_undegraded":"2024-02-26T10:41:58.263699+0000","last_fullsized":"2024-02-26T10:41:58.263699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.267511+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:41:58.267511+0000","last_peered":"2024-02-26T10:41:58.267511+0000","last_clean":"2024-02-26T10:41:58.267511+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:41:58.267511+0000","last_undegraded":"2024-02-26T10:41:58.267511+0000","last_fullsized":"2024-02-26T10:41:58.267511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:12:16.519035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.251246+0000","last_change":"2024-02-26T10:41:58.251246+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.251246+0000","last_undegraded":"2024-02-26T10:41:58.251246+0000","last_fullsized":"2024-02-26T10:41:58.251246+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.276875+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:41:58.276875+0000","last_peered":"2024-02-26T10:41:58.276875+0000","last_clean":"2024-02-26T10:41:58.276875+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:41:58.276875+0000","last_undegraded":"2024-02-26T10:41:58.276875+0000","last_fullsized":"2024-02-26T10:41:58.276875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:12:54.837264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-02-26T10:41:58.248807+0000","last_change":"2024-02-26T10:41:58.248807+0000","last_active":"2024-02-26T10:41:58.225047+0000","last_peered":"2024-02-26T10:41:58.225047+0000","last_clean":"2024-02-26T10:41:58.225047+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-26T10:41:58.248807+0000","last_undegraded":"2024-02-26T10:41:58.248807+0000","last_fullsized":"2024-02-26T10:41:58.248807+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.275381+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:41:58.275381+0000","last_peered":"2024-02-26T10:41:58.275381+0000","last_clean":"2024-02-26T10:41:58.275381+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:41:58.275381+0000","last_undegraded":"2024-02-26T10:41:58.275381+0000","last_fullsized":"2024-02-26T10:41:58.275381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":270,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-26T10:41:58.266978+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:41:58.266978+0000","last_peered":"2024-02-26T10:41:58.266978+0000","last_clean":"2024-02-26T10:41:58.266978+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:41:58.266978+0000","last_undegraded":"2024-02-26T10:41:58.266978+0000","last_fullsized":"2024-02-26T10:41:58.266978+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":985011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":905216,"data_stored":400918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":388826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":394643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1486848,"data_stored":985011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:04.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:04.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.542+0000 7ff7a9c00640 1 Processor -- start 2024-02-26T10:42:04.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.543+0000 7ff7a9c00640 1 -- start start 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.544+0000 7ff7a9c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff7a40c8510 0x7ff7a40c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.544+0000 7ff7a9c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a4152760 con 0x7ff7a40c7020 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.544+0000 7ff7a9c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a406d9a0 con 0x7ff7a4152380 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.544+0000 7ff7a9c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a40c7550 con 0x7ff7a40c8510 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.544+0000 7ff79be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff7a40c7020 0x7ff7a406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56122/0 (socket says 172.21.15.67:56122) 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.545+0000 7ff79be00640 1 -- 172.21.15.67:0/196203981 learned_addr learned my addr 172.21.15.67:0/196203981 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.545+0000 7ff7a9200640 1 --2- 172.21.15.67:0/196203981 >> v2:172.21.15.82:3300/0 conn(0x7ff7a40c8510 0x7ff7a40c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.545+0000 7ff7a3400640 1 -- 172.21.15.67:0/196203981 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809033923 0 0) 0x7ff7a4152760 con 0x7ff7a40c7020 2024-02-26T10:42:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.545+0000 7ff7a3400640 1 -- 172.21.15.67:0/196203981 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff788003520 con 0x7ff7a40c7020 2024-02-26T10:42:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.546+0000 7ff7a3400640 1 -- 172.21.15.67:0/196203981 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600179491 0 0) 0x7ff7a406d9a0 con 0x7ff7a4152380 2024-02-26T10:42:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.546+0000 7ff7a9200640 1 -- 172.21.15.67:0/196203981 >> v1:172.21.15.82:6789/0 conn(0x7ff7a4152380 legacy=0x7ff7a4155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.546+0000 7ff7a9200640 1 -- 172.21.15.67:0/196203981 >> v1:172.21.15.67:6789/0 conn(0x7ff7a40c7020 legacy=0x7ff7a406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.546+0000 7ff7a9200640 1 -- 172.21.15.67:0/196203981 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40c7830 con 0x7ff7a40c8510 2024-02-26T10:42:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.547+0000 7ff7a9200640 1 --2- 172.21.15.67:0/196203981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c8510 0x7ff7a40c4f90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009d90 tx=0x7ff79c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0c307b02564a2f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.547+0000 7ff7a3400640 1 -- 172.21.15.67:0/196203981 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c039070 con 0x7ff7a40c8510 2024-02-26T10:42:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a3400640 1 -- 172.21.15.67:0/196203981 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c02cc60 con 0x7ff7a40c8510 2024-02-26T10:42:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a3400640 1 -- 172.21.15.67:0/196203981 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c034640 con 0x7ff7a40c8510 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 -- 172.21.15.67:0/196203981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c8510 msgr2=0x7ff7a40c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/196203981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c8510 0x7ff7a40c4f90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009d90 tx=0x7ff79c02beb0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 -- 172.21.15.67:0/196203981 shutdown_connections 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/196203981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c8510 0x7ff7a40c4f90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 -- 172.21.15.67:0/196203981 >> 172.21.15.67:0/196203981 conn(0x7ff7a40ac1a0 msgr2=0x7ff7a40ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 -- 172.21.15.67:0/196203981 shutdown_connections 2024-02-26T10:42:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.548+0000 7ff7a9c00640 1 -- 172.21.15.67:0/196203981 wait complete. 2024-02-26T10:42:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.550+0000 7ff7a9c00640 1 Processor -- start 2024-02-26T10:42:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.550+0000 7ff7a9c00640 1 -- start start 2024-02-26T10:42:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff7a9c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c7020 0x7ff7a4157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff7a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 0x7ff7a40ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff7a9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7a4152380 0x7ff7a40adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff7a9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40c54d0 con 0x7ff7a40c8510 2024-02-26T10:42:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff7a9c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a406cd70 con 0x7ff7a40c7020 2024-02-26T10:42:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff7a9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a40c5650 con 0x7ff7a4152380 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.551+0000 7ff79be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 0x7ff7a40ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 0x7ff7a40ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43472/0 (socket says 172.21.15.67:43472) 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 -- 172.21.15.67:0/1124954713 learned_addr learned my addr 172.21.15.67:0/1124954713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff7a3e00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c7020 0x7ff7a4157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff7a9200640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7a4152380 0x7ff7a40adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 -- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7a4152380 msgr2=0x7ff7a40adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7a4152380 0x7ff7a40adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 -- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c7020 msgr2=0x7ff7a4157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c7020 0x7ff7a4157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 -- 172.21.15.67:0/1124954713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4150df0 con 0x7ff7a40c8510 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff7a9200640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7a4152380 0x7ff7a40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff7a3e00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c7020 0x7ff7a4157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.552+0000 7ff79be00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 0x7ff7a40ad6e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00c430 tx=0x7ff78c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.553+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c017070 con 0x7ff7a40c8510 2024-02-26T10:42:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.553+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c005ce0 con 0x7ff7a40c8510 2024-02-26T10:42:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.553+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0072f0 con 0x7ff7a40c8510 2024-02-26T10:42:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.553+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a41587e0 con 0x7ff7a40c8510 2024-02-26T10:42:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.553+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a41564b0 con 0x7ff7a40c8510 2024-02-26T10:42:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.554+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a406bf80 con 0x7ff7a40c8510 2024-02-26T10:42:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.555+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff78c01a070 con 0x7ff7a40c8510 2024-02-26T10:42:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.556+0000 7ff7a0c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78006b4d0 0x7ff78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.556+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff78c0896a0 con 0x7ff7a40c8510 2024-02-26T10:42:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.557+0000 7ff7a3e00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78006b4d0 0x7ff78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:04.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.558+0000 7ff7a3e00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78006b4d0 0x7ff78006d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff794000f60 tx=0x7ff794009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:04.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.560+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7a406bf80 con 0x7ff7a40c8510 2024-02-26T10:42:04.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.825+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7a4074a30 con 0x7ff78006b4d0 2024-02-26T10:42:04.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.844+0000 7ff7a0c00640 1 -- 172.21.15.67:0/1124954713 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252078 (secure 0 0 0) 0x7ff7a4074a30 con 0x7ff78006b4d0 2024-02-26T10:42:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:04.847 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78006b4d0 msgr2=0x7ff78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78006b4d0 0x7ff78006d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff794000f60 tx=0x7ff794009040 comp rx=0 tx=0).stop 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 msgr2=0x7ff7a40ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 0x7ff7a40ad6e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00c430 tx=0x7ff78c00c8f0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 shutdown_connections 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff7a4152380 0x7ff7a40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a40c8510 0x7ff7a40ad6e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff78006b4d0 0x7ff78006d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 --2- 172.21.15.67:0/1124954713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a40c7020 0x7ff7a4157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:04.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 >> 172.21.15.67:0/1124954713 conn(0x7ff7a40ac1a0 msgr2=0x7ff7a40c88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:04.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 shutdown_connections 2024-02-26T10:42:04.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:04.850+0000 7ff7a9c00640 1 -- 172.21.15.67:0/1124954713 wait complete. 2024-02-26T10:42:04.867 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-02-26T10:42:04.403661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249636,"kb_used_data":8488,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725084,"statfs":{"total":773094113280,"available":772838486016,"internally_reserved":0,"allocated":8691712,"data_stored":4497500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.169336"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146868+0000","last_change":"2024-02-26T10:41:59.308476+0000","last_active":"2024-02-26T10:42:01.146868+0000","last_peered":"2024-02-26T10:42:01.146868+0000","last_clean":"2024-02-26T10:42:01.146868+0000","last_became_active":"2024-02-26T10:41:59.307995+0000","last_became_peered":"2024-02-26T10:41:59.307995+0000","last_unstale":"2024-02-26T10:42:01.146868+0000","last_undegraded":"2024-02-26T10:42:01.146868+0000","last_fullsized":"2024-02-26T10:42:01.146868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290470+0000","last_change":"2024-02-26T10:41:59.300851+0000","last_active":"2024-02-26T10:42:00.290470+0000","last_peered":"2024-02-26T10:42:00.290470+0000","last_clean":"2024-02-26T10:42:00.290470+0000","last_became_active":"2024-02-26T10:41:59.300574+0000","last_became_peered":"2024-02-26T10:41:59.300574+0000","last_unstale":"2024-02-26T10:42:00.290470+0000","last_undegraded":"2024-02-26T10:42:00.290470+0000","last_fullsized":"2024-02-26T10:42:00.290470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:04.528547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284215+0000","last_change":"2024-02-26T10:41:59.293058+0000","last_active":"2024-02-26T10:42:00.284215+0000","last_peered":"2024-02-26T10:42:00.284215+0000","last_clean":"2024-02-26T10:42:00.284215+0000","last_became_active":"2024-02-26T10:41:59.292789+0000","last_became_peered":"2024-02-26T10:41:59.292789+0000","last_unstale":"2024-02-26T10:42:00.284215+0000","last_undegraded":"2024-02-26T10:42:00.284215+0000","last_fullsized":"2024-02-26T10:42:00.284215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114472+0000","last_change":"2024-02-26T10:41:59.296577+0000","last_active":"2024-02-26T10:42:01.114472+0000","last_peered":"2024-02-26T10:42:01.114472+0000","last_clean":"2024-02-26T10:42:01.114472+0000","last_became_active":"2024-02-26T10:41:59.296264+0000","last_became_peered":"2024-02-26T10:41:59.296264+0000","last_unstale":"2024-02-26T10:42:01.114472+0000","last_undegraded":"2024-02-26T10:42:01.114472+0000","last_fullsized":"2024-02-26T10:42:01.114472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:13.257965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079286+0000","last_change":"2024-02-26T10:41:59.298038+0000","last_active":"2024-02-26T10:42:01.079286+0000","last_peered":"2024-02-26T10:42:01.079286+0000","last_clean":"2024-02-26T10:42:01.079286+0000","last_became_active":"2024-02-26T10:41:59.297699+0000","last_became_peered":"2024-02-26T10:41:59.297699+0000","last_unstale":"2024-02-26T10:42:01.079286+0000","last_undegraded":"2024-02-26T10:42:01.079286+0000","last_fullsized":"2024-02-26T10:42:01.079286+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:30.879707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145586+0000","last_change":"2024-02-26T10:41:59.295626+0000","last_active":"2024-02-26T10:42:01.145586+0000","last_peered":"2024-02-26T10:42:01.145586+0000","last_clean":"2024-02-26T10:42:01.145586+0000","last_became_active":"2024-02-26T10:41:59.295266+0000","last_became_peered":"2024-02-26T10:41:59.295266+0000","last_unstale":"2024-02-26T10:42:01.145586+0000","last_undegraded":"2024-02-26T10:42:01.145586+0000","last_fullsized":"2024-02-26T10:42:01.145586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113506+0000","last_change":"2024-02-26T10:41:59.287398+0000","last_active":"2024-02-26T10:42:01.113506+0000","last_peered":"2024-02-26T10:42:01.113506+0000","last_clean":"2024-02-26T10:42:01.113506+0000","last_became_active":"2024-02-26T10:41:59.286305+0000","last_became_peered":"2024-02-26T10:41:59.286305+0000","last_unstale":"2024-02-26T10:42:01.113506+0000","last_undegraded":"2024-02-26T10:42:01.113506+0000","last_fullsized":"2024-02-26T10:42:01.113506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:54.865254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144514+0000","last_change":"2024-02-26T10:41:59.307667+0000","last_active":"2024-02-26T10:42:01.144514+0000","last_peered":"2024-02-26T10:42:01.144514+0000","last_clean":"2024-02-26T10:42:01.144514+0000","last_became_active":"2024-02-26T10:41:59.307319+0000","last_became_peered":"2024-02-26T10:41:59.307319+0000","last_unstale":"2024-02-26T10:42:01.144514+0000","last_undegraded":"2024-02-26T10:42:01.144514+0000","last_fullsized":"2024-02-26T10:42:01.144514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144909+0000","last_change":"2024-02-26T10:41:59.304927+0000","last_active":"2024-02-26T10:42:01.144909+0000","last_peered":"2024-02-26T10:42:01.144909+0000","last_clean":"2024-02-26T10:42:01.144909+0000","last_became_active":"2024-02-26T10:41:59.304641+0000","last_became_peered":"2024-02-26T10:41:59.304641+0000","last_unstale":"2024-02-26T10:42:01.144909+0000","last_undegraded":"2024-02-26T10:42:01.144909+0000","last_fullsized":"2024-02-26T10:42:01.144909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146925+0000","last_change":"2024-02-26T10:41:59.293335+0000","last_active":"2024-02-26T10:42:01.146925+0000","last_peered":"2024-02-26T10:42:01.146925+0000","last_clean":"2024-02-26T10:42:01.146925+0000","last_became_active":"2024-02-26T10:41:59.292937+0000","last_became_peered":"2024-02-26T10:41:59.292937+0000","last_unstale":"2024-02-26T10:42:01.146925+0000","last_undegraded":"2024-02-26T10:42:01.146925+0000","last_fullsized":"2024-02-26T10:42:01.146925+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289248+0000","last_change":"2024-02-26T10:41:59.282216+0000","last_active":"2024-02-26T10:42:00.289248+0000","last_peered":"2024-02-26T10:42:00.289248+0000","last_clean":"2024-02-26T10:42:00.289248+0000","last_became_active":"2024-02-26T10:41:59.281891+0000","last_became_peered":"2024-02-26T10:41:59.281891+0000","last_unstale":"2024-02-26T10:42:00.289248+0000","last_undegraded":"2024-02-26T10:42:00.289248+0000","last_fullsized":"2024-02-26T10:42:00.289248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:07.272911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146079+0000","last_change":"2024-02-26T10:41:59.290942+0000","last_active":"2024-02-26T10:42:01.146079+0000","last_peered":"2024-02-26T10:42:01.146079+0000","last_clean":"2024-02-26T10:42:01.146079+0000","last_became_active":"2024-02-26T10:41:59.288616+0000","last_became_peered":"2024-02-26T10:41:59.288616+0000","last_unstale":"2024-02-26T10:42:01.146079+0000","last_undegraded":"2024-02-26T10:42:01.146079+0000","last_fullsized":"2024-02-26T10:42:01.146079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284058+0000","last_change":"2024-02-26T10:41:59.281911+0000","last_active":"2024-02-26T10:42:00.284058+0000","last_peered":"2024-02-26T10:42:00.284058+0000","last_clean":"2024-02-26T10:42:00.284058+0000","last_became_active":"2024-02-26T10:41:59.281270+0000","last_became_peered":"2024-02-26T10:41:59.281270+0000","last_unstale":"2024-02-26T10:42:00.284058+0000","last_undegraded":"2024-02-26T10:42:00.284058+0000","last_fullsized":"2024-02-26T10:42:00.284058+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113712+0000","last_change":"2024-02-26T10:41:59.310387+0000","last_active":"2024-02-26T10:42:01.113712+0000","last_peered":"2024-02-26T10:42:01.113712+0000","last_clean":"2024-02-26T10:42:01.113712+0000","last_became_active":"2024-02-26T10:41:59.310111+0000","last_became_peered":"2024-02-26T10:41:59.310111+0000","last_unstale":"2024-02-26T10:42:01.113712+0000","last_undegraded":"2024-02-26T10:42:01.113712+0000","last_fullsized":"2024-02-26T10:42:01.113712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:53:52.167978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281188+0000","last_change":"2024-02-26T10:41:59.289328+0000","last_active":"2024-02-26T10:42:00.281188+0000","last_peered":"2024-02-26T10:42:00.281188+0000","last_clean":"2024-02-26T10:42:00.281188+0000","last_became_active":"2024-02-26T10:41:59.288951+0000","last_became_peered":"2024-02-26T10:41:59.288951+0000","last_unstale":"2024-02-26T10:42:00.281188+0000","last_undegraded":"2024-02-26T10:42:00.281188+0000","last_fullsized":"2024-02-26T10:42:00.281188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:00.613487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078476+0000","last_change":"2024-02-26T10:41:59.290536+0000","last_active":"2024-02-26T10:42:01.078476+0000","last_peered":"2024-02-26T10:42:01.078476+0000","last_clean":"2024-02-26T10:42:01.078476+0000","last_became_active":"2024-02-26T10:41:59.289661+0000","last_became_peered":"2024-02-26T10:41:59.289661+0000","last_unstale":"2024-02-26T10:42:01.078476+0000","last_undegraded":"2024-02-26T10:42:01.078476+0000","last_fullsized":"2024-02-26T10:42:01.078476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:56.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079115+0000","last_change":"2024-02-26T10:41:59.291203+0000","last_active":"2024-02-26T10:42:01.079115+0000","last_peered":"2024-02-26T10:42:01.079115+0000","last_clean":"2024-02-26T10:42:01.079115+0000","last_became_active":"2024-02-26T10:41:59.290916+0000","last_became_peered":"2024-02-26T10:41:59.290916+0000","last_unstale":"2024-02-26T10:42:01.079115+0000","last_undegraded":"2024-02-26T10:42:01.079115+0000","last_fullsized":"2024-02-26T10:42:01.079115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:09.397334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114840+0000","last_change":"2024-02-26T10:41:59.291282+0000","last_active":"2024-02-26T10:42:01.114840+0000","last_peered":"2024-02-26T10:42:01.114840+0000","last_clean":"2024-02-26T10:42:01.114840+0000","last_became_active":"2024-02-26T10:41:59.291008+0000","last_became_peered":"2024-02-26T10:41:59.291008+0000","last_unstale":"2024-02-26T10:42:01.114840+0000","last_undegraded":"2024-02-26T10:42:01.114840+0000","last_fullsized":"2024-02-26T10:42:01.114840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:55:38.972893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283816+0000","last_change":"2024-02-26T10:41:59.289830+0000","last_active":"2024-02-26T10:42:00.283816+0000","last_peered":"2024-02-26T10:42:00.283816+0000","last_clean":"2024-02-26T10:42:00.283816+0000","last_became_active":"2024-02-26T10:41:59.289466+0000","last_became_peered":"2024-02-26T10:41:59.289466+0000","last_unstale":"2024-02-26T10:42:00.283816+0000","last_undegraded":"2024-02-26T10:42:00.283816+0000","last_fullsized":"2024-02-26T10:42:00.283816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114572+0000","last_change":"2024-02-26T10:41:59.294466+0000","last_active":"2024-02-26T10:42:01.114572+0000","last_peered":"2024-02-26T10:42:01.114572+0000","last_clean":"2024-02-26T10:42:01.114572+0000","last_became_active":"2024-02-26T10:41:59.294199+0000","last_became_peered":"2024-02-26T10:41:59.294199+0000","last_unstale":"2024-02-26T10:42:01.114572+0000","last_undegraded":"2024-02-26T10:42:01.114572+0000","last_fullsized":"2024-02-26T10:42:01.114572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:06.679617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114210+0000","last_change":"2024-02-26T10:41:59.302058+0000","last_active":"2024-02-26T10:42:01.114210+0000","last_peered":"2024-02-26T10:42:01.114210+0000","last_clean":"2024-02-26T10:42:01.114210+0000","last_became_active":"2024-02-26T10:41:59.301773+0000","last_became_peered":"2024-02-26T10:41:59.301773+0000","last_unstale":"2024-02-26T10:42:01.114210+0000","last_undegraded":"2024-02-26T10:42:01.114210+0000","last_fullsized":"2024-02-26T10:42:01.114210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:58.489330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289203+0000","last_change":"2024-02-26T10:41:59.306729+0000","last_active":"2024-02-26T10:42:00.289203+0000","last_peered":"2024-02-26T10:42:00.289203+0000","last_clean":"2024-02-26T10:42:00.289203+0000","last_became_active":"2024-02-26T10:41:59.306464+0000","last_became_peered":"2024-02-26T10:41:59.306464+0000","last_unstale":"2024-02-26T10:42:00.289203+0000","last_undegraded":"2024-02-26T10:42:00.289203+0000","last_fullsized":"2024-02-26T10:42:00.289203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:56.409926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145960+0000","last_change":"2024-02-26T10:41:59.304328+0000","last_active":"2024-02-26T10:42:01.145960+0000","last_peered":"2024-02-26T10:42:01.145960+0000","last_clean":"2024-02-26T10:42:01.145960+0000","last_became_active":"2024-02-26T10:41:59.304025+0000","last_became_peered":"2024-02-26T10:41:59.304025+0000","last_unstale":"2024-02-26T10:42:01.145960+0000","last_undegraded":"2024-02-26T10:42:01.145960+0000","last_fullsized":"2024-02-26T10:42:01.145960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113484+0000","last_change":"2024-02-26T10:41:59.303897+0000","last_active":"2024-02-26T10:42:01.113484+0000","last_peered":"2024-02-26T10:42:01.113484+0000","last_clean":"2024-02-26T10:42:01.113484+0000","last_became_active":"2024-02-26T10:41:59.301812+0000","last_became_peered":"2024-02-26T10:41:59.301812+0000","last_unstale":"2024-02-26T10:42:01.113484+0000","last_undegraded":"2024-02-26T10:42:01.113484+0000","last_fullsized":"2024-02-26T10:42:01.113484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:03:57.549048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290341+0000","last_change":"2024-02-26T10:41:59.297092+0000","last_active":"2024-02-26T10:42:00.290341+0000","last_peered":"2024-02-26T10:42:00.290341+0000","last_clean":"2024-02-26T10:42:00.290341+0000","last_became_active":"2024-02-26T10:41:59.295816+0000","last_became_peered":"2024-02-26T10:41:59.295816+0000","last_unstale":"2024-02-26T10:42:00.290341+0000","last_undegraded":"2024-02-26T10:42:00.290341+0000","last_fullsized":"2024-02-26T10:42:00.290341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:05.502778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284127+0000","last_change":"2024-02-26T10:41:59.307675+0000","last_active":"2024-02-26T10:42:00.284127+0000","last_peered":"2024-02-26T10:42:00.284127+0000","last_clean":"2024-02-26T10:42:00.284127+0000","last_became_active":"2024-02-26T10:41:59.307360+0000","last_became_peered":"2024-02-26T10:41:59.307360+0000","last_unstale":"2024-02-26T10:42:00.284127+0000","last_undegraded":"2024-02-26T10:42:00.284127+0000","last_fullsized":"2024-02-26T10:42:00.284127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079335+0000","last_change":"2024-02-26T10:41:59.287990+0000","last_active":"2024-02-26T10:42:01.079335+0000","last_peered":"2024-02-26T10:42:01.079335+0000","last_clean":"2024-02-26T10:42:01.079335+0000","last_became_active":"2024-02-26T10:41:59.286925+0000","last_became_peered":"2024-02-26T10:41:59.286925+0000","last_unstale":"2024-02-26T10:42:01.079335+0000","last_undegraded":"2024-02-26T10:42:01.079335+0000","last_fullsized":"2024-02-26T10:42:01.079335+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:08.646623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146292+0000","last_change":"2024-02-26T10:41:59.297459+0000","last_active":"2024-02-26T10:42:01.146292+0000","last_peered":"2024-02-26T10:42:01.146292+0000","last_clean":"2024-02-26T10:42:01.146292+0000","last_became_active":"2024-02-26T10:41:59.296494+0000","last_became_peered":"2024-02-26T10:41:59.296494+0000","last_unstale":"2024-02-26T10:42:01.146292+0000","last_undegraded":"2024-02-26T10:42:01.146292+0000","last_fullsized":"2024-02-26T10:42:01.146292+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289123+0000","last_change":"2024-02-26T10:41:59.288740+0000","last_active":"2024-02-26T10:42:00.289123+0000","last_peered":"2024-02-26T10:42:00.289123+0000","last_clean":"2024-02-26T10:42:00.289123+0000","last_became_active":"2024-02-26T10:41:59.288433+0000","last_became_peered":"2024-02-26T10:41:59.288433+0000","last_unstale":"2024-02-26T10:42:00.289123+0000","last_undegraded":"2024-02-26T10:42:00.289123+0000","last_fullsized":"2024-02-26T10:42:00.289123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:21.766000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289299+0000","last_change":"2024-02-26T10:41:59.293103+0000","last_active":"2024-02-26T10:42:00.289299+0000","last_peered":"2024-02-26T10:42:00.289299+0000","last_clean":"2024-02-26T10:42:00.289299+0000","last_became_active":"2024-02-26T10:41:59.292473+0000","last_became_peered":"2024-02-26T10:41:59.292473+0000","last_unstale":"2024-02-26T10:42:00.289299+0000","last_undegraded":"2024-02-26T10:42:00.289299+0000","last_fullsized":"2024-02-26T10:42:00.289299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:37.822421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078673+0000","last_change":"2024-02-26T10:41:59.294484+0000","last_active":"2024-02-26T10:42:01.078673+0000","last_peered":"2024-02-26T10:42:01.078673+0000","last_clean":"2024-02-26T10:42:01.078673+0000","last_became_active":"2024-02-26T10:41:59.293628+0000","last_became_peered":"2024-02-26T10:41:59.293628+0000","last_unstale":"2024-02-26T10:42:01.078673+0000","last_undegraded":"2024-02-26T10:42:01.078673+0000","last_fullsized":"2024-02-26T10:42:01.078673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:04.694202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145442+0000","last_change":"2024-02-26T10:41:59.289588+0000","last_active":"2024-02-26T10:42:01.145442+0000","last_peered":"2024-02-26T10:42:01.145442+0000","last_clean":"2024-02-26T10:42:01.145442+0000","last_became_active":"2024-02-26T10:41:59.289223+0000","last_became_peered":"2024-02-26T10:41:59.289223+0000","last_unstale":"2024-02-26T10:42:01.145442+0000","last_undegraded":"2024-02-26T10:42:01.145442+0000","last_fullsized":"2024-02-26T10:42:01.145442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079143+0000","last_change":"2024-02-26T10:41:59.296979+0000","last_active":"2024-02-26T10:42:01.079143+0000","last_peered":"2024-02-26T10:42:01.079143+0000","last_clean":"2024-02-26T10:42:01.079143+0000","last_became_active":"2024-02-26T10:41:59.295612+0000","last_became_peered":"2024-02-26T10:41:59.295612+0000","last_unstale":"2024-02-26T10:42:01.079143+0000","last_undegraded":"2024-02-26T10:42:01.079143+0000","last_fullsized":"2024-02-26T10:42:01.079143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:53.564566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147187+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:42:01.147187+0000","last_peered":"2024-02-26T10:42:01.147187+0000","last_clean":"2024-02-26T10:42:01.147187+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:42:01.147187+0000","last_undegraded":"2024-02-26T10:42:01.147187+0000","last_fullsized":"2024-02-26T10:42:01.147187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:03.513627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146982+0000","last_change":"2024-02-26T10:41:59.304661+0000","last_active":"2024-02-26T10:42:01.146982+0000","last_peered":"2024-02-26T10:42:01.146982+0000","last_clean":"2024-02-26T10:42:01.146982+0000","last_became_active":"2024-02-26T10:41:59.304358+0000","last_became_peered":"2024-02-26T10:41:59.304358+0000","last_unstale":"2024-02-26T10:42:01.146982+0000","last_undegraded":"2024-02-26T10:42:01.146982+0000","last_fullsized":"2024-02-26T10:42:01.146982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079625+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:42:01.079625+0000","last_peered":"2024-02-26T10:42:01.079625+0000","last_clean":"2024-02-26T10:42:01.079625+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:42:01.079625+0000","last_undegraded":"2024-02-26T10:42:01.079625+0000","last_fullsized":"2024-02-26T10:42:01.079625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:07.218989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147061+0000","last_change":"2024-02-26T10:41:59.310348+0000","last_active":"2024-02-26T10:42:01.147061+0000","last_peered":"2024-02-26T10:42:01.147061+0000","last_clean":"2024-02-26T10:42:01.147061+0000","last_became_active":"2024-02-26T10:41:59.309976+0000","last_became_peered":"2024-02-26T10:41:59.309976+0000","last_unstale":"2024-02-26T10:42:01.147061+0000","last_undegraded":"2024-02-26T10:42:01.147061+0000","last_fullsized":"2024-02-26T10:42:01.147061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283912+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:42:00.283912+0000","last_peered":"2024-02-26T10:42:00.283912+0000","last_clean":"2024-02-26T10:42:00.283912+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:42:00.283912+0000","last_undegraded":"2024-02-26T10:42:00.283912+0000","last_fullsized":"2024-02-26T10:42:00.283912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:15.932239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284613+0000","last_change":"2024-02-26T10:41:59.287952+0000","last_active":"2024-02-26T10:42:00.284613+0000","last_peered":"2024-02-26T10:42:00.284613+0000","last_clean":"2024-02-26T10:42:00.284613+0000","last_became_active":"2024-02-26T10:41:59.287395+0000","last_became_peered":"2024-02-26T10:41:59.287395+0000","last_unstale":"2024-02-26T10:42:00.284613+0000","last_undegraded":"2024-02-26T10:42:00.284613+0000","last_fullsized":"2024-02-26T10:42:00.284613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:54.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283312+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:42:00.283312+0000","last_peered":"2024-02-26T10:42:00.283312+0000","last_clean":"2024-02-26T10:42:00.283312+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:42:00.283312+0000","last_undegraded":"2024-02-26T10:42:00.283312+0000","last_fullsized":"2024-02-26T10:42:00.283312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289627+0000","last_change":"2024-02-26T10:41:59.292771+0000","last_active":"2024-02-26T10:42:00.289627+0000","last_peered":"2024-02-26T10:42:00.289627+0000","last_clean":"2024-02-26T10:42:00.289627+0000","last_became_active":"2024-02-26T10:41:59.288778+0000","last_became_peered":"2024-02-26T10:41:59.288778+0000","last_unstale":"2024-02-26T10:42:00.289627+0000","last_undegraded":"2024-02-26T10:42:00.289627+0000","last_fullsized":"2024-02-26T10:42:00.289627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:18.908425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079500+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:42:01.079500+0000","last_peered":"2024-02-26T10:42:01.079500+0000","last_clean":"2024-02-26T10:42:01.079500+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:42:01.079500+0000","last_undegraded":"2024-02-26T10:42:01.079500+0000","last_fullsized":"2024-02-26T10:42:01.079500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:20.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078896+0000","last_change":"2024-02-26T10:41:59.287430+0000","last_active":"2024-02-26T10:42:01.078896+0000","last_peered":"2024-02-26T10:42:01.078896+0000","last_clean":"2024-02-26T10:42:01.078896+0000","last_became_active":"2024-02-26T10:41:59.287010+0000","last_became_peered":"2024-02-26T10:41:59.287010+0000","last_unstale":"2024-02-26T10:42:01.078896+0000","last_undegraded":"2024-02-26T10:42:01.078896+0000","last_fullsized":"2024-02-26T10:42:01.078896+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:24.065359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078521+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:42:01.078521+0000","last_peered":"2024-02-26T10:42:01.078521+0000","last_clean":"2024-02-26T10:42:01.078521+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:42:01.078521+0000","last_undegraded":"2024-02-26T10:42:01.078521+0000","last_fullsized":"2024-02-26T10:42:01.078521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:19.393230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283719+0000","last_change":"2024-02-26T10:41:59.301966+0000","last_active":"2024-02-26T10:42:00.283719+0000","last_peered":"2024-02-26T10:42:00.283719+0000","last_clean":"2024-02-26T10:42:00.283719+0000","last_became_active":"2024-02-26T10:41:59.301282+0000","last_became_peered":"2024-02-26T10:41:59.301282+0000","last_unstale":"2024-02-26T10:42:00.283719+0000","last_undegraded":"2024-02-26T10:42:00.283719+0000","last_fullsized":"2024-02-26T10:42:00.283719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282755+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:42:00.282755+0000","last_peered":"2024-02-26T10:42:00.282755+0000","last_clean":"2024-02-26T10:42:00.282755+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:42:00.282755+0000","last_undegraded":"2024-02-26T10:42:00.282755+0000","last_fullsized":"2024-02-26T10:42:00.282755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:07.631703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281456+0000","last_change":"2024-02-26T10:41:59.289078+0000","last_active":"2024-02-26T10:42:00.281456+0000","last_peered":"2024-02-26T10:42:00.281456+0000","last_clean":"2024-02-26T10:42:00.281456+0000","last_became_active":"2024-02-26T10:41:59.288529+0000","last_became_peered":"2024-02-26T10:41:59.288529+0000","last_unstale":"2024-02-26T10:42:00.281456+0000","last_undegraded":"2024-02-26T10:42:00.281456+0000","last_fullsized":"2024-02-26T10:42:00.281456+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:28.358063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288136+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:42:00.288136+0000","last_peered":"2024-02-26T10:42:00.288136+0000","last_clean":"2024-02-26T10:42:00.288136+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:42:00.288136+0000","last_undegraded":"2024-02-26T10:42:00.288136+0000","last_fullsized":"2024-02-26T10:42:00.288136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:56.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282489+0000","last_change":"2024-02-26T10:41:59.294420+0000","last_active":"2024-02-26T10:42:00.282489+0000","last_peered":"2024-02-26T10:42:00.282489+0000","last_clean":"2024-02-26T10:42:00.282489+0000","last_became_active":"2024-02-26T10:41:59.294147+0000","last_became_peered":"2024-02-26T10:41:59.294147+0000","last_unstale":"2024-02-26T10:42:00.282489+0000","last_undegraded":"2024-02-26T10:42:00.282489+0000","last_fullsized":"2024-02-26T10:42:00.282489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:32.774644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147339+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:42:01.147339+0000","last_peered":"2024-02-26T10:42:01.147339+0000","last_clean":"2024-02-26T10:42:01.147339+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:42:01.147339+0000","last_undegraded":"2024-02-26T10:42:01.147339+0000","last_fullsized":"2024-02-26T10:42:01.147339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145312+0000","last_change":"2024-02-26T10:41:59.296775+0000","last_active":"2024-02-26T10:42:01.145312+0000","last_peered":"2024-02-26T10:42:01.145312+0000","last_clean":"2024-02-26T10:42:01.145312+0000","last_became_active":"2024-02-26T10:41:59.296431+0000","last_became_peered":"2024-02-26T10:41:59.296431+0000","last_unstale":"2024-02-26T10:42:01.145312+0000","last_undegraded":"2024-02-26T10:42:01.145312+0000","last_fullsized":"2024-02-26T10:42:01.145312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146925+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:42:01.146925+0000","last_peered":"2024-02-26T10:42:01.146925+0000","last_clean":"2024-02-26T10:42:01.146925+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:42:01.146925+0000","last_undegraded":"2024-02-26T10:42:01.146925+0000","last_fullsized":"2024-02-26T10:42:01.146925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:04.487859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283172+0000","last_change":"2024-02-26T10:41:59.300848+0000","last_active":"2024-02-26T10:42:00.283172+0000","last_peered":"2024-02-26T10:42:00.283172+0000","last_clean":"2024-02-26T10:42:00.283172+0000","last_became_active":"2024-02-26T10:41:59.300577+0000","last_became_peered":"2024-02-26T10:41:59.300577+0000","last_unstale":"2024-02-26T10:42:00.283172+0000","last_undegraded":"2024-02-26T10:42:00.283172+0000","last_fullsized":"2024-02-26T10:42:00.283172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146088+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:42:01.146088+0000","last_peered":"2024-02-26T10:42:01.146088+0000","last_clean":"2024-02-26T10:42:01.146088+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:42:01.146088+0000","last_undegraded":"2024-02-26T10:42:01.146088+0000","last_fullsized":"2024-02-26T10:42:01.146088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079430+0000","last_change":"2024-02-26T10:41:59.303988+0000","last_active":"2024-02-26T10:42:01.079430+0000","last_peered":"2024-02-26T10:42:01.079430+0000","last_clean":"2024-02-26T10:42:01.079430+0000","last_became_active":"2024-02-26T10:41:59.303677+0000","last_became_peered":"2024-02-26T10:41:59.303677+0000","last_unstale":"2024-02-26T10:42:01.079430+0000","last_undegraded":"2024-02-26T10:42:01.079430+0000","last_fullsized":"2024-02-26T10:42:01.079430+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:08.404204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145212+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:42:01.145212+0000","last_peered":"2024-02-26T10:42:01.145212+0000","last_clean":"2024-02-26T10:42:01.145212+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:42:01.145212+0000","last_undegraded":"2024-02-26T10:42:01.145212+0000","last_fullsized":"2024-02-26T10:42:01.145212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283321+0000","last_change":"2024-02-26T10:41:59.298661+0000","last_active":"2024-02-26T10:42:00.283321+0000","last_peered":"2024-02-26T10:42:00.283321+0000","last_clean":"2024-02-26T10:42:00.283321+0000","last_became_active":"2024-02-26T10:41:59.298384+0000","last_became_peered":"2024-02-26T10:41:59.298384+0000","last_unstale":"2024-02-26T10:42:00.283321+0000","last_undegraded":"2024-02-26T10:42:00.283321+0000","last_fullsized":"2024-02-26T10:42:00.283321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289264+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:42:00.289264+0000","last_peered":"2024-02-26T10:42:00.289264+0000","last_clean":"2024-02-26T10:42:00.289264+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:42:00.289264+0000","last_undegraded":"2024-02-26T10:42:00.289264+0000","last_fullsized":"2024-02-26T10:42:00.289264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:06.345056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284045+0000","last_change":"2024-02-26T10:41:59.303009+0000","last_active":"2024-02-26T10:42:00.284045+0000","last_peered":"2024-02-26T10:42:00.284045+0000","last_clean":"2024-02-26T10:42:00.284045+0000","last_became_active":"2024-02-26T10:41:59.301703+0000","last_became_peered":"2024-02-26T10:41:59.301703+0000","last_unstale":"2024-02-26T10:42:00.284045+0000","last_undegraded":"2024-02-26T10:42:00.284045+0000","last_fullsized":"2024-02-26T10:42:00.284045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281389+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:42:00.281389+0000","last_peered":"2024-02-26T10:42:00.281389+0000","last_clean":"2024-02-26T10:42:00.281389+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:42:00.281389+0000","last_undegraded":"2024-02-26T10:42:00.281389+0000","last_fullsized":"2024-02-26T10:42:00.281389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:15.119340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288109+0000","last_change":"2024-02-26T10:41:59.285657+0000","last_active":"2024-02-26T10:42:00.288109+0000","last_peered":"2024-02-26T10:42:00.288109+0000","last_clean":"2024-02-26T10:42:00.288109+0000","last_became_active":"2024-02-26T10:41:59.285365+0000","last_became_peered":"2024-02-26T10:41:59.285365+0000","last_unstale":"2024-02-26T10:42:00.288109+0000","last_undegraded":"2024-02-26T10:42:00.288109+0000","last_fullsized":"2024-02-26T10:42:00.288109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:10.689718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144425+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:42:01.144425+0000","last_peered":"2024-02-26T10:42:01.144425+0000","last_clean":"2024-02-26T10:42:01.144425+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:42:01.144425+0000","last_undegraded":"2024-02-26T10:42:01.144425+0000","last_fullsized":"2024-02-26T10:42:01.144425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288015+0000","last_change":"2024-02-26T10:41:59.299415+0000","last_active":"2024-02-26T10:42:00.288015+0000","last_peered":"2024-02-26T10:42:00.288015+0000","last_clean":"2024-02-26T10:42:00.288015+0000","last_became_active":"2024-02-26T10:41:59.298995+0000","last_became_peered":"2024-02-26T10:41:59.298995+0000","last_unstale":"2024-02-26T10:42:00.288015+0000","last_undegraded":"2024-02-26T10:42:00.288015+0000","last_fullsized":"2024-02-26T10:42:00.288015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:38.058564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282242+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:42:00.282242+0000","last_peered":"2024-02-26T10:42:00.282242+0000","last_clean":"2024-02-26T10:42:00.282242+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:42:00.282242+0000","last_undegraded":"2024-02-26T10:42:00.282242+0000","last_fullsized":"2024-02-26T10:42:00.282242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284598+0000","last_change":"2024-02-26T10:41:59.290032+0000","last_active":"2024-02-26T10:42:00.284598+0000","last_peered":"2024-02-26T10:42:00.284598+0000","last_clean":"2024-02-26T10:42:00.284598+0000","last_became_active":"2024-02-26T10:41:59.289604+0000","last_became_peered":"2024-02-26T10:41:59.289604+0000","last_unstale":"2024-02-26T10:42:00.284598+0000","last_undegraded":"2024-02-26T10:42:00.284598+0000","last_fullsized":"2024-02-26T10:42:00.284598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:23.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147525+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:42:01.147525+0000","last_peered":"2024-02-26T10:42:01.147525+0000","last_clean":"2024-02-26T10:42:01.147525+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:42:01.147525+0000","last_undegraded":"2024-02-26T10:42:01.147525+0000","last_fullsized":"2024-02-26T10:42:01.147525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147305+0000","last_change":"2024-02-26T10:41:59.299545+0000","last_active":"2024-02-26T10:42:01.147305+0000","last_peered":"2024-02-26T10:42:01.147305+0000","last_clean":"2024-02-26T10:42:01.147305+0000","last_became_active":"2024-02-26T10:41:59.298970+0000","last_became_peered":"2024-02-26T10:41:59.298970+0000","last_unstale":"2024-02-26T10:42:01.147305+0000","last_undegraded":"2024-02-26T10:42:01.147305+0000","last_fullsized":"2024-02-26T10:42:01.147305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283786+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:42:00.283786+0000","last_peered":"2024-02-26T10:42:00.283786+0000","last_clean":"2024-02-26T10:42:00.283786+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:42:00.283786+0000","last_undegraded":"2024-02-26T10:42:00.283786+0000","last_fullsized":"2024-02-26T10:42:00.283786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079654+0000","last_change":"2024-02-26T10:41:59.283472+0000","last_active":"2024-02-26T10:42:01.079654+0000","last_peered":"2024-02-26T10:42:01.079654+0000","last_clean":"2024-02-26T10:42:01.079654+0000","last_became_active":"2024-02-26T10:41:59.282978+0000","last_became_peered":"2024-02-26T10:41:59.282978+0000","last_unstale":"2024-02-26T10:42:01.079654+0000","last_undegraded":"2024-02-26T10:42:01.079654+0000","last_fullsized":"2024-02-26T10:42:01.079654+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:50.202422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283462+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:42:00.283462+0000","last_peered":"2024-02-26T10:42:00.283462+0000","last_clean":"2024-02-26T10:42:00.283462+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:42:00.283462+0000","last_undegraded":"2024-02-26T10:42:00.283462+0000","last_fullsized":"2024-02-26T10:42:00.283462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113746+0000","last_change":"2024-02-26T10:41:59.297414+0000","last_active":"2024-02-26T10:42:01.113746+0000","last_peered":"2024-02-26T10:42:01.113746+0000","last_clean":"2024-02-26T10:42:01.113746+0000","last_became_active":"2024-02-26T10:41:59.296486+0000","last_became_peered":"2024-02-26T10:41:59.296486+0000","last_unstale":"2024-02-26T10:42:01.113746+0000","last_undegraded":"2024-02-26T10:42:01.113746+0000","last_fullsized":"2024-02-26T10:42:01.113746+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:48.071684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281498+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:42:00.281498+0000","last_peered":"2024-02-26T10:42:00.281498+0000","last_clean":"2024-02-26T10:42:00.281498+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:42:00.281498+0000","last_undegraded":"2024-02-26T10:42:00.281498+0000","last_fullsized":"2024-02-26T10:42:00.281498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:46.825934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144660+0000","last_change":"2024-02-26T10:41:59.292484+0000","last_active":"2024-02-26T10:42:01.144660+0000","last_peered":"2024-02-26T10:42:01.144660+0000","last_clean":"2024-02-26T10:42:01.144660+0000","last_became_active":"2024-02-26T10:41:59.292140+0000","last_became_peered":"2024-02-26T10:41:59.292140+0000","last_unstale":"2024-02-26T10:42:01.144660+0000","last_undegraded":"2024-02-26T10:42:01.144660+0000","last_fullsized":"2024-02-26T10:42:01.144660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113819+0000","last_change":"2024-02-26T10:41:58.269128+0000","last_active":"2024-02-26T10:42:01.113819+0000","last_peered":"2024-02-26T10:42:01.113819+0000","last_clean":"2024-02-26T10:42:01.113819+0000","last_became_active":"2024-02-26T10:41:58.268746+0000","last_became_peered":"2024-02-26T10:41:58.268746+0000","last_unstale":"2024-02-26T10:42:01.113819+0000","last_undegraded":"2024-02-26T10:42:01.113819+0000","last_fullsized":"2024-02-26T10:42:01.113819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:46:53.257949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113664+0000","last_change":"2024-02-26T10:41:59.292780+0000","last_active":"2024-02-26T10:42:01.113664+0000","last_peered":"2024-02-26T10:42:01.113664+0000","last_clean":"2024-02-26T10:42:01.113664+0000","last_became_active":"2024-02-26T10:41:59.292526+0000","last_became_peered":"2024-02-26T10:41:59.292526+0000","last_unstale":"2024-02-26T10:42:01.113664+0000","last_undegraded":"2024-02-26T10:42:01.113664+0000","last_fullsized":"2024-02-26T10:42:01.113664+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:12.549758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113389+0000","last_change":"2024-02-26T10:41:58.269538+0000","last_active":"2024-02-26T10:42:01.113389+0000","last_peered":"2024-02-26T10:42:01.113389+0000","last_clean":"2024-02-26T10:42:01.113389+0000","last_became_active":"2024-02-26T10:41:58.269200+0000","last_became_peered":"2024-02-26T10:41:58.269200+0000","last_unstale":"2024-02-26T10:42:01.113389+0000","last_undegraded":"2024-02-26T10:42:01.113389+0000","last_fullsized":"2024-02-26T10:42:01.113389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:27.687984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144458+0000","last_change":"2024-02-26T10:41:59.305037+0000","last_active":"2024-02-26T10:42:01.144458+0000","last_peered":"2024-02-26T10:42:01.144458+0000","last_clean":"2024-02-26T10:42:01.144458+0000","last_became_active":"2024-02-26T10:41:59.304739+0000","last_became_peered":"2024-02-26T10:41:59.304739+0000","last_unstale":"2024-02-26T10:42:01.144458+0000","last_undegraded":"2024-02-26T10:42:01.144458+0000","last_fullsized":"2024-02-26T10:42:01.144458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.280914+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:42:00.280914+0000","last_peered":"2024-02-26T10:42:00.280914+0000","last_clean":"2024-02-26T10:42:00.280914+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:42:00.280914+0000","last_undegraded":"2024-02-26T10:42:00.280914+0000","last_fullsized":"2024-02-26T10:42:00.280914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:26.696259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079328+0000","last_change":"2024-02-26T10:41:59.289483+0000","last_active":"2024-02-26T10:42:01.079328+0000","last_peered":"2024-02-26T10:42:01.079328+0000","last_clean":"2024-02-26T10:42:01.079328+0000","last_became_active":"2024-02-26T10:41:59.288870+0000","last_became_peered":"2024-02-26T10:41:59.288870+0000","last_unstale":"2024-02-26T10:42:01.079328+0000","last_undegraded":"2024-02-26T10:42:01.079328+0000","last_fullsized":"2024-02-26T10:42:01.079328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:02.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290450+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:42:00.290450+0000","last_peered":"2024-02-26T10:42:00.290450+0000","last_clean":"2024-02-26T10:42:00.290450+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:42:00.290450+0000","last_undegraded":"2024-02-26T10:42:00.290450+0000","last_fullsized":"2024-02-26T10:42:00.290450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284373+0000","last_change":"2024-02-26T10:41:59.294199+0000","last_active":"2024-02-26T10:42:00.284373+0000","last_peered":"2024-02-26T10:42:00.284373+0000","last_clean":"2024-02-26T10:42:00.284373+0000","last_became_active":"2024-02-26T10:41:59.293906+0000","last_became_peered":"2024-02-26T10:41:59.293906+0000","last_unstale":"2024-02-26T10:42:00.284373+0000","last_undegraded":"2024-02-26T10:42:00.284373+0000","last_fullsized":"2024-02-26T10:42:00.284373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:27.711178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290377+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:42:00.290377+0000","last_peered":"2024-02-26T10:42:00.290377+0000","last_clean":"2024-02-26T10:42:00.290377+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:42:00.290377+0000","last_undegraded":"2024-02-26T10:42:00.290377+0000","last_fullsized":"2024-02-26T10:42:00.290377+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:02:34.734605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146776+0000","last_change":"2024-02-26T10:41:59.298323+0000","last_active":"2024-02-26T10:42:01.146776+0000","last_peered":"2024-02-26T10:42:01.146776+0000","last_clean":"2024-02-26T10:42:01.146776+0000","last_became_active":"2024-02-26T10:41:59.295932+0000","last_became_peered":"2024-02-26T10:41:59.295932+0000","last_unstale":"2024-02-26T10:42:01.146776+0000","last_undegraded":"2024-02-26T10:42:01.146776+0000","last_fullsized":"2024-02-26T10:42:01.146776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146596+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:42:01.146596+0000","last_peered":"2024-02-26T10:42:01.146596+0000","last_clean":"2024-02-26T10:42:01.146596+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:42:01.146596+0000","last_undegraded":"2024-02-26T10:42:01.146596+0000","last_fullsized":"2024-02-26T10:42:01.146596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079836+0000","last_change":"2024-02-26T10:41:59.293572+0000","last_active":"2024-02-26T10:42:01.079836+0000","last_peered":"2024-02-26T10:42:01.079836+0000","last_clean":"2024-02-26T10:42:01.079836+0000","last_became_active":"2024-02-26T10:41:59.292292+0000","last_became_peered":"2024-02-26T10:41:59.292292+0000","last_unstale":"2024-02-26T10:42:01.079836+0000","last_undegraded":"2024-02-26T10:42:01.079836+0000","last_fullsized":"2024-02-26T10:42:01.079836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:26.922522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147038+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:42:01.147038+0000","last_peered":"2024-02-26T10:42:01.147038+0000","last_clean":"2024-02-26T10:42:01.147038+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:42:01.147038+0000","last_undegraded":"2024-02-26T10:42:01.147038+0000","last_fullsized":"2024-02-26T10:42:01.147038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:14.749617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144982+0000","last_change":"2024-02-26T10:41:59.296354+0000","last_active":"2024-02-26T10:42:01.144982+0000","last_peered":"2024-02-26T10:42:01.144982+0000","last_clean":"2024-02-26T10:42:01.144982+0000","last_became_active":"2024-02-26T10:41:59.296012+0000","last_became_peered":"2024-02-26T10:41:59.296012+0000","last_unstale":"2024-02-26T10:42:01.144982+0000","last_undegraded":"2024-02-26T10:42:01.144982+0000","last_fullsized":"2024-02-26T10:42:01.144982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284034+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:42:00.284034+0000","last_peered":"2024-02-26T10:42:00.284034+0000","last_clean":"2024-02-26T10:42:00.284034+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:42:00.284034+0000","last_undegraded":"2024-02-26T10:42:00.284034+0000","last_fullsized":"2024-02-26T10:42:00.284034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:50:59.443811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284365+0000","last_change":"2024-02-26T10:41:59.297360+0000","last_active":"2024-02-26T10:42:00.284365+0000","last_peered":"2024-02-26T10:42:00.284365+0000","last_clean":"2024-02-26T10:42:00.284365+0000","last_became_active":"2024-02-26T10:41:59.297031+0000","last_became_peered":"2024-02-26T10:41:59.297031+0000","last_unstale":"2024-02-26T10:42:00.284365+0000","last_undegraded":"2024-02-26T10:42:00.284365+0000","last_fullsized":"2024-02-26T10:42:00.284365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078834+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:42:01.078834+0000","last_peered":"2024-02-26T10:42:01.078834+0000","last_clean":"2024-02-26T10:42:01.078834+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:42:01.078834+0000","last_undegraded":"2024-02-26T10:42:01.078834+0000","last_fullsized":"2024-02-26T10:42:01.078834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:58:22.045852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283703+0000","last_change":"2024-02-26T10:41:59.295832+0000","last_active":"2024-02-26T10:42:00.283703+0000","last_peered":"2024-02-26T10:42:00.283703+0000","last_clean":"2024-02-26T10:42:00.283703+0000","last_became_active":"2024-02-26T10:41:59.295548+0000","last_became_peered":"2024-02-26T10:41:59.295548+0000","last_unstale":"2024-02-26T10:42:00.283703+0000","last_undegraded":"2024-02-26T10:42:00.283703+0000","last_fullsized":"2024-02-26T10:42:00.283703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146232+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:42:01.146232+0000","last_peered":"2024-02-26T10:42:01.146232+0000","last_clean":"2024-02-26T10:42:01.146232+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:42:01.146232+0000","last_undegraded":"2024-02-26T10:42:01.146232+0000","last_fullsized":"2024-02-26T10:42:01.146232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145956+0000","last_change":"2024-02-26T10:41:59.300681+0000","last_active":"2024-02-26T10:42:01.145956+0000","last_peered":"2024-02-26T10:42:01.145956+0000","last_clean":"2024-02-26T10:42:01.145956+0000","last_became_active":"2024-02-26T10:41:59.299849+0000","last_became_peered":"2024-02-26T10:41:59.299849+0000","last_unstale":"2024-02-26T10:42:01.145956+0000","last_undegraded":"2024-02-26T10:42:01.145956+0000","last_fullsized":"2024-02-26T10:42:01.145956+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078965+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:42:01.078965+0000","last_peered":"2024-02-26T10:42:01.078965+0000","last_clean":"2024-02-26T10:42:01.078965+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:42:01.078965+0000","last_undegraded":"2024-02-26T10:42:01.078965+0000","last_fullsized":"2024-02-26T10:42:01.078965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:40:26.347537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290123+0000","last_change":"2024-02-26T10:41:59.295485+0000","last_active":"2024-02-26T10:42:00.290123+0000","last_peered":"2024-02-26T10:42:00.290123+0000","last_clean":"2024-02-26T10:42:00.290123+0000","last_became_active":"2024-02-26T10:41:59.295154+0000","last_became_peered":"2024-02-26T10:41:59.295154+0000","last_unstale":"2024-02-26T10:42:00.290123+0000","last_undegraded":"2024-02-26T10:42:00.290123+0000","last_fullsized":"2024-02-26T10:42:00.290123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:15.764536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284516+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:42:00.284516+0000","last_peered":"2024-02-26T10:42:00.284516+0000","last_clean":"2024-02-26T10:42:00.284516+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:42:00.284516+0000","last_undegraded":"2024-02-26T10:42:00.284516+0000","last_fullsized":"2024-02-26T10:42:00.284516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284158+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:42:00.284158+0000","last_peered":"2024-02-26T10:42:00.284158+0000","last_clean":"2024-02-26T10:42:00.284158+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:42:00.284158+0000","last_undegraded":"2024-02-26T10:42:00.284158+0000","last_fullsized":"2024-02-26T10:42:00.284158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1004431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":414687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":415627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":413993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":415605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":414575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":414187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1004395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:04.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-02-26T10:42:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.132+0000 7f9b03200640 1 Processor -- start 2024-02-26T10:42:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.132+0000 7f9b03200640 1 -- start start 2024-02-26T10:42:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.134+0000 7f9b03200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b040ab540 0x7f9b04154e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.134+0000 7f9b03200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b040c30c0 0x7f9b040c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.134+0000 7f9b03200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b04158f30 con 0x7f9b0406e860 2024-02-26T10:42:05.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.134+0000 7f9b03200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b041553d0 con 0x7f9b040c30c0 2024-02-26T10:42:05.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.134+0000 7f9b03200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04159230 con 0x7f9b040ab540 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b01400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9b0406e860 0x7f9b040c1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56128/0 (socket says 172.21.15.67:56128) 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b01400640 1 -- 172.21.15.67:0/2643642973 learned_addr learned my addr 172.21.15.67:0/2643642973 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b01e00640 1 --2- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b040ab540 0x7f9b04154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b02800640 1 --2- 172.21.15.67:0/2643642973 >> v2:172.21.15.82:3300/0 conn(0x7f9b040c30c0 0x7f9b040c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b00a00640 1 -- 172.21.15.67:0/2643642973 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320322394 0 0) 0x7f9b04158f30 con 0x7f9b0406e860 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b00a00640 1 -- 172.21.15.67:0/2643642973 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003520 con 0x7f9b0406e860 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b02800640 1 -- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b040ab540 msgr2=0x7f9b04154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b02800640 1 --2- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b040ab540 0x7f9b04154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b00a00640 1 -- 172.21.15.67:0/2643642973 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3459001691 0 0) 0x7f9ae8003520 con 0x7f9b0406e860 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.135+0000 7f9b02800640 1 -- 172.21.15.67:0/2643642973 >> v1:172.21.15.67:6789/0 conn(0x7f9b0406e860 legacy=0x7f9b040c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.136+0000 7f9b02800640 1 -- 172.21.15.67:0/2643642973 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040c1fa0 con 0x7f9b040c30c0 2024-02-26T10:42:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.137+0000 7f9b02800640 1 --2- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040c30c0 0x7f9b040c34f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009d90 tx=0x7f9afc02bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd642be5017352f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.137+0000 7f9b00a00640 1 -- 172.21.15.67:0/2643642973 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc039070 con 0x7f9b040c30c0 2024-02-26T10:42:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.137+0000 7f9b00a00640 1 -- 172.21.15.67:0/2643642973 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc02c810 con 0x7f9b040c30c0 2024-02-26T10:42:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.137+0000 7f9b00a00640 1 -- 172.21.15.67:0/2643642973 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4001670 con 0x7f9b040c30c0 2024-02-26T10:42:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 -- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040c30c0 msgr2=0x7f9b040c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 --2- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040c30c0 0x7f9b040c34f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009d90 tx=0x7f9afc02bce0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 -- 172.21.15.67:0/2643642973 shutdown_connections 2024-02-26T10:42:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 --2- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040c30c0 0x7f9b040c34f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 --2- 172.21.15.67:0/2643642973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b040ab540 0x7f9b04154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 -- 172.21.15.67:0/2643642973 >> 172.21.15.67:0/2643642973 conn(0x7f9b040a84a0 msgr2=0x7f9b040a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 -- 172.21.15.67:0/2643642973 shutdown_connections 2024-02-26T10:42:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.138+0000 7f9b03200640 1 -- 172.21.15.67:0/2643642973 wait complete. 2024-02-26T10:42:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.140+0000 7f9b03200640 1 Processor -- start 2024-02-26T10:42:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.140+0000 7f9b03200640 1 -- start start 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b03200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b0406e860 0x7f9b04152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b03200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040ab540 0x7f9b041ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b03200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 0x7f9b04071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b03200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0406e3e0 con 0x7f9b040c30c0 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b03200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040c0290 con 0x7f9b040ab540 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b03200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040c0570 con 0x7f9b0406e860 2024-02-26T10:42:05.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b02800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 0x7f9b04071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b02800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 0x7f9b04071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43474/0 (socket says 172.21.15.67:43474) 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.141+0000 7f9b02800640 1 -- 172.21.15.67:0/199851855 learned_addr learned my addr 172.21.15.67:0/199851855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b01e00640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b0406e860 0x7f9b04152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b01400640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040ab540 0x7f9b041ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b02800640 1 -- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b0406e860 msgr2=0x7f9b04152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b02800640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b0406e860 0x7f9b04152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b02800640 1 -- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040ab540 msgr2=0x7f9b041ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b02800640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040ab540 0x7f9b041ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b02800640 1 -- 172.21.15.67:0/199851855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04072ec0 con 0x7f9b040c30c0 2024-02-26T10:42:05.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b02800640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 0x7f9b04071f30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9afc037040 tx=0x7f9afc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc039050 con 0x7f9b040c30c0 2024-02-26T10:42:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc045070 con 0x7f9b040c30c0 2024-02-26T10:42:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc009450 con 0x7f9b040c30c0 2024-02-26T10:42:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.142+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b040731a0 con 0x7f9b040c30c0 2024-02-26T10:42:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.143+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b041edec0 con 0x7f9b040c30c0 2024-02-26T10:42:05.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.144+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04074a30 con 0x7f9b040c30c0 2024-02-26T10:42:05.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.145+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9afc040690 con 0x7f9b040c30c0 2024-02-26T10:42:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.146+0000 7f9afa000640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ae006b500 0x7f9ae006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.146+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9afc0ac9e0 con 0x7f9b040c30c0 2024-02-26T10:42:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.146+0000 7f9b01e00640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ae006b500 0x7f9ae006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.148+0000 7f9b01e00640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ae006b500 0x7f9ae006d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9b040a8ea0 tx=0x7f9aec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.151+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b040c1fa0 con 0x7f9b040c30c0 2024-02-26T10:42:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.427+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9b040c0a60 con 0x7f9b040c30c0 2024-02-26T10:42:05.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.428+0000 7f9afa000640 1 -- 172.21.15.67:0/199851855 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1303 (secure 0 0 0) 0x7f9afc032340 con 0x7f9b040c30c0 2024-02-26T10:42:05.426 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:05.427 INFO:teuthology.orchestra.run.smithi067.stdout:{"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":230,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":76,"num_osds":8,"num_up_osds":8,"osd_up_since":1708943907,"num_in_osds":8,"osd_in_since":1708943898,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":255627264,"bytes_avail":772838486016,"bytes_total":773094113280,"write_bytes_sec":2323,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":48,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":6450}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-02-26T10:41:40.396166+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-02-26T10:42:05.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ae006b500 msgr2=0x7f9ae006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ae006b500 0x7f9ae006d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9b040a8ea0 tx=0x7f9aec008040 comp rx=0 tx=0).stop 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 msgr2=0x7f9b04071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 0x7f9b04071f30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9afc037040 tx=0x7f9afc004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 shutdown_connections 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b040c30c0 0x7f9b04071f30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9ae006b500 0x7f9ae006d9c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b040ab540 0x7f9b041ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 --2- 172.21.15.67:0/199851855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b0406e860 0x7f9b04152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 >> 172.21.15.67:0/199851855 conn(0x7f9b040a84a0 msgr2=0x7f9b040aa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 shutdown_connections 2024-02-26T10:42:05.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.431+0000 7f9b03200640 1 -- 172.21.15.67:0/199851855 wait complete. 2024-02-26T10:42:05.450 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-02-26T10:42:05.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.678+0000 7f3d1be00640 1 Processor -- start 2024-02-26T10:42:05.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.679+0000 7f3d1be00640 1 -- start start 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.680+0000 7f3d1be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3d1c0a94a0 0x7f3d1c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.680+0000 7f3d1be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c06df70 con 0x7f3d1c06db00 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.680+0000 7f3d1be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c0a9a80 con 0x7f3d1c06ac20 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.680+0000 7f3d1be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c0a9d60 con 0x7f3d1c0a94a0 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d13e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3d1c0a94a0 0x7f3d1c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d13e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3d1c0a94a0 0x7f3d1c06b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43486/0 (socket says 172.21.15.67:43486) 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d1b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3d1c06db00 0x7f3d1c0abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56138/0 (socket says 172.21.15.67:56138) 2024-02-26T10:42:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d13e00640 1 -- 172.21.15.67:0/1826034932 learned_addr learned my addr 172.21.15.67:0/1826034932 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d13e00640 1 -- 172.21.15.67:0/1826034932 >> v1:172.21.15.82:6789/0 conn(0x7f3d1c06ac20 legacy=0x7f3d1c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d13e00640 1 -- 172.21.15.67:0/1826034932 >> v1:172.21.15.67:6789/0 conn(0x7f3d1c06db00 legacy=0x7f3d1c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d13e00640 1 -- 172.21.15.67:0/1826034932 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c0ab750 con 0x7f3d1c0a94a0 2024-02-26T10:42:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.681+0000 7f3d1a000640 1 -- 172.21.15.67:0/1826034932 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770866010 0 0) 0x7f3d1c06df70 con 0x7f3d1c06db00 2024-02-26T10:42:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.682+0000 7f3d13e00640 1 --2- 172.21.15.67:0/1826034932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c0a94a0 0x7f3d1c06b000 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009870 tx=0x7f3d0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f3784c90f24de4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.683+0000 7f3d1a000640 1 -- 172.21.15.67:0/1826034932 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0402f840 con 0x7f3d1c0a94a0 2024-02-26T10:42:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.683+0000 7f3d1a000640 1 -- 172.21.15.67:0/1826034932 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0402f9e0 con 0x7f3d1c0a94a0 2024-02-26T10:42:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.683+0000 7f3d1a000640 1 -- 172.21.15.67:0/1826034932 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0402fdb0 con 0x7f3d1c0a94a0 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.683+0000 7f3d1be00640 1 -- 172.21.15.67:0/1826034932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c0a94a0 msgr2=0x7f3d1c06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.683+0000 7f3d1be00640 1 --2- 172.21.15.67:0/1826034932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c0a94a0 0x7f3d1c06b000 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009870 tx=0x7f3d0402ba20 comp rx=0 tx=0).stop 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.684+0000 7f3d1be00640 1 -- 172.21.15.67:0/1826034932 shutdown_connections 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.684+0000 7f3d1be00640 1 --2- 172.21.15.67:0/1826034932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c0a94a0 0x7f3d1c06b000 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.684+0000 7f3d1be00640 1 -- 172.21.15.67:0/1826034932 >> 172.21.15.67:0/1826034932 conn(0x7f3d1c0c63f0 msgr2=0x7f3d1c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.684+0000 7f3d1be00640 1 -- 172.21.15.67:0/1826034932 shutdown_connections 2024-02-26T10:42:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.684+0000 7f3d1be00640 1 -- 172.21.15.67:0/1826034932 wait complete. 2024-02-26T10:42:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.685+0000 7f3d1be00640 1 Processor -- start 2024-02-26T10:42:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.685+0000 7f3d1be00640 1 -- start start 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.686+0000 7f3d1be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 0x7f3d1c142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.686+0000 7f3d1be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d1c06db00 0x7f3d1c0c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.686+0000 7f3d1be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1c0a94a0 0x7f3d1c149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.686+0000 7f3d1be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c150590 con 0x7f3d1c06ac20 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.686+0000 7f3d1be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c150410 con 0x7f3d1c0a94a0 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.686+0000 7f3d1be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c15e520 con 0x7f3d1c06db00 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 0x7f3d1c142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 0x7f3d1c142230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43488/0 (socket says 172.21.15.67:43488) 2024-02-26T10:42:05.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 -- 172.21.15.67:0/3483293522 learned_addr learned my addr 172.21.15.67:0/3483293522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 -- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d1c06db00 msgr2=0x7f3d1c0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d1c06db00 0x7f3d1c0c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 -- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1c0a94a0 msgr2=0x7f3d1c149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1c0a94a0 0x7f3d1c149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 -- 172.21.15.67:0/3483293522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c1514b0 con 0x7f3d1c06ac20 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.687+0000 7f3d1aa00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 0x7f3d1c142230 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c480 tx=0x7f3d0c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.688+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0052d0 con 0x7f3d1c06ac20 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.688+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c005470 con 0x7f3d1c06ac20 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.688+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c151790 con 0x7f3d1c06ac20 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.688+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c012870 con 0x7f3d1c06ac20 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.688+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c0c5910 con 0x7f3d1c06ac20 2024-02-26T10:42:05.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.689+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c074a30 con 0x7f3d1c06ac20 2024-02-26T10:42:05.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.690+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d0c012a10 con 0x7f3d1c06ac20 2024-02-26T10:42:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.691+0000 7f3d12a00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3cf806b4d0 0x7f3cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.691+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d0c089f10 con 0x7f3d1c06ac20 2024-02-26T10:42:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.691+0000 7f3d13e00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3cf806b4d0 0x7f3cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.693+0000 7f3d13e00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3cf806b4d0 0x7f3cf806d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d04036000 tx=0x7f3d04039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:05.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.696+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d0c058d00 con 0x7f3d1c06ac20 2024-02-26T10:42:05.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.890+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d1c06b9b0 con 0x7f3cf806b4d0 2024-02-26T10:42:05.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.909+0000 7f3d12a00640 1 -- 172.21.15.67:0/3483293522 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252078 (secure 0 0 0) 0x7f3d1c06b9b0 con 0x7f3cf806b4d0 2024-02-26T10:42:05.911 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:05.912 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3cf806b4d0 msgr2=0x7f3cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3cf806b4d0 0x7f3cf806d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d04036000 tx=0x7f3d04039040 comp rx=0 tx=0).stop 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 msgr2=0x7f3d1c142230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 0x7f3d1c142230 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00c480 tx=0x7f3d0c00c940 comp rx=0 tx=0).stop 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 shutdown_connections 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3cf806b4d0 0x7f3cf806d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1c0a94a0 0x7f3d1c149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d1c06db00 0x7f3d1c0c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.914+0000 7f3d1be00640 1 --2- 172.21.15.67:0/3483293522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d1c06ac20 0x7f3d1c142230 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:05.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.915+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 >> 172.21.15.67:0/3483293522 conn(0x7f3d1c0c63f0 msgr2=0x7f3d1c153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:05.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.915+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 shutdown_connections 2024-02-26T10:42:05.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:05.915+0000 7f3d1be00640 1 -- 172.21.15.67:0/3483293522 wait complete. 2024-02-26T10:42:05.932 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-02-26T10:42:04.403661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249636,"kb_used_data":8488,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725084,"statfs":{"total":773094113280,"available":772838486016,"internally_reserved":0,"allocated":8691712,"data_stored":4497500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.169336"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146868+0000","last_change":"2024-02-26T10:41:59.308476+0000","last_active":"2024-02-26T10:42:01.146868+0000","last_peered":"2024-02-26T10:42:01.146868+0000","last_clean":"2024-02-26T10:42:01.146868+0000","last_became_active":"2024-02-26T10:41:59.307995+0000","last_became_peered":"2024-02-26T10:41:59.307995+0000","last_unstale":"2024-02-26T10:42:01.146868+0000","last_undegraded":"2024-02-26T10:42:01.146868+0000","last_fullsized":"2024-02-26T10:42:01.146868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290470+0000","last_change":"2024-02-26T10:41:59.300851+0000","last_active":"2024-02-26T10:42:00.290470+0000","last_peered":"2024-02-26T10:42:00.290470+0000","last_clean":"2024-02-26T10:42:00.290470+0000","last_became_active":"2024-02-26T10:41:59.300574+0000","last_became_peered":"2024-02-26T10:41:59.300574+0000","last_unstale":"2024-02-26T10:42:00.290470+0000","last_undegraded":"2024-02-26T10:42:00.290470+0000","last_fullsized":"2024-02-26T10:42:00.290470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:04.528547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284215+0000","last_change":"2024-02-26T10:41:59.293058+0000","last_active":"2024-02-26T10:42:00.284215+0000","last_peered":"2024-02-26T10:42:00.284215+0000","last_clean":"2024-02-26T10:42:00.284215+0000","last_became_active":"2024-02-26T10:41:59.292789+0000","last_became_peered":"2024-02-26T10:41:59.292789+0000","last_unstale":"2024-02-26T10:42:00.284215+0000","last_undegraded":"2024-02-26T10:42:00.284215+0000","last_fullsized":"2024-02-26T10:42:00.284215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114472+0000","last_change":"2024-02-26T10:41:59.296577+0000","last_active":"2024-02-26T10:42:01.114472+0000","last_peered":"2024-02-26T10:42:01.114472+0000","last_clean":"2024-02-26T10:42:01.114472+0000","last_became_active":"2024-02-26T10:41:59.296264+0000","last_became_peered":"2024-02-26T10:41:59.296264+0000","last_unstale":"2024-02-26T10:42:01.114472+0000","last_undegraded":"2024-02-26T10:42:01.114472+0000","last_fullsized":"2024-02-26T10:42:01.114472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:13.257965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079286+0000","last_change":"2024-02-26T10:41:59.298038+0000","last_active":"2024-02-26T10:42:01.079286+0000","last_peered":"2024-02-26T10:42:01.079286+0000","last_clean":"2024-02-26T10:42:01.079286+0000","last_became_active":"2024-02-26T10:41:59.297699+0000","last_became_peered":"2024-02-26T10:41:59.297699+0000","last_unstale":"2024-02-26T10:42:01.079286+0000","last_undegraded":"2024-02-26T10:42:01.079286+0000","last_fullsized":"2024-02-26T10:42:01.079286+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:30.879707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145586+0000","last_change":"2024-02-26T10:41:59.295626+0000","last_active":"2024-02-26T10:42:01.145586+0000","last_peered":"2024-02-26T10:42:01.145586+0000","last_clean":"2024-02-26T10:42:01.145586+0000","last_became_active":"2024-02-26T10:41:59.295266+0000","last_became_peered":"2024-02-26T10:41:59.295266+0000","last_unstale":"2024-02-26T10:42:01.145586+0000","last_undegraded":"2024-02-26T10:42:01.145586+0000","last_fullsized":"2024-02-26T10:42:01.145586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113506+0000","last_change":"2024-02-26T10:41:59.287398+0000","last_active":"2024-02-26T10:42:01.113506+0000","last_peered":"2024-02-26T10:42:01.113506+0000","last_clean":"2024-02-26T10:42:01.113506+0000","last_became_active":"2024-02-26T10:41:59.286305+0000","last_became_peered":"2024-02-26T10:41:59.286305+0000","last_unstale":"2024-02-26T10:42:01.113506+0000","last_undegraded":"2024-02-26T10:42:01.113506+0000","last_fullsized":"2024-02-26T10:42:01.113506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:54.865254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144514+0000","last_change":"2024-02-26T10:41:59.307667+0000","last_active":"2024-02-26T10:42:01.144514+0000","last_peered":"2024-02-26T10:42:01.144514+0000","last_clean":"2024-02-26T10:42:01.144514+0000","last_became_active":"2024-02-26T10:41:59.307319+0000","last_became_peered":"2024-02-26T10:41:59.307319+0000","last_unstale":"2024-02-26T10:42:01.144514+0000","last_undegraded":"2024-02-26T10:42:01.144514+0000","last_fullsized":"2024-02-26T10:42:01.144514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144909+0000","last_change":"2024-02-26T10:41:59.304927+0000","last_active":"2024-02-26T10:42:01.144909+0000","last_peered":"2024-02-26T10:42:01.144909+0000","last_clean":"2024-02-26T10:42:01.144909+0000","last_became_active":"2024-02-26T10:41:59.304641+0000","last_became_peered":"2024-02-26T10:41:59.304641+0000","last_unstale":"2024-02-26T10:42:01.144909+0000","last_undegraded":"2024-02-26T10:42:01.144909+0000","last_fullsized":"2024-02-26T10:42:01.144909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146925+0000","last_change":"2024-02-26T10:41:59.293335+0000","last_active":"2024-02-26T10:42:01.146925+0000","last_peered":"2024-02-26T10:42:01.146925+0000","last_clean":"2024-02-26T10:42:01.146925+0000","last_became_active":"2024-02-26T10:41:59.292937+0000","last_became_peered":"2024-02-26T10:41:59.292937+0000","last_unstale":"2024-02-26T10:42:01.146925+0000","last_undegraded":"2024-02-26T10:42:01.146925+0000","last_fullsized":"2024-02-26T10:42:01.146925+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289248+0000","last_change":"2024-02-26T10:41:59.282216+0000","last_active":"2024-02-26T10:42:00.289248+0000","last_peered":"2024-02-26T10:42:00.289248+0000","last_clean":"2024-02-26T10:42:00.289248+0000","last_became_active":"2024-02-26T10:41:59.281891+0000","last_became_peered":"2024-02-26T10:41:59.281891+0000","last_unstale":"2024-02-26T10:42:00.289248+0000","last_undegraded":"2024-02-26T10:42:00.289248+0000","last_fullsized":"2024-02-26T10:42:00.289248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:07.272911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146079+0000","last_change":"2024-02-26T10:41:59.290942+0000","last_active":"2024-02-26T10:42:01.146079+0000","last_peered":"2024-02-26T10:42:01.146079+0000","last_clean":"2024-02-26T10:42:01.146079+0000","last_became_active":"2024-02-26T10:41:59.288616+0000","last_became_peered":"2024-02-26T10:41:59.288616+0000","last_unstale":"2024-02-26T10:42:01.146079+0000","last_undegraded":"2024-02-26T10:42:01.146079+0000","last_fullsized":"2024-02-26T10:42:01.146079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284058+0000","last_change":"2024-02-26T10:41:59.281911+0000","last_active":"2024-02-26T10:42:00.284058+0000","last_peered":"2024-02-26T10:42:00.284058+0000","last_clean":"2024-02-26T10:42:00.284058+0000","last_became_active":"2024-02-26T10:41:59.281270+0000","last_became_peered":"2024-02-26T10:41:59.281270+0000","last_unstale":"2024-02-26T10:42:00.284058+0000","last_undegraded":"2024-02-26T10:42:00.284058+0000","last_fullsized":"2024-02-26T10:42:00.284058+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113712+0000","last_change":"2024-02-26T10:41:59.310387+0000","last_active":"2024-02-26T10:42:01.113712+0000","last_peered":"2024-02-26T10:42:01.113712+0000","last_clean":"2024-02-26T10:42:01.113712+0000","last_became_active":"2024-02-26T10:41:59.310111+0000","last_became_peered":"2024-02-26T10:41:59.310111+0000","last_unstale":"2024-02-26T10:42:01.113712+0000","last_undegraded":"2024-02-26T10:42:01.113712+0000","last_fullsized":"2024-02-26T10:42:01.113712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:53:52.167978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281188+0000","last_change":"2024-02-26T10:41:59.289328+0000","last_active":"2024-02-26T10:42:00.281188+0000","last_peered":"2024-02-26T10:42:00.281188+0000","last_clean":"2024-02-26T10:42:00.281188+0000","last_became_active":"2024-02-26T10:41:59.288951+0000","last_became_peered":"2024-02-26T10:41:59.288951+0000","last_unstale":"2024-02-26T10:42:00.281188+0000","last_undegraded":"2024-02-26T10:42:00.281188+0000","last_fullsized":"2024-02-26T10:42:00.281188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:00.613487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078476+0000","last_change":"2024-02-26T10:41:59.290536+0000","last_active":"2024-02-26T10:42:01.078476+0000","last_peered":"2024-02-26T10:42:01.078476+0000","last_clean":"2024-02-26T10:42:01.078476+0000","last_became_active":"2024-02-26T10:41:59.289661+0000","last_became_peered":"2024-02-26T10:41:59.289661+0000","last_unstale":"2024-02-26T10:42:01.078476+0000","last_undegraded":"2024-02-26T10:42:01.078476+0000","last_fullsized":"2024-02-26T10:42:01.078476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:56.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079115+0000","last_change":"2024-02-26T10:41:59.291203+0000","last_active":"2024-02-26T10:42:01.079115+0000","last_peered":"2024-02-26T10:42:01.079115+0000","last_clean":"2024-02-26T10:42:01.079115+0000","last_became_active":"2024-02-26T10:41:59.290916+0000","last_became_peered":"2024-02-26T10:41:59.290916+0000","last_unstale":"2024-02-26T10:42:01.079115+0000","last_undegraded":"2024-02-26T10:42:01.079115+0000","last_fullsized":"2024-02-26T10:42:01.079115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:09.397334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114840+0000","last_change":"2024-02-26T10:41:59.291282+0000","last_active":"2024-02-26T10:42:01.114840+0000","last_peered":"2024-02-26T10:42:01.114840+0000","last_clean":"2024-02-26T10:42:01.114840+0000","last_became_active":"2024-02-26T10:41:59.291008+0000","last_became_peered":"2024-02-26T10:41:59.291008+0000","last_unstale":"2024-02-26T10:42:01.114840+0000","last_undegraded":"2024-02-26T10:42:01.114840+0000","last_fullsized":"2024-02-26T10:42:01.114840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:55:38.972893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283816+0000","last_change":"2024-02-26T10:41:59.289830+0000","last_active":"2024-02-26T10:42:00.283816+0000","last_peered":"2024-02-26T10:42:00.283816+0000","last_clean":"2024-02-26T10:42:00.283816+0000","last_became_active":"2024-02-26T10:41:59.289466+0000","last_became_peered":"2024-02-26T10:41:59.289466+0000","last_unstale":"2024-02-26T10:42:00.283816+0000","last_undegraded":"2024-02-26T10:42:00.283816+0000","last_fullsized":"2024-02-26T10:42:00.283816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114572+0000","last_change":"2024-02-26T10:41:59.294466+0000","last_active":"2024-02-26T10:42:01.114572+0000","last_peered":"2024-02-26T10:42:01.114572+0000","last_clean":"2024-02-26T10:42:01.114572+0000","last_became_active":"2024-02-26T10:41:59.294199+0000","last_became_peered":"2024-02-26T10:41:59.294199+0000","last_unstale":"2024-02-26T10:42:01.114572+0000","last_undegraded":"2024-02-26T10:42:01.114572+0000","last_fullsized":"2024-02-26T10:42:01.114572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:06.679617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114210+0000","last_change":"2024-02-26T10:41:59.302058+0000","last_active":"2024-02-26T10:42:01.114210+0000","last_peered":"2024-02-26T10:42:01.114210+0000","last_clean":"2024-02-26T10:42:01.114210+0000","last_became_active":"2024-02-26T10:41:59.301773+0000","last_became_peered":"2024-02-26T10:41:59.301773+0000","last_unstale":"2024-02-26T10:42:01.114210+0000","last_undegraded":"2024-02-26T10:42:01.114210+0000","last_fullsized":"2024-02-26T10:42:01.114210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:58.489330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289203+0000","last_change":"2024-02-26T10:41:59.306729+0000","last_active":"2024-02-26T10:42:00.289203+0000","last_peered":"2024-02-26T10:42:00.289203+0000","last_clean":"2024-02-26T10:42:00.289203+0000","last_became_active":"2024-02-26T10:41:59.306464+0000","last_became_peered":"2024-02-26T10:41:59.306464+0000","last_unstale":"2024-02-26T10:42:00.289203+0000","last_undegraded":"2024-02-26T10:42:00.289203+0000","last_fullsized":"2024-02-26T10:42:00.289203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:56.409926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145960+0000","last_change":"2024-02-26T10:41:59.304328+0000","last_active":"2024-02-26T10:42:01.145960+0000","last_peered":"2024-02-26T10:42:01.145960+0000","last_clean":"2024-02-26T10:42:01.145960+0000","last_became_active":"2024-02-26T10:41:59.304025+0000","last_became_peered":"2024-02-26T10:41:59.304025+0000","last_unstale":"2024-02-26T10:42:01.145960+0000","last_undegraded":"2024-02-26T10:42:01.145960+0000","last_fullsized":"2024-02-26T10:42:01.145960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113484+0000","last_change":"2024-02-26T10:41:59.303897+0000","last_active":"2024-02-26T10:42:01.113484+0000","last_peered":"2024-02-26T10:42:01.113484+0000","last_clean":"2024-02-26T10:42:01.113484+0000","last_became_active":"2024-02-26T10:41:59.301812+0000","last_became_peered":"2024-02-26T10:41:59.301812+0000","last_unstale":"2024-02-26T10:42:01.113484+0000","last_undegraded":"2024-02-26T10:42:01.113484+0000","last_fullsized":"2024-02-26T10:42:01.113484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:03:57.549048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290341+0000","last_change":"2024-02-26T10:41:59.297092+0000","last_active":"2024-02-26T10:42:00.290341+0000","last_peered":"2024-02-26T10:42:00.290341+0000","last_clean":"2024-02-26T10:42:00.290341+0000","last_became_active":"2024-02-26T10:41:59.295816+0000","last_became_peered":"2024-02-26T10:41:59.295816+0000","last_unstale":"2024-02-26T10:42:00.290341+0000","last_undegraded":"2024-02-26T10:42:00.290341+0000","last_fullsized":"2024-02-26T10:42:00.290341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:05.502778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284127+0000","last_change":"2024-02-26T10:41:59.307675+0000","last_active":"2024-02-26T10:42:00.284127+0000","last_peered":"2024-02-26T10:42:00.284127+0000","last_clean":"2024-02-26T10:42:00.284127+0000","last_became_active":"2024-02-26T10:41:59.307360+0000","last_became_peered":"2024-02-26T10:41:59.307360+0000","last_unstale":"2024-02-26T10:42:00.284127+0000","last_undegraded":"2024-02-26T10:42:00.284127+0000","last_fullsized":"2024-02-26T10:42:00.284127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079335+0000","last_change":"2024-02-26T10:41:59.287990+0000","last_active":"2024-02-26T10:42:01.079335+0000","last_peered":"2024-02-26T10:42:01.079335+0000","last_clean":"2024-02-26T10:42:01.079335+0000","last_became_active":"2024-02-26T10:41:59.286925+0000","last_became_peered":"2024-02-26T10:41:59.286925+0000","last_unstale":"2024-02-26T10:42:01.079335+0000","last_undegraded":"2024-02-26T10:42:01.079335+0000","last_fullsized":"2024-02-26T10:42:01.079335+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:08.646623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146292+0000","last_change":"2024-02-26T10:41:59.297459+0000","last_active":"2024-02-26T10:42:01.146292+0000","last_peered":"2024-02-26T10:42:01.146292+0000","last_clean":"2024-02-26T10:42:01.146292+0000","last_became_active":"2024-02-26T10:41:59.296494+0000","last_became_peered":"2024-02-26T10:41:59.296494+0000","last_unstale":"2024-02-26T10:42:01.146292+0000","last_undegraded":"2024-02-26T10:42:01.146292+0000","last_fullsized":"2024-02-26T10:42:01.146292+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289123+0000","last_change":"2024-02-26T10:41:59.288740+0000","last_active":"2024-02-26T10:42:00.289123+0000","last_peered":"2024-02-26T10:42:00.289123+0000","last_clean":"2024-02-26T10:42:00.289123+0000","last_became_active":"2024-02-26T10:41:59.288433+0000","last_became_peered":"2024-02-26T10:41:59.288433+0000","last_unstale":"2024-02-26T10:42:00.289123+0000","last_undegraded":"2024-02-26T10:42:00.289123+0000","last_fullsized":"2024-02-26T10:42:00.289123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:21.766000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289299+0000","last_change":"2024-02-26T10:41:59.293103+0000","last_active":"2024-02-26T10:42:00.289299+0000","last_peered":"2024-02-26T10:42:00.289299+0000","last_clean":"2024-02-26T10:42:00.289299+0000","last_became_active":"2024-02-26T10:41:59.292473+0000","last_became_peered":"2024-02-26T10:41:59.292473+0000","last_unstale":"2024-02-26T10:42:00.289299+0000","last_undegraded":"2024-02-26T10:42:00.289299+0000","last_fullsized":"2024-02-26T10:42:00.289299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:37.822421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078673+0000","last_change":"2024-02-26T10:41:59.294484+0000","last_active":"2024-02-26T10:42:01.078673+0000","last_peered":"2024-02-26T10:42:01.078673+0000","last_clean":"2024-02-26T10:42:01.078673+0000","last_became_active":"2024-02-26T10:41:59.293628+0000","last_became_peered":"2024-02-26T10:41:59.293628+0000","last_unstale":"2024-02-26T10:42:01.078673+0000","last_undegraded":"2024-02-26T10:42:01.078673+0000","last_fullsized":"2024-02-26T10:42:01.078673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:04.694202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145442+0000","last_change":"2024-02-26T10:41:59.289588+0000","last_active":"2024-02-26T10:42:01.145442+0000","last_peered":"2024-02-26T10:42:01.145442+0000","last_clean":"2024-02-26T10:42:01.145442+0000","last_became_active":"2024-02-26T10:41:59.289223+0000","last_became_peered":"2024-02-26T10:41:59.289223+0000","last_unstale":"2024-02-26T10:42:01.145442+0000","last_undegraded":"2024-02-26T10:42:01.145442+0000","last_fullsized":"2024-02-26T10:42:01.145442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079143+0000","last_change":"2024-02-26T10:41:59.296979+0000","last_active":"2024-02-26T10:42:01.079143+0000","last_peered":"2024-02-26T10:42:01.079143+0000","last_clean":"2024-02-26T10:42:01.079143+0000","last_became_active":"2024-02-26T10:41:59.295612+0000","last_became_peered":"2024-02-26T10:41:59.295612+0000","last_unstale":"2024-02-26T10:42:01.079143+0000","last_undegraded":"2024-02-26T10:42:01.079143+0000","last_fullsized":"2024-02-26T10:42:01.079143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:53.564566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147187+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:42:01.147187+0000","last_peered":"2024-02-26T10:42:01.147187+0000","last_clean":"2024-02-26T10:42:01.147187+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:42:01.147187+0000","last_undegraded":"2024-02-26T10:42:01.147187+0000","last_fullsized":"2024-02-26T10:42:01.147187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:03.513627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146982+0000","last_change":"2024-02-26T10:41:59.304661+0000","last_active":"2024-02-26T10:42:01.146982+0000","last_peered":"2024-02-26T10:42:01.146982+0000","last_clean":"2024-02-26T10:42:01.146982+0000","last_became_active":"2024-02-26T10:41:59.304358+0000","last_became_peered":"2024-02-26T10:41:59.304358+0000","last_unstale":"2024-02-26T10:42:01.146982+0000","last_undegraded":"2024-02-26T10:42:01.146982+0000","last_fullsized":"2024-02-26T10:42:01.146982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079625+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:42:01.079625+0000","last_peered":"2024-02-26T10:42:01.079625+0000","last_clean":"2024-02-26T10:42:01.079625+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:42:01.079625+0000","last_undegraded":"2024-02-26T10:42:01.079625+0000","last_fullsized":"2024-02-26T10:42:01.079625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:07.218989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147061+0000","last_change":"2024-02-26T10:41:59.310348+0000","last_active":"2024-02-26T10:42:01.147061+0000","last_peered":"2024-02-26T10:42:01.147061+0000","last_clean":"2024-02-26T10:42:01.147061+0000","last_became_active":"2024-02-26T10:41:59.309976+0000","last_became_peered":"2024-02-26T10:41:59.309976+0000","last_unstale":"2024-02-26T10:42:01.147061+0000","last_undegraded":"2024-02-26T10:42:01.147061+0000","last_fullsized":"2024-02-26T10:42:01.147061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283912+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:42:00.283912+0000","last_peered":"2024-02-26T10:42:00.283912+0000","last_clean":"2024-02-26T10:42:00.283912+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:42:00.283912+0000","last_undegraded":"2024-02-26T10:42:00.283912+0000","last_fullsized":"2024-02-26T10:42:00.283912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:15.932239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284613+0000","last_change":"2024-02-26T10:41:59.287952+0000","last_active":"2024-02-26T10:42:00.284613+0000","last_peered":"2024-02-26T10:42:00.284613+0000","last_clean":"2024-02-26T10:42:00.284613+0000","last_became_active":"2024-02-26T10:41:59.287395+0000","last_became_peered":"2024-02-26T10:41:59.287395+0000","last_unstale":"2024-02-26T10:42:00.284613+0000","last_undegraded":"2024-02-26T10:42:00.284613+0000","last_fullsized":"2024-02-26T10:42:00.284613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:54.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283312+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:42:00.283312+0000","last_peered":"2024-02-26T10:42:00.283312+0000","last_clean":"2024-02-26T10:42:00.283312+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:42:00.283312+0000","last_undegraded":"2024-02-26T10:42:00.283312+0000","last_fullsized":"2024-02-26T10:42:00.283312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289627+0000","last_change":"2024-02-26T10:41:59.292771+0000","last_active":"2024-02-26T10:42:00.289627+0000","last_peered":"2024-02-26T10:42:00.289627+0000","last_clean":"2024-02-26T10:42:00.289627+0000","last_became_active":"2024-02-26T10:41:59.288778+0000","last_became_peered":"2024-02-26T10:41:59.288778+0000","last_unstale":"2024-02-26T10:42:00.289627+0000","last_undegraded":"2024-02-26T10:42:00.289627+0000","last_fullsized":"2024-02-26T10:42:00.289627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:18.908425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079500+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:42:01.079500+0000","last_peered":"2024-02-26T10:42:01.079500+0000","last_clean":"2024-02-26T10:42:01.079500+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:42:01.079500+0000","last_undegraded":"2024-02-26T10:42:01.079500+0000","last_fullsized":"2024-02-26T10:42:01.079500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:20.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078896+0000","last_change":"2024-02-26T10:41:59.287430+0000","last_active":"2024-02-26T10:42:01.078896+0000","last_peered":"2024-02-26T10:42:01.078896+0000","last_clean":"2024-02-26T10:42:01.078896+0000","last_became_active":"2024-02-26T10:41:59.287010+0000","last_became_peered":"2024-02-26T10:41:59.287010+0000","last_unstale":"2024-02-26T10:42:01.078896+0000","last_undegraded":"2024-02-26T10:42:01.078896+0000","last_fullsized":"2024-02-26T10:42:01.078896+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:24.065359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078521+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:42:01.078521+0000","last_peered":"2024-02-26T10:42:01.078521+0000","last_clean":"2024-02-26T10:42:01.078521+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:42:01.078521+0000","last_undegraded":"2024-02-26T10:42:01.078521+0000","last_fullsized":"2024-02-26T10:42:01.078521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:19.393230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283719+0000","last_change":"2024-02-26T10:41:59.301966+0000","last_active":"2024-02-26T10:42:00.283719+0000","last_peered":"2024-02-26T10:42:00.283719+0000","last_clean":"2024-02-26T10:42:00.283719+0000","last_became_active":"2024-02-26T10:41:59.301282+0000","last_became_peered":"2024-02-26T10:41:59.301282+0000","last_unstale":"2024-02-26T10:42:00.283719+0000","last_undegraded":"2024-02-26T10:42:00.283719+0000","last_fullsized":"2024-02-26T10:42:00.283719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282755+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:42:00.282755+0000","last_peered":"2024-02-26T10:42:00.282755+0000","last_clean":"2024-02-26T10:42:00.282755+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:42:00.282755+0000","last_undegraded":"2024-02-26T10:42:00.282755+0000","last_fullsized":"2024-02-26T10:42:00.282755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:07.631703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281456+0000","last_change":"2024-02-26T10:41:59.289078+0000","last_active":"2024-02-26T10:42:00.281456+0000","last_peered":"2024-02-26T10:42:00.281456+0000","last_clean":"2024-02-26T10:42:00.281456+0000","last_became_active":"2024-02-26T10:41:59.288529+0000","last_became_peered":"2024-02-26T10:41:59.288529+0000","last_unstale":"2024-02-26T10:42:00.281456+0000","last_undegraded":"2024-02-26T10:42:00.281456+0000","last_fullsized":"2024-02-26T10:42:00.281456+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:28.358063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288136+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:42:00.288136+0000","last_peered":"2024-02-26T10:42:00.288136+0000","last_clean":"2024-02-26T10:42:00.288136+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:42:00.288136+0000","last_undegraded":"2024-02-26T10:42:00.288136+0000","last_fullsized":"2024-02-26T10:42:00.288136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:56.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282489+0000","last_change":"2024-02-26T10:41:59.294420+0000","last_active":"2024-02-26T10:42:00.282489+0000","last_peered":"2024-02-26T10:42:00.282489+0000","last_clean":"2024-02-26T10:42:00.282489+0000","last_became_active":"2024-02-26T10:41:59.294147+0000","last_became_peered":"2024-02-26T10:41:59.294147+0000","last_unstale":"2024-02-26T10:42:00.282489+0000","last_undegraded":"2024-02-26T10:42:00.282489+0000","last_fullsized":"2024-02-26T10:42:00.282489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:32.774644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147339+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:42:01.147339+0000","last_peered":"2024-02-26T10:42:01.147339+0000","last_clean":"2024-02-26T10:42:01.147339+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:42:01.147339+0000","last_undegraded":"2024-02-26T10:42:01.147339+0000","last_fullsized":"2024-02-26T10:42:01.147339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145312+0000","last_change":"2024-02-26T10:41:59.296775+0000","last_active":"2024-02-26T10:42:01.145312+0000","last_peered":"2024-02-26T10:42:01.145312+0000","last_clean":"2024-02-26T10:42:01.145312+0000","last_became_active":"2024-02-26T10:41:59.296431+0000","last_became_peered":"2024-02-26T10:41:59.296431+0000","last_unstale":"2024-02-26T10:42:01.145312+0000","last_undegraded":"2024-02-26T10:42:01.145312+0000","last_fullsized":"2024-02-26T10:42:01.145312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146925+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:42:01.146925+0000","last_peered":"2024-02-26T10:42:01.146925+0000","last_clean":"2024-02-26T10:42:01.146925+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:42:01.146925+0000","last_undegraded":"2024-02-26T10:42:01.146925+0000","last_fullsized":"2024-02-26T10:42:01.146925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:04.487859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283172+0000","last_change":"2024-02-26T10:41:59.300848+0000","last_active":"2024-02-26T10:42:00.283172+0000","last_peered":"2024-02-26T10:42:00.283172+0000","last_clean":"2024-02-26T10:42:00.283172+0000","last_became_active":"2024-02-26T10:41:59.300577+0000","last_became_peered":"2024-02-26T10:41:59.300577+0000","last_unstale":"2024-02-26T10:42:00.283172+0000","last_undegraded":"2024-02-26T10:42:00.283172+0000","last_fullsized":"2024-02-26T10:42:00.283172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146088+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:42:01.146088+0000","last_peered":"2024-02-26T10:42:01.146088+0000","last_clean":"2024-02-26T10:42:01.146088+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:42:01.146088+0000","last_undegraded":"2024-02-26T10:42:01.146088+0000","last_fullsized":"2024-02-26T10:42:01.146088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079430+0000","last_change":"2024-02-26T10:41:59.303988+0000","last_active":"2024-02-26T10:42:01.079430+0000","last_peered":"2024-02-26T10:42:01.079430+0000","last_clean":"2024-02-26T10:42:01.079430+0000","last_became_active":"2024-02-26T10:41:59.303677+0000","last_became_peered":"2024-02-26T10:41:59.303677+0000","last_unstale":"2024-02-26T10:42:01.079430+0000","last_undegraded":"2024-02-26T10:42:01.079430+0000","last_fullsized":"2024-02-26T10:42:01.079430+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:08.404204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145212+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:42:01.145212+0000","last_peered":"2024-02-26T10:42:01.145212+0000","last_clean":"2024-02-26T10:42:01.145212+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:42:01.145212+0000","last_undegraded":"2024-02-26T10:42:01.145212+0000","last_fullsized":"2024-02-26T10:42:01.145212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283321+0000","last_change":"2024-02-26T10:41:59.298661+0000","last_active":"2024-02-26T10:42:00.283321+0000","last_peered":"2024-02-26T10:42:00.283321+0000","last_clean":"2024-02-26T10:42:00.283321+0000","last_became_active":"2024-02-26T10:41:59.298384+0000","last_became_peered":"2024-02-26T10:41:59.298384+0000","last_unstale":"2024-02-26T10:42:00.283321+0000","last_undegraded":"2024-02-26T10:42:00.283321+0000","last_fullsized":"2024-02-26T10:42:00.283321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289264+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:42:00.289264+0000","last_peered":"2024-02-26T10:42:00.289264+0000","last_clean":"2024-02-26T10:42:00.289264+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:42:00.289264+0000","last_undegraded":"2024-02-26T10:42:00.289264+0000","last_fullsized":"2024-02-26T10:42:00.289264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:06.345056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284045+0000","last_change":"2024-02-26T10:41:59.303009+0000","last_active":"2024-02-26T10:42:00.284045+0000","last_peered":"2024-02-26T10:42:00.284045+0000","last_clean":"2024-02-26T10:42:00.284045+0000","last_became_active":"2024-02-26T10:41:59.301703+0000","last_became_peered":"2024-02-26T10:41:59.301703+0000","last_unstale":"2024-02-26T10:42:00.284045+0000","last_undegraded":"2024-02-26T10:42:00.284045+0000","last_fullsized":"2024-02-26T10:42:00.284045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281389+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:42:00.281389+0000","last_peered":"2024-02-26T10:42:00.281389+0000","last_clean":"2024-02-26T10:42:00.281389+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:42:00.281389+0000","last_undegraded":"2024-02-26T10:42:00.281389+0000","last_fullsized":"2024-02-26T10:42:00.281389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:15.119340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288109+0000","last_change":"2024-02-26T10:41:59.285657+0000","last_active":"2024-02-26T10:42:00.288109+0000","last_peered":"2024-02-26T10:42:00.288109+0000","last_clean":"2024-02-26T10:42:00.288109+0000","last_became_active":"2024-02-26T10:41:59.285365+0000","last_became_peered":"2024-02-26T10:41:59.285365+0000","last_unstale":"2024-02-26T10:42:00.288109+0000","last_undegraded":"2024-02-26T10:42:00.288109+0000","last_fullsized":"2024-02-26T10:42:00.288109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:10.689718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144425+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:42:01.144425+0000","last_peered":"2024-02-26T10:42:01.144425+0000","last_clean":"2024-02-26T10:42:01.144425+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:42:01.144425+0000","last_undegraded":"2024-02-26T10:42:01.144425+0000","last_fullsized":"2024-02-26T10:42:01.144425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288015+0000","last_change":"2024-02-26T10:41:59.299415+0000","last_active":"2024-02-26T10:42:00.288015+0000","last_peered":"2024-02-26T10:42:00.288015+0000","last_clean":"2024-02-26T10:42:00.288015+0000","last_became_active":"2024-02-26T10:41:59.298995+0000","last_became_peered":"2024-02-26T10:41:59.298995+0000","last_unstale":"2024-02-26T10:42:00.288015+0000","last_undegraded":"2024-02-26T10:42:00.288015+0000","last_fullsized":"2024-02-26T10:42:00.288015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:38.058564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282242+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:42:00.282242+0000","last_peered":"2024-02-26T10:42:00.282242+0000","last_clean":"2024-02-26T10:42:00.282242+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:42:00.282242+0000","last_undegraded":"2024-02-26T10:42:00.282242+0000","last_fullsized":"2024-02-26T10:42:00.282242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284598+0000","last_change":"2024-02-26T10:41:59.290032+0000","last_active":"2024-02-26T10:42:00.284598+0000","last_peered":"2024-02-26T10:42:00.284598+0000","last_clean":"2024-02-26T10:42:00.284598+0000","last_became_active":"2024-02-26T10:41:59.289604+0000","last_became_peered":"2024-02-26T10:41:59.289604+0000","last_unstale":"2024-02-26T10:42:00.284598+0000","last_undegraded":"2024-02-26T10:42:00.284598+0000","last_fullsized":"2024-02-26T10:42:00.284598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:23.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147525+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:42:01.147525+0000","last_peered":"2024-02-26T10:42:01.147525+0000","last_clean":"2024-02-26T10:42:01.147525+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:42:01.147525+0000","last_undegraded":"2024-02-26T10:42:01.147525+0000","last_fullsized":"2024-02-26T10:42:01.147525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147305+0000","last_change":"2024-02-26T10:41:59.299545+0000","last_active":"2024-02-26T10:42:01.147305+0000","last_peered":"2024-02-26T10:42:01.147305+0000","last_clean":"2024-02-26T10:42:01.147305+0000","last_became_active":"2024-02-26T10:41:59.298970+0000","last_became_peered":"2024-02-26T10:41:59.298970+0000","last_unstale":"2024-02-26T10:42:01.147305+0000","last_undegraded":"2024-02-26T10:42:01.147305+0000","last_fullsized":"2024-02-26T10:42:01.147305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283786+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:42:00.283786+0000","last_peered":"2024-02-26T10:42:00.283786+0000","last_clean":"2024-02-26T10:42:00.283786+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:42:00.283786+0000","last_undegraded":"2024-02-26T10:42:00.283786+0000","last_fullsized":"2024-02-26T10:42:00.283786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079654+0000","last_change":"2024-02-26T10:41:59.283472+0000","last_active":"2024-02-26T10:42:01.079654+0000","last_peered":"2024-02-26T10:42:01.079654+0000","last_clean":"2024-02-26T10:42:01.079654+0000","last_became_active":"2024-02-26T10:41:59.282978+0000","last_became_peered":"2024-02-26T10:41:59.282978+0000","last_unstale":"2024-02-26T10:42:01.079654+0000","last_undegraded":"2024-02-26T10:42:01.079654+0000","last_fullsized":"2024-02-26T10:42:01.079654+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:50.202422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283462+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:42:00.283462+0000","last_peered":"2024-02-26T10:42:00.283462+0000","last_clean":"2024-02-26T10:42:00.283462+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:42:00.283462+0000","last_undegraded":"2024-02-26T10:42:00.283462+0000","last_fullsized":"2024-02-26T10:42:00.283462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113746+0000","last_change":"2024-02-26T10:41:59.297414+0000","last_active":"2024-02-26T10:42:01.113746+0000","last_peered":"2024-02-26T10:42:01.113746+0000","last_clean":"2024-02-26T10:42:01.113746+0000","last_became_active":"2024-02-26T10:41:59.296486+0000","last_became_peered":"2024-02-26T10:41:59.296486+0000","last_unstale":"2024-02-26T10:42:01.113746+0000","last_undegraded":"2024-02-26T10:42:01.113746+0000","last_fullsized":"2024-02-26T10:42:01.113746+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:48.071684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281498+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:42:00.281498+0000","last_peered":"2024-02-26T10:42:00.281498+0000","last_clean":"2024-02-26T10:42:00.281498+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:42:00.281498+0000","last_undegraded":"2024-02-26T10:42:00.281498+0000","last_fullsized":"2024-02-26T10:42:00.281498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:46.825934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144660+0000","last_change":"2024-02-26T10:41:59.292484+0000","last_active":"2024-02-26T10:42:01.144660+0000","last_peered":"2024-02-26T10:42:01.144660+0000","last_clean":"2024-02-26T10:42:01.144660+0000","last_became_active":"2024-02-26T10:41:59.292140+0000","last_became_peered":"2024-02-26T10:41:59.292140+0000","last_unstale":"2024-02-26T10:42:01.144660+0000","last_undegraded":"2024-02-26T10:42:01.144660+0000","last_fullsized":"2024-02-26T10:42:01.144660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113819+0000","last_change":"2024-02-26T10:41:58.269128+0000","last_active":"2024-02-26T10:42:01.113819+0000","last_peered":"2024-02-26T10:42:01.113819+0000","last_clean":"2024-02-26T10:42:01.113819+0000","last_became_active":"2024-02-26T10:41:58.268746+0000","last_became_peered":"2024-02-26T10:41:58.268746+0000","last_unstale":"2024-02-26T10:42:01.113819+0000","last_undegraded":"2024-02-26T10:42:01.113819+0000","last_fullsized":"2024-02-26T10:42:01.113819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:46:53.257949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113664+0000","last_change":"2024-02-26T10:41:59.292780+0000","last_active":"2024-02-26T10:42:01.113664+0000","last_peered":"2024-02-26T10:42:01.113664+0000","last_clean":"2024-02-26T10:42:01.113664+0000","last_became_active":"2024-02-26T10:41:59.292526+0000","last_became_peered":"2024-02-26T10:41:59.292526+0000","last_unstale":"2024-02-26T10:42:01.113664+0000","last_undegraded":"2024-02-26T10:42:01.113664+0000","last_fullsized":"2024-02-26T10:42:01.113664+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:12.549758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113389+0000","last_change":"2024-02-26T10:41:58.269538+0000","last_active":"2024-02-26T10:42:01.113389+0000","last_peered":"2024-02-26T10:42:01.113389+0000","last_clean":"2024-02-26T10:42:01.113389+0000","last_became_active":"2024-02-26T10:41:58.269200+0000","last_became_peered":"2024-02-26T10:41:58.269200+0000","last_unstale":"2024-02-26T10:42:01.113389+0000","last_undegraded":"2024-02-26T10:42:01.113389+0000","last_fullsized":"2024-02-26T10:42:01.113389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:27.687984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144458+0000","last_change":"2024-02-26T10:41:59.305037+0000","last_active":"2024-02-26T10:42:01.144458+0000","last_peered":"2024-02-26T10:42:01.144458+0000","last_clean":"2024-02-26T10:42:01.144458+0000","last_became_active":"2024-02-26T10:41:59.304739+0000","last_became_peered":"2024-02-26T10:41:59.304739+0000","last_unstale":"2024-02-26T10:42:01.144458+0000","last_undegraded":"2024-02-26T10:42:01.144458+0000","last_fullsized":"2024-02-26T10:42:01.144458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.280914+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:42:00.280914+0000","last_peered":"2024-02-26T10:42:00.280914+0000","last_clean":"2024-02-26T10:42:00.280914+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:42:00.280914+0000","last_undegraded":"2024-02-26T10:42:00.280914+0000","last_fullsized":"2024-02-26T10:42:00.280914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:26.696259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079328+0000","last_change":"2024-02-26T10:41:59.289483+0000","last_active":"2024-02-26T10:42:01.079328+0000","last_peered":"2024-02-26T10:42:01.079328+0000","last_clean":"2024-02-26T10:42:01.079328+0000","last_became_active":"2024-02-26T10:41:59.288870+0000","last_became_peered":"2024-02-26T10:41:59.288870+0000","last_unstale":"2024-02-26T10:42:01.079328+0000","last_undegraded":"2024-02-26T10:42:01.079328+0000","last_fullsized":"2024-02-26T10:42:01.079328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:02.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290450+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:42:00.290450+0000","last_peered":"2024-02-26T10:42:00.290450+0000","last_clean":"2024-02-26T10:42:00.290450+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:42:00.290450+0000","last_undegraded":"2024-02-26T10:42:00.290450+0000","last_fullsized":"2024-02-26T10:42:00.290450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284373+0000","last_change":"2024-02-26T10:41:59.294199+0000","last_active":"2024-02-26T10:42:00.284373+0000","last_peered":"2024-02-26T10:42:00.284373+0000","last_clean":"2024-02-26T10:42:00.284373+0000","last_became_active":"2024-02-26T10:41:59.293906+0000","last_became_peered":"2024-02-26T10:41:59.293906+0000","last_unstale":"2024-02-26T10:42:00.284373+0000","last_undegraded":"2024-02-26T10:42:00.284373+0000","last_fullsized":"2024-02-26T10:42:00.284373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:27.711178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290377+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:42:00.290377+0000","last_peered":"2024-02-26T10:42:00.290377+0000","last_clean":"2024-02-26T10:42:00.290377+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:42:00.290377+0000","last_undegraded":"2024-02-26T10:42:00.290377+0000","last_fullsized":"2024-02-26T10:42:00.290377+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:02:34.734605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146776+0000","last_change":"2024-02-26T10:41:59.298323+0000","last_active":"2024-02-26T10:42:01.146776+0000","last_peered":"2024-02-26T10:42:01.146776+0000","last_clean":"2024-02-26T10:42:01.146776+0000","last_became_active":"2024-02-26T10:41:59.295932+0000","last_became_peered":"2024-02-26T10:41:59.295932+0000","last_unstale":"2024-02-26T10:42:01.146776+0000","last_undegraded":"2024-02-26T10:42:01.146776+0000","last_fullsized":"2024-02-26T10:42:01.146776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146596+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:42:01.146596+0000","last_peered":"2024-02-26T10:42:01.146596+0000","last_clean":"2024-02-26T10:42:01.146596+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:42:01.146596+0000","last_undegraded":"2024-02-26T10:42:01.146596+0000","last_fullsized":"2024-02-26T10:42:01.146596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079836+0000","last_change":"2024-02-26T10:41:59.293572+0000","last_active":"2024-02-26T10:42:01.079836+0000","last_peered":"2024-02-26T10:42:01.079836+0000","last_clean":"2024-02-26T10:42:01.079836+0000","last_became_active":"2024-02-26T10:41:59.292292+0000","last_became_peered":"2024-02-26T10:41:59.292292+0000","last_unstale":"2024-02-26T10:42:01.079836+0000","last_undegraded":"2024-02-26T10:42:01.079836+0000","last_fullsized":"2024-02-26T10:42:01.079836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:26.922522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147038+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:42:01.147038+0000","last_peered":"2024-02-26T10:42:01.147038+0000","last_clean":"2024-02-26T10:42:01.147038+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:42:01.147038+0000","last_undegraded":"2024-02-26T10:42:01.147038+0000","last_fullsized":"2024-02-26T10:42:01.147038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:14.749617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144982+0000","last_change":"2024-02-26T10:41:59.296354+0000","last_active":"2024-02-26T10:42:01.144982+0000","last_peered":"2024-02-26T10:42:01.144982+0000","last_clean":"2024-02-26T10:42:01.144982+0000","last_became_active":"2024-02-26T10:41:59.296012+0000","last_became_peered":"2024-02-26T10:41:59.296012+0000","last_unstale":"2024-02-26T10:42:01.144982+0000","last_undegraded":"2024-02-26T10:42:01.144982+0000","last_fullsized":"2024-02-26T10:42:01.144982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284034+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:42:00.284034+0000","last_peered":"2024-02-26T10:42:00.284034+0000","last_clean":"2024-02-26T10:42:00.284034+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:42:00.284034+0000","last_undegraded":"2024-02-26T10:42:00.284034+0000","last_fullsized":"2024-02-26T10:42:00.284034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:50:59.443811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284365+0000","last_change":"2024-02-26T10:41:59.297360+0000","last_active":"2024-02-26T10:42:00.284365+0000","last_peered":"2024-02-26T10:42:00.284365+0000","last_clean":"2024-02-26T10:42:00.284365+0000","last_became_active":"2024-02-26T10:41:59.297031+0000","last_became_peered":"2024-02-26T10:41:59.297031+0000","last_unstale":"2024-02-26T10:42:00.284365+0000","last_undegraded":"2024-02-26T10:42:00.284365+0000","last_fullsized":"2024-02-26T10:42:00.284365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078834+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:42:01.078834+0000","last_peered":"2024-02-26T10:42:01.078834+0000","last_clean":"2024-02-26T10:42:01.078834+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:42:01.078834+0000","last_undegraded":"2024-02-26T10:42:01.078834+0000","last_fullsized":"2024-02-26T10:42:01.078834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:58:22.045852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283703+0000","last_change":"2024-02-26T10:41:59.295832+0000","last_active":"2024-02-26T10:42:00.283703+0000","last_peered":"2024-02-26T10:42:00.283703+0000","last_clean":"2024-02-26T10:42:00.283703+0000","last_became_active":"2024-02-26T10:41:59.295548+0000","last_became_peered":"2024-02-26T10:41:59.295548+0000","last_unstale":"2024-02-26T10:42:00.283703+0000","last_undegraded":"2024-02-26T10:42:00.283703+0000","last_fullsized":"2024-02-26T10:42:00.283703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146232+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:42:01.146232+0000","last_peered":"2024-02-26T10:42:01.146232+0000","last_clean":"2024-02-26T10:42:01.146232+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:42:01.146232+0000","last_undegraded":"2024-02-26T10:42:01.146232+0000","last_fullsized":"2024-02-26T10:42:01.146232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145956+0000","last_change":"2024-02-26T10:41:59.300681+0000","last_active":"2024-02-26T10:42:01.145956+0000","last_peered":"2024-02-26T10:42:01.145956+0000","last_clean":"2024-02-26T10:42:01.145956+0000","last_became_active":"2024-02-26T10:41:59.299849+0000","last_became_peered":"2024-02-26T10:41:59.299849+0000","last_unstale":"2024-02-26T10:42:01.145956+0000","last_undegraded":"2024-02-26T10:42:01.145956+0000","last_fullsized":"2024-02-26T10:42:01.145956+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078965+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:42:01.078965+0000","last_peered":"2024-02-26T10:42:01.078965+0000","last_clean":"2024-02-26T10:42:01.078965+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:42:01.078965+0000","last_undegraded":"2024-02-26T10:42:01.078965+0000","last_fullsized":"2024-02-26T10:42:01.078965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:40:26.347537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290123+0000","last_change":"2024-02-26T10:41:59.295485+0000","last_active":"2024-02-26T10:42:00.290123+0000","last_peered":"2024-02-26T10:42:00.290123+0000","last_clean":"2024-02-26T10:42:00.290123+0000","last_became_active":"2024-02-26T10:41:59.295154+0000","last_became_peered":"2024-02-26T10:41:59.295154+0000","last_unstale":"2024-02-26T10:42:00.290123+0000","last_undegraded":"2024-02-26T10:42:00.290123+0000","last_fullsized":"2024-02-26T10:42:00.290123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:15.764536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284516+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:42:00.284516+0000","last_peered":"2024-02-26T10:42:00.284516+0000","last_clean":"2024-02-26T10:42:00.284516+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:42:00.284516+0000","last_undegraded":"2024-02-26T10:42:00.284516+0000","last_fullsized":"2024-02-26T10:42:00.284516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284158+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:42:00.284158+0000","last_peered":"2024-02-26T10:42:00.284158+0000","last_clean":"2024-02-26T10:42:00.284158+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:42:00.284158+0000","last_undegraded":"2024-02-26T10:42:00.284158+0000","last_fullsized":"2024-02-26T10:42:00.284158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1004431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":414687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":415627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":413993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":415605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":414575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":414187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1004395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:08.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:09.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.171+0000 7fa90fe00640 1 Processor -- start 2024-02-26T10:42:09.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.171+0000 7fa90fe00640 1 -- start start 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9100a94a0 0x7fa91006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 0x7fa9100abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91006df90 con 0x7fa91006ac20 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9100a9a80 con 0x7fa9100a94a0 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100a9d60 con 0x7fa91006db00 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa91006ac20 0x7fa910153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56146/0 (socket says 172.21.15.67:56146) 2024-02-26T10:42:09.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.173+0000 7fa90ea00640 1 -- 172.21.15.67:0/4045804316 learned_addr learned my addr 172.21.15.67:0/4045804316 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90f400640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 0x7fa9100abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa907e00640 1 --2- 172.21.15.67:0/4045804316 >> v2:172.21.15.82:3300/0 conn(0x7fa9100a94a0 0x7fa91006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366451983 0 0) 0x7fa91006df90 con 0x7fa91006ac20 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f4003520 con 0x7fa91006ac20 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981728893 0 0) 0x7fa8f4003520 con 0x7fa91006ac20 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 msgr2=0x7fa9100abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 0x7fa9100abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 >> v2:172.21.15.82:3300/0 conn(0x7fa9100a94a0 msgr2=0x7fa91006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 --2- 172.21.15.67:0/4045804316 >> v2:172.21.15.82:3300/0 conn(0x7fa9100a94a0 0x7fa91006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100ab750 con 0x7fa91006ac20 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fa900001670 con 0x7fa91006ac20 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 >> v1:172.21.15.67:6789/0 conn(0x7fa91006ac20 legacy=0x7fa910153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40039b0 0x7fa8f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa8f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.174+0000 7fa90e000640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 0x7fa8f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100a9a80 con 0x7fa8f4004800 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100a9d60 con 0x7fa8f40039b0 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f400b9c0 con 0x7fa91006db00 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa907e00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa8f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa90f400640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 0x7fa8f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa90ea00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40039b0 0x7fa8f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa907e00640 1 -- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 msgr2=0x7fa8f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa907e00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa91006db00 0x7fa8f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa907e00640 1 -- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40039b0 msgr2=0x7fa8f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa907e00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40039b0 0x7fa8f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa907e00640 1 -- 172.21.15.67:0/4045804316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f4003520 con 0x7fa8f4004800 2024-02-26T10:42:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.175+0000 7fa90ea00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40039b0 0x7fa8f4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.177+0000 7fa907e00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa8f4004c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800ba30 tx=0x7fa8f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda279b0754054d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.177+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8036020 con 0x7fa8f4004800 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.177+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f8032680 con 0x7fa8f4004800 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.177+0000 7fa90e000640 1 -- 172.21.15.67:0/4045804316 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800b600 con 0x7fa8f4004800 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90fe00640 1 -- 172.21.15.67:0/4045804316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 msgr2=0x7fa8f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90fe00640 1 --2- 172.21.15.67:0/4045804316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa8f4004c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800ba30 tx=0x7fa8f4005830 comp rx=0 tx=0).stop 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90f400640 1 -- 172.21.15.67:0/4045804316 reap_dead start 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90fe00640 1 -- 172.21.15.67:0/4045804316 shutdown_connections 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90fe00640 1 -- 172.21.15.67:0/4045804316 >> 172.21.15.67:0/4045804316 conn(0x7fa9100c63f0 msgr2=0x7fa9100c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90fe00640 1 -- 172.21.15.67:0/4045804316 shutdown_connections 2024-02-26T10:42:09.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.178+0000 7fa90fe00640 1 -- 172.21.15.67:0/4045804316 wait complete. 2024-02-26T10:42:09.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.180+0000 7fa90fe00640 1 Processor -- start 2024-02-26T10:42:09.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.180+0000 7fa90fe00640 1 -- start start 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa910147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa910153550 0x7fa9101ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9101ee350 0x7fa9101e1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa910150590 con 0x7fa8f4004800 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa910150410 con 0x7fa9101ee350 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100abf50 con 0x7fa910153550 2024-02-26T10:42:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa910147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa910147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43504/0 (socket says 172.21.15.67:43504) 2024-02-26T10:42:09.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90ea00640 1 -- 172.21.15.67:0/896740812 learned_addr learned my addr 172.21.15.67:0/896740812 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:09.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa907e00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa910153550 0x7fa9101ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90f400640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9101ee350 0x7fa9101e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90ea00640 1 -- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa910153550 msgr2=0x7fa9101ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90ea00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa910153550 0x7fa9101ede10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.181+0000 7fa90ea00640 1 -- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9101ee350 msgr2=0x7fa9101e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa90ea00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9101ee350 0x7fa9101e1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa90ea00640 1 -- 172.21.15.67:0/896740812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100c5c60 con 0x7fa8f4004800 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa90ea00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa910147e60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa90000d7e0 tx=0x7fa90000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900019070 con 0x7fa8f4004800 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9000074e0 con 0x7fa8f4004800 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000049e0 con 0x7fa8f4004800 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa910156f60 con 0x7fa8f4004800 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.182+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa910073020 con 0x7fa8f4004800 2024-02-26T10:42:09.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.183+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa910074a30 con 0x7fa8f4004800 2024-02-26T10:42:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.184+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa90001c070 con 0x7fa8f4004800 2024-02-26T10:42:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.185+0000 7fa906a00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.185+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa900089d30 con 0x7fa8f4004800 2024-02-26T10:42:09.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.186+0000 7fa907e00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.187+0000 7fa907e00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800ba00 tx=0x7fa8f8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:09.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.191+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa900058b10 con 0x7fa8f4004800 2024-02-26T10:42:09.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.385+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa910073260 con 0x7fa8ec06b4d0 2024-02-26T10:42:09.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.403+0000 7fa906a00640 1 -- 172.21.15.67:0/896740812 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251962 (secure 0 0 0) 0x7fa910073260 con 0x7fa8ec06b4d0 2024-02-26T10:42:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:09.405 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8ec06b4d0 msgr2=0x7fa8ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800ba00 tx=0x7fa8f8030000 comp rx=0 tx=0).stop 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 msgr2=0x7fa910147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa910147e60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa90000d7e0 tx=0x7fa90000dca0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 shutdown_connections 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9101ee350 0x7fa9101e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa910153550 0x7fa9101ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 --2- 172.21.15.67:0/896740812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8f4004800 0x7fa910147e60 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 >> 172.21.15.67:0/896740812 conn(0x7fa9100c63f0 msgr2=0x7fa9100c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:09.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 shutdown_connections 2024-02-26T10:42:09.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.408+0000 7fa90fe00640 1 -- 172.21.15.67:0/896740812 wait complete. 2024-02-26T10:42:09.425 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-02-26T10:42:08.405628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249636,"kb_used_data":8488,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725084,"statfs":{"total":773094113280,"available":772838486016,"internally_reserved":0,"allocated":8691712,"data_stored":4497500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.148625"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146868+0000","last_change":"2024-02-26T10:41:59.308476+0000","last_active":"2024-02-26T10:42:01.146868+0000","last_peered":"2024-02-26T10:42:01.146868+0000","last_clean":"2024-02-26T10:42:01.146868+0000","last_became_active":"2024-02-26T10:41:59.307995+0000","last_became_peered":"2024-02-26T10:41:59.307995+0000","last_unstale":"2024-02-26T10:42:01.146868+0000","last_undegraded":"2024-02-26T10:42:01.146868+0000","last_fullsized":"2024-02-26T10:42:01.146868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290470+0000","last_change":"2024-02-26T10:41:59.300851+0000","last_active":"2024-02-26T10:42:00.290470+0000","last_peered":"2024-02-26T10:42:00.290470+0000","last_clean":"2024-02-26T10:42:00.290470+0000","last_became_active":"2024-02-26T10:41:59.300574+0000","last_became_peered":"2024-02-26T10:41:59.300574+0000","last_unstale":"2024-02-26T10:42:00.290470+0000","last_undegraded":"2024-02-26T10:42:00.290470+0000","last_fullsized":"2024-02-26T10:42:00.290470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:04.528547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284215+0000","last_change":"2024-02-26T10:41:59.293058+0000","last_active":"2024-02-26T10:42:00.284215+0000","last_peered":"2024-02-26T10:42:00.284215+0000","last_clean":"2024-02-26T10:42:00.284215+0000","last_became_active":"2024-02-26T10:41:59.292789+0000","last_became_peered":"2024-02-26T10:41:59.292789+0000","last_unstale":"2024-02-26T10:42:00.284215+0000","last_undegraded":"2024-02-26T10:42:00.284215+0000","last_fullsized":"2024-02-26T10:42:00.284215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114472+0000","last_change":"2024-02-26T10:41:59.296577+0000","last_active":"2024-02-26T10:42:01.114472+0000","last_peered":"2024-02-26T10:42:01.114472+0000","last_clean":"2024-02-26T10:42:01.114472+0000","last_became_active":"2024-02-26T10:41:59.296264+0000","last_became_peered":"2024-02-26T10:41:59.296264+0000","last_unstale":"2024-02-26T10:42:01.114472+0000","last_undegraded":"2024-02-26T10:42:01.114472+0000","last_fullsized":"2024-02-26T10:42:01.114472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:13.257965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079286+0000","last_change":"2024-02-26T10:41:59.298038+0000","last_active":"2024-02-26T10:42:01.079286+0000","last_peered":"2024-02-26T10:42:01.079286+0000","last_clean":"2024-02-26T10:42:01.079286+0000","last_became_active":"2024-02-26T10:41:59.297699+0000","last_became_peered":"2024-02-26T10:41:59.297699+0000","last_unstale":"2024-02-26T10:42:01.079286+0000","last_undegraded":"2024-02-26T10:42:01.079286+0000","last_fullsized":"2024-02-26T10:42:01.079286+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:30.879707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145586+0000","last_change":"2024-02-26T10:41:59.295626+0000","last_active":"2024-02-26T10:42:01.145586+0000","last_peered":"2024-02-26T10:42:01.145586+0000","last_clean":"2024-02-26T10:42:01.145586+0000","last_became_active":"2024-02-26T10:41:59.295266+0000","last_became_peered":"2024-02-26T10:41:59.295266+0000","last_unstale":"2024-02-26T10:42:01.145586+0000","last_undegraded":"2024-02-26T10:42:01.145586+0000","last_fullsized":"2024-02-26T10:42:01.145586+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113506+0000","last_change":"2024-02-26T10:41:59.287398+0000","last_active":"2024-02-26T10:42:01.113506+0000","last_peered":"2024-02-26T10:42:01.113506+0000","last_clean":"2024-02-26T10:42:01.113506+0000","last_became_active":"2024-02-26T10:41:59.286305+0000","last_became_peered":"2024-02-26T10:41:59.286305+0000","last_unstale":"2024-02-26T10:42:01.113506+0000","last_undegraded":"2024-02-26T10:42:01.113506+0000","last_fullsized":"2024-02-26T10:42:01.113506+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:54.865254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144514+0000","last_change":"2024-02-26T10:41:59.307667+0000","last_active":"2024-02-26T10:42:01.144514+0000","last_peered":"2024-02-26T10:42:01.144514+0000","last_clean":"2024-02-26T10:42:01.144514+0000","last_became_active":"2024-02-26T10:41:59.307319+0000","last_became_peered":"2024-02-26T10:41:59.307319+0000","last_unstale":"2024-02-26T10:42:01.144514+0000","last_undegraded":"2024-02-26T10:42:01.144514+0000","last_fullsized":"2024-02-26T10:42:01.144514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144909+0000","last_change":"2024-02-26T10:41:59.304927+0000","last_active":"2024-02-26T10:42:01.144909+0000","last_peered":"2024-02-26T10:42:01.144909+0000","last_clean":"2024-02-26T10:42:01.144909+0000","last_became_active":"2024-02-26T10:41:59.304641+0000","last_became_peered":"2024-02-26T10:41:59.304641+0000","last_unstale":"2024-02-26T10:42:01.144909+0000","last_undegraded":"2024-02-26T10:42:01.144909+0000","last_fullsized":"2024-02-26T10:42:01.144909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146925+0000","last_change":"2024-02-26T10:41:59.293335+0000","last_active":"2024-02-26T10:42:01.146925+0000","last_peered":"2024-02-26T10:42:01.146925+0000","last_clean":"2024-02-26T10:42:01.146925+0000","last_became_active":"2024-02-26T10:41:59.292937+0000","last_became_peered":"2024-02-26T10:41:59.292937+0000","last_unstale":"2024-02-26T10:42:01.146925+0000","last_undegraded":"2024-02-26T10:42:01.146925+0000","last_fullsized":"2024-02-26T10:42:01.146925+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289248+0000","last_change":"2024-02-26T10:41:59.282216+0000","last_active":"2024-02-26T10:42:00.289248+0000","last_peered":"2024-02-26T10:42:00.289248+0000","last_clean":"2024-02-26T10:42:00.289248+0000","last_became_active":"2024-02-26T10:41:59.281891+0000","last_became_peered":"2024-02-26T10:41:59.281891+0000","last_unstale":"2024-02-26T10:42:00.289248+0000","last_undegraded":"2024-02-26T10:42:00.289248+0000","last_fullsized":"2024-02-26T10:42:00.289248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:07.272911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146079+0000","last_change":"2024-02-26T10:41:59.290942+0000","last_active":"2024-02-26T10:42:01.146079+0000","last_peered":"2024-02-26T10:42:01.146079+0000","last_clean":"2024-02-26T10:42:01.146079+0000","last_became_active":"2024-02-26T10:41:59.288616+0000","last_became_peered":"2024-02-26T10:41:59.288616+0000","last_unstale":"2024-02-26T10:42:01.146079+0000","last_undegraded":"2024-02-26T10:42:01.146079+0000","last_fullsized":"2024-02-26T10:42:01.146079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284058+0000","last_change":"2024-02-26T10:41:59.281911+0000","last_active":"2024-02-26T10:42:00.284058+0000","last_peered":"2024-02-26T10:42:00.284058+0000","last_clean":"2024-02-26T10:42:00.284058+0000","last_became_active":"2024-02-26T10:41:59.281270+0000","last_became_peered":"2024-02-26T10:41:59.281270+0000","last_unstale":"2024-02-26T10:42:00.284058+0000","last_undegraded":"2024-02-26T10:42:00.284058+0000","last_fullsized":"2024-02-26T10:42:00.284058+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113712+0000","last_change":"2024-02-26T10:41:59.310387+0000","last_active":"2024-02-26T10:42:01.113712+0000","last_peered":"2024-02-26T10:42:01.113712+0000","last_clean":"2024-02-26T10:42:01.113712+0000","last_became_active":"2024-02-26T10:41:59.310111+0000","last_became_peered":"2024-02-26T10:41:59.310111+0000","last_unstale":"2024-02-26T10:42:01.113712+0000","last_undegraded":"2024-02-26T10:42:01.113712+0000","last_fullsized":"2024-02-26T10:42:01.113712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:53:52.167978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281188+0000","last_change":"2024-02-26T10:41:59.289328+0000","last_active":"2024-02-26T10:42:00.281188+0000","last_peered":"2024-02-26T10:42:00.281188+0000","last_clean":"2024-02-26T10:42:00.281188+0000","last_became_active":"2024-02-26T10:41:59.288951+0000","last_became_peered":"2024-02-26T10:41:59.288951+0000","last_unstale":"2024-02-26T10:42:00.281188+0000","last_undegraded":"2024-02-26T10:42:00.281188+0000","last_fullsized":"2024-02-26T10:42:00.281188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:00.613487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078476+0000","last_change":"2024-02-26T10:41:59.290536+0000","last_active":"2024-02-26T10:42:01.078476+0000","last_peered":"2024-02-26T10:42:01.078476+0000","last_clean":"2024-02-26T10:42:01.078476+0000","last_became_active":"2024-02-26T10:41:59.289661+0000","last_became_peered":"2024-02-26T10:41:59.289661+0000","last_unstale":"2024-02-26T10:42:01.078476+0000","last_undegraded":"2024-02-26T10:42:01.078476+0000","last_fullsized":"2024-02-26T10:42:01.078476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:56.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079115+0000","last_change":"2024-02-26T10:41:59.291203+0000","last_active":"2024-02-26T10:42:01.079115+0000","last_peered":"2024-02-26T10:42:01.079115+0000","last_clean":"2024-02-26T10:42:01.079115+0000","last_became_active":"2024-02-26T10:41:59.290916+0000","last_became_peered":"2024-02-26T10:41:59.290916+0000","last_unstale":"2024-02-26T10:42:01.079115+0000","last_undegraded":"2024-02-26T10:42:01.079115+0000","last_fullsized":"2024-02-26T10:42:01.079115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:09.397334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114840+0000","last_change":"2024-02-26T10:41:59.291282+0000","last_active":"2024-02-26T10:42:01.114840+0000","last_peered":"2024-02-26T10:42:01.114840+0000","last_clean":"2024-02-26T10:42:01.114840+0000","last_became_active":"2024-02-26T10:41:59.291008+0000","last_became_peered":"2024-02-26T10:41:59.291008+0000","last_unstale":"2024-02-26T10:42:01.114840+0000","last_undegraded":"2024-02-26T10:42:01.114840+0000","last_fullsized":"2024-02-26T10:42:01.114840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:55:38.972893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283816+0000","last_change":"2024-02-26T10:41:59.289830+0000","last_active":"2024-02-26T10:42:00.283816+0000","last_peered":"2024-02-26T10:42:00.283816+0000","last_clean":"2024-02-26T10:42:00.283816+0000","last_became_active":"2024-02-26T10:41:59.289466+0000","last_became_peered":"2024-02-26T10:41:59.289466+0000","last_unstale":"2024-02-26T10:42:00.283816+0000","last_undegraded":"2024-02-26T10:42:00.283816+0000","last_fullsized":"2024-02-26T10:42:00.283816+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114572+0000","last_change":"2024-02-26T10:41:59.294466+0000","last_active":"2024-02-26T10:42:01.114572+0000","last_peered":"2024-02-26T10:42:01.114572+0000","last_clean":"2024-02-26T10:42:01.114572+0000","last_became_active":"2024-02-26T10:41:59.294199+0000","last_became_peered":"2024-02-26T10:41:59.294199+0000","last_unstale":"2024-02-26T10:42:01.114572+0000","last_undegraded":"2024-02-26T10:42:01.114572+0000","last_fullsized":"2024-02-26T10:42:01.114572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:06.679617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.114210+0000","last_change":"2024-02-26T10:41:59.302058+0000","last_active":"2024-02-26T10:42:01.114210+0000","last_peered":"2024-02-26T10:42:01.114210+0000","last_clean":"2024-02-26T10:42:01.114210+0000","last_became_active":"2024-02-26T10:41:59.301773+0000","last_became_peered":"2024-02-26T10:41:59.301773+0000","last_unstale":"2024-02-26T10:42:01.114210+0000","last_undegraded":"2024-02-26T10:42:01.114210+0000","last_fullsized":"2024-02-26T10:42:01.114210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:58.489330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289203+0000","last_change":"2024-02-26T10:41:59.306729+0000","last_active":"2024-02-26T10:42:00.289203+0000","last_peered":"2024-02-26T10:42:00.289203+0000","last_clean":"2024-02-26T10:42:00.289203+0000","last_became_active":"2024-02-26T10:41:59.306464+0000","last_became_peered":"2024-02-26T10:41:59.306464+0000","last_unstale":"2024-02-26T10:42:00.289203+0000","last_undegraded":"2024-02-26T10:42:00.289203+0000","last_fullsized":"2024-02-26T10:42:00.289203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:56.409926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145960+0000","last_change":"2024-02-26T10:41:59.304328+0000","last_active":"2024-02-26T10:42:01.145960+0000","last_peered":"2024-02-26T10:42:01.145960+0000","last_clean":"2024-02-26T10:42:01.145960+0000","last_became_active":"2024-02-26T10:41:59.304025+0000","last_became_peered":"2024-02-26T10:41:59.304025+0000","last_unstale":"2024-02-26T10:42:01.145960+0000","last_undegraded":"2024-02-26T10:42:01.145960+0000","last_fullsized":"2024-02-26T10:42:01.145960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113484+0000","last_change":"2024-02-26T10:41:59.303897+0000","last_active":"2024-02-26T10:42:01.113484+0000","last_peered":"2024-02-26T10:42:01.113484+0000","last_clean":"2024-02-26T10:42:01.113484+0000","last_became_active":"2024-02-26T10:41:59.301812+0000","last_became_peered":"2024-02-26T10:41:59.301812+0000","last_unstale":"2024-02-26T10:42:01.113484+0000","last_undegraded":"2024-02-26T10:42:01.113484+0000","last_fullsized":"2024-02-26T10:42:01.113484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:03:57.549048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290341+0000","last_change":"2024-02-26T10:41:59.297092+0000","last_active":"2024-02-26T10:42:00.290341+0000","last_peered":"2024-02-26T10:42:00.290341+0000","last_clean":"2024-02-26T10:42:00.290341+0000","last_became_active":"2024-02-26T10:41:59.295816+0000","last_became_peered":"2024-02-26T10:41:59.295816+0000","last_unstale":"2024-02-26T10:42:00.290341+0000","last_undegraded":"2024-02-26T10:42:00.290341+0000","last_fullsized":"2024-02-26T10:42:00.290341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:05.502778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284127+0000","last_change":"2024-02-26T10:41:59.307675+0000","last_active":"2024-02-26T10:42:00.284127+0000","last_peered":"2024-02-26T10:42:00.284127+0000","last_clean":"2024-02-26T10:42:00.284127+0000","last_became_active":"2024-02-26T10:41:59.307360+0000","last_became_peered":"2024-02-26T10:41:59.307360+0000","last_unstale":"2024-02-26T10:42:00.284127+0000","last_undegraded":"2024-02-26T10:42:00.284127+0000","last_fullsized":"2024-02-26T10:42:00.284127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079335+0000","last_change":"2024-02-26T10:41:59.287990+0000","last_active":"2024-02-26T10:42:01.079335+0000","last_peered":"2024-02-26T10:42:01.079335+0000","last_clean":"2024-02-26T10:42:01.079335+0000","last_became_active":"2024-02-26T10:41:59.286925+0000","last_became_peered":"2024-02-26T10:41:59.286925+0000","last_unstale":"2024-02-26T10:42:01.079335+0000","last_undegraded":"2024-02-26T10:42:01.079335+0000","last_fullsized":"2024-02-26T10:42:01.079335+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:08.646623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146292+0000","last_change":"2024-02-26T10:41:59.297459+0000","last_active":"2024-02-26T10:42:01.146292+0000","last_peered":"2024-02-26T10:42:01.146292+0000","last_clean":"2024-02-26T10:42:01.146292+0000","last_became_active":"2024-02-26T10:41:59.296494+0000","last_became_peered":"2024-02-26T10:41:59.296494+0000","last_unstale":"2024-02-26T10:42:01.146292+0000","last_undegraded":"2024-02-26T10:42:01.146292+0000","last_fullsized":"2024-02-26T10:42:01.146292+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289123+0000","last_change":"2024-02-26T10:41:59.288740+0000","last_active":"2024-02-26T10:42:00.289123+0000","last_peered":"2024-02-26T10:42:00.289123+0000","last_clean":"2024-02-26T10:42:00.289123+0000","last_became_active":"2024-02-26T10:41:59.288433+0000","last_became_peered":"2024-02-26T10:41:59.288433+0000","last_unstale":"2024-02-26T10:42:00.289123+0000","last_undegraded":"2024-02-26T10:42:00.289123+0000","last_fullsized":"2024-02-26T10:42:00.289123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:21.766000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289299+0000","last_change":"2024-02-26T10:41:59.293103+0000","last_active":"2024-02-26T10:42:00.289299+0000","last_peered":"2024-02-26T10:42:00.289299+0000","last_clean":"2024-02-26T10:42:00.289299+0000","last_became_active":"2024-02-26T10:41:59.292473+0000","last_became_peered":"2024-02-26T10:41:59.292473+0000","last_unstale":"2024-02-26T10:42:00.289299+0000","last_undegraded":"2024-02-26T10:42:00.289299+0000","last_fullsized":"2024-02-26T10:42:00.289299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:37.822421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078673+0000","last_change":"2024-02-26T10:41:59.294484+0000","last_active":"2024-02-26T10:42:01.078673+0000","last_peered":"2024-02-26T10:42:01.078673+0000","last_clean":"2024-02-26T10:42:01.078673+0000","last_became_active":"2024-02-26T10:41:59.293628+0000","last_became_peered":"2024-02-26T10:41:59.293628+0000","last_unstale":"2024-02-26T10:42:01.078673+0000","last_undegraded":"2024-02-26T10:42:01.078673+0000","last_fullsized":"2024-02-26T10:42:01.078673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:04.694202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145442+0000","last_change":"2024-02-26T10:41:59.289588+0000","last_active":"2024-02-26T10:42:01.145442+0000","last_peered":"2024-02-26T10:42:01.145442+0000","last_clean":"2024-02-26T10:42:01.145442+0000","last_became_active":"2024-02-26T10:41:59.289223+0000","last_became_peered":"2024-02-26T10:41:59.289223+0000","last_unstale":"2024-02-26T10:42:01.145442+0000","last_undegraded":"2024-02-26T10:42:01.145442+0000","last_fullsized":"2024-02-26T10:42:01.145442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079143+0000","last_change":"2024-02-26T10:41:59.296979+0000","last_active":"2024-02-26T10:42:01.079143+0000","last_peered":"2024-02-26T10:42:01.079143+0000","last_clean":"2024-02-26T10:42:01.079143+0000","last_became_active":"2024-02-26T10:41:59.295612+0000","last_became_peered":"2024-02-26T10:41:59.295612+0000","last_unstale":"2024-02-26T10:42:01.079143+0000","last_undegraded":"2024-02-26T10:42:01.079143+0000","last_fullsized":"2024-02-26T10:42:01.079143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:53.564566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147187+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:42:01.147187+0000","last_peered":"2024-02-26T10:42:01.147187+0000","last_clean":"2024-02-26T10:42:01.147187+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:42:01.147187+0000","last_undegraded":"2024-02-26T10:42:01.147187+0000","last_fullsized":"2024-02-26T10:42:01.147187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:03.513627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146982+0000","last_change":"2024-02-26T10:41:59.304661+0000","last_active":"2024-02-26T10:42:01.146982+0000","last_peered":"2024-02-26T10:42:01.146982+0000","last_clean":"2024-02-26T10:42:01.146982+0000","last_became_active":"2024-02-26T10:41:59.304358+0000","last_became_peered":"2024-02-26T10:41:59.304358+0000","last_unstale":"2024-02-26T10:42:01.146982+0000","last_undegraded":"2024-02-26T10:42:01.146982+0000","last_fullsized":"2024-02-26T10:42:01.146982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079625+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:42:01.079625+0000","last_peered":"2024-02-26T10:42:01.079625+0000","last_clean":"2024-02-26T10:42:01.079625+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:42:01.079625+0000","last_undegraded":"2024-02-26T10:42:01.079625+0000","last_fullsized":"2024-02-26T10:42:01.079625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:07.218989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147061+0000","last_change":"2024-02-26T10:41:59.310348+0000","last_active":"2024-02-26T10:42:01.147061+0000","last_peered":"2024-02-26T10:42:01.147061+0000","last_clean":"2024-02-26T10:42:01.147061+0000","last_became_active":"2024-02-26T10:41:59.309976+0000","last_became_peered":"2024-02-26T10:41:59.309976+0000","last_unstale":"2024-02-26T10:42:01.147061+0000","last_undegraded":"2024-02-26T10:42:01.147061+0000","last_fullsized":"2024-02-26T10:42:01.147061+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283912+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:42:00.283912+0000","last_peered":"2024-02-26T10:42:00.283912+0000","last_clean":"2024-02-26T10:42:00.283912+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:42:00.283912+0000","last_undegraded":"2024-02-26T10:42:00.283912+0000","last_fullsized":"2024-02-26T10:42:00.283912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:15.932239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284613+0000","last_change":"2024-02-26T10:41:59.287952+0000","last_active":"2024-02-26T10:42:00.284613+0000","last_peered":"2024-02-26T10:42:00.284613+0000","last_clean":"2024-02-26T10:42:00.284613+0000","last_became_active":"2024-02-26T10:41:59.287395+0000","last_became_peered":"2024-02-26T10:41:59.287395+0000","last_unstale":"2024-02-26T10:42:00.284613+0000","last_undegraded":"2024-02-26T10:42:00.284613+0000","last_fullsized":"2024-02-26T10:42:00.284613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:54.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283312+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:42:00.283312+0000","last_peered":"2024-02-26T10:42:00.283312+0000","last_clean":"2024-02-26T10:42:00.283312+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:42:00.283312+0000","last_undegraded":"2024-02-26T10:42:00.283312+0000","last_fullsized":"2024-02-26T10:42:00.283312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289627+0000","last_change":"2024-02-26T10:41:59.292771+0000","last_active":"2024-02-26T10:42:00.289627+0000","last_peered":"2024-02-26T10:42:00.289627+0000","last_clean":"2024-02-26T10:42:00.289627+0000","last_became_active":"2024-02-26T10:41:59.288778+0000","last_became_peered":"2024-02-26T10:41:59.288778+0000","last_unstale":"2024-02-26T10:42:00.289627+0000","last_undegraded":"2024-02-26T10:42:00.289627+0000","last_fullsized":"2024-02-26T10:42:00.289627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:18.908425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079500+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:42:01.079500+0000","last_peered":"2024-02-26T10:42:01.079500+0000","last_clean":"2024-02-26T10:42:01.079500+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:42:01.079500+0000","last_undegraded":"2024-02-26T10:42:01.079500+0000","last_fullsized":"2024-02-26T10:42:01.079500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:20.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078896+0000","last_change":"2024-02-26T10:41:59.287430+0000","last_active":"2024-02-26T10:42:01.078896+0000","last_peered":"2024-02-26T10:42:01.078896+0000","last_clean":"2024-02-26T10:42:01.078896+0000","last_became_active":"2024-02-26T10:41:59.287010+0000","last_became_peered":"2024-02-26T10:41:59.287010+0000","last_unstale":"2024-02-26T10:42:01.078896+0000","last_undegraded":"2024-02-26T10:42:01.078896+0000","last_fullsized":"2024-02-26T10:42:01.078896+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:24.065359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078521+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:42:01.078521+0000","last_peered":"2024-02-26T10:42:01.078521+0000","last_clean":"2024-02-26T10:42:01.078521+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:42:01.078521+0000","last_undegraded":"2024-02-26T10:42:01.078521+0000","last_fullsized":"2024-02-26T10:42:01.078521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:19.393230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283719+0000","last_change":"2024-02-26T10:41:59.301966+0000","last_active":"2024-02-26T10:42:00.283719+0000","last_peered":"2024-02-26T10:42:00.283719+0000","last_clean":"2024-02-26T10:42:00.283719+0000","last_became_active":"2024-02-26T10:41:59.301282+0000","last_became_peered":"2024-02-26T10:41:59.301282+0000","last_unstale":"2024-02-26T10:42:00.283719+0000","last_undegraded":"2024-02-26T10:42:00.283719+0000","last_fullsized":"2024-02-26T10:42:00.283719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282755+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:42:00.282755+0000","last_peered":"2024-02-26T10:42:00.282755+0000","last_clean":"2024-02-26T10:42:00.282755+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:42:00.282755+0000","last_undegraded":"2024-02-26T10:42:00.282755+0000","last_fullsized":"2024-02-26T10:42:00.282755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:07.631703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281456+0000","last_change":"2024-02-26T10:41:59.289078+0000","last_active":"2024-02-26T10:42:00.281456+0000","last_peered":"2024-02-26T10:42:00.281456+0000","last_clean":"2024-02-26T10:42:00.281456+0000","last_became_active":"2024-02-26T10:41:59.288529+0000","last_became_peered":"2024-02-26T10:41:59.288529+0000","last_unstale":"2024-02-26T10:42:00.281456+0000","last_undegraded":"2024-02-26T10:42:00.281456+0000","last_fullsized":"2024-02-26T10:42:00.281456+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:28.358063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288136+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:42:00.288136+0000","last_peered":"2024-02-26T10:42:00.288136+0000","last_clean":"2024-02-26T10:42:00.288136+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:42:00.288136+0000","last_undegraded":"2024-02-26T10:42:00.288136+0000","last_fullsized":"2024-02-26T10:42:00.288136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:56.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282489+0000","last_change":"2024-02-26T10:41:59.294420+0000","last_active":"2024-02-26T10:42:00.282489+0000","last_peered":"2024-02-26T10:42:00.282489+0000","last_clean":"2024-02-26T10:42:00.282489+0000","last_became_active":"2024-02-26T10:41:59.294147+0000","last_became_peered":"2024-02-26T10:41:59.294147+0000","last_unstale":"2024-02-26T10:42:00.282489+0000","last_undegraded":"2024-02-26T10:42:00.282489+0000","last_fullsized":"2024-02-26T10:42:00.282489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:32.774644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147339+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:42:01.147339+0000","last_peered":"2024-02-26T10:42:01.147339+0000","last_clean":"2024-02-26T10:42:01.147339+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:42:01.147339+0000","last_undegraded":"2024-02-26T10:42:01.147339+0000","last_fullsized":"2024-02-26T10:42:01.147339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145312+0000","last_change":"2024-02-26T10:41:59.296775+0000","last_active":"2024-02-26T10:42:01.145312+0000","last_peered":"2024-02-26T10:42:01.145312+0000","last_clean":"2024-02-26T10:42:01.145312+0000","last_became_active":"2024-02-26T10:41:59.296431+0000","last_became_peered":"2024-02-26T10:41:59.296431+0000","last_unstale":"2024-02-26T10:42:01.145312+0000","last_undegraded":"2024-02-26T10:42:01.145312+0000","last_fullsized":"2024-02-26T10:42:01.145312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146925+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:42:01.146925+0000","last_peered":"2024-02-26T10:42:01.146925+0000","last_clean":"2024-02-26T10:42:01.146925+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:42:01.146925+0000","last_undegraded":"2024-02-26T10:42:01.146925+0000","last_fullsized":"2024-02-26T10:42:01.146925+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:04.487859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283172+0000","last_change":"2024-02-26T10:41:59.300848+0000","last_active":"2024-02-26T10:42:00.283172+0000","last_peered":"2024-02-26T10:42:00.283172+0000","last_clean":"2024-02-26T10:42:00.283172+0000","last_became_active":"2024-02-26T10:41:59.300577+0000","last_became_peered":"2024-02-26T10:41:59.300577+0000","last_unstale":"2024-02-26T10:42:00.283172+0000","last_undegraded":"2024-02-26T10:42:00.283172+0000","last_fullsized":"2024-02-26T10:42:00.283172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146088+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:42:01.146088+0000","last_peered":"2024-02-26T10:42:01.146088+0000","last_clean":"2024-02-26T10:42:01.146088+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:42:01.146088+0000","last_undegraded":"2024-02-26T10:42:01.146088+0000","last_fullsized":"2024-02-26T10:42:01.146088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079430+0000","last_change":"2024-02-26T10:41:59.303988+0000","last_active":"2024-02-26T10:42:01.079430+0000","last_peered":"2024-02-26T10:42:01.079430+0000","last_clean":"2024-02-26T10:42:01.079430+0000","last_became_active":"2024-02-26T10:41:59.303677+0000","last_became_peered":"2024-02-26T10:41:59.303677+0000","last_unstale":"2024-02-26T10:42:01.079430+0000","last_undegraded":"2024-02-26T10:42:01.079430+0000","last_fullsized":"2024-02-26T10:42:01.079430+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:08.404204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145212+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:42:01.145212+0000","last_peered":"2024-02-26T10:42:01.145212+0000","last_clean":"2024-02-26T10:42:01.145212+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:42:01.145212+0000","last_undegraded":"2024-02-26T10:42:01.145212+0000","last_fullsized":"2024-02-26T10:42:01.145212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283321+0000","last_change":"2024-02-26T10:41:59.298661+0000","last_active":"2024-02-26T10:42:00.283321+0000","last_peered":"2024-02-26T10:42:00.283321+0000","last_clean":"2024-02-26T10:42:00.283321+0000","last_became_active":"2024-02-26T10:41:59.298384+0000","last_became_peered":"2024-02-26T10:41:59.298384+0000","last_unstale":"2024-02-26T10:42:00.283321+0000","last_undegraded":"2024-02-26T10:42:00.283321+0000","last_fullsized":"2024-02-26T10:42:00.283321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.289264+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:42:00.289264+0000","last_peered":"2024-02-26T10:42:00.289264+0000","last_clean":"2024-02-26T10:42:00.289264+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:42:00.289264+0000","last_undegraded":"2024-02-26T10:42:00.289264+0000","last_fullsized":"2024-02-26T10:42:00.289264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:06.345056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284045+0000","last_change":"2024-02-26T10:41:59.303009+0000","last_active":"2024-02-26T10:42:00.284045+0000","last_peered":"2024-02-26T10:42:00.284045+0000","last_clean":"2024-02-26T10:42:00.284045+0000","last_became_active":"2024-02-26T10:41:59.301703+0000","last_became_peered":"2024-02-26T10:41:59.301703+0000","last_unstale":"2024-02-26T10:42:00.284045+0000","last_undegraded":"2024-02-26T10:42:00.284045+0000","last_fullsized":"2024-02-26T10:42:00.284045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281389+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:42:00.281389+0000","last_peered":"2024-02-26T10:42:00.281389+0000","last_clean":"2024-02-26T10:42:00.281389+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:42:00.281389+0000","last_undegraded":"2024-02-26T10:42:00.281389+0000","last_fullsized":"2024-02-26T10:42:00.281389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:15.119340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288109+0000","last_change":"2024-02-26T10:41:59.285657+0000","last_active":"2024-02-26T10:42:00.288109+0000","last_peered":"2024-02-26T10:42:00.288109+0000","last_clean":"2024-02-26T10:42:00.288109+0000","last_became_active":"2024-02-26T10:41:59.285365+0000","last_became_peered":"2024-02-26T10:41:59.285365+0000","last_unstale":"2024-02-26T10:42:00.288109+0000","last_undegraded":"2024-02-26T10:42:00.288109+0000","last_fullsized":"2024-02-26T10:42:00.288109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:10.689718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144425+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:42:01.144425+0000","last_peered":"2024-02-26T10:42:01.144425+0000","last_clean":"2024-02-26T10:42:01.144425+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:42:01.144425+0000","last_undegraded":"2024-02-26T10:42:01.144425+0000","last_fullsized":"2024-02-26T10:42:01.144425+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.288015+0000","last_change":"2024-02-26T10:41:59.299415+0000","last_active":"2024-02-26T10:42:00.288015+0000","last_peered":"2024-02-26T10:42:00.288015+0000","last_clean":"2024-02-26T10:42:00.288015+0000","last_became_active":"2024-02-26T10:41:59.298995+0000","last_became_peered":"2024-02-26T10:41:59.298995+0000","last_unstale":"2024-02-26T10:42:00.288015+0000","last_undegraded":"2024-02-26T10:42:00.288015+0000","last_fullsized":"2024-02-26T10:42:00.288015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:38.058564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.282242+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:42:00.282242+0000","last_peered":"2024-02-26T10:42:00.282242+0000","last_clean":"2024-02-26T10:42:00.282242+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:42:00.282242+0000","last_undegraded":"2024-02-26T10:42:00.282242+0000","last_fullsized":"2024-02-26T10:42:00.282242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284598+0000","last_change":"2024-02-26T10:41:59.290032+0000","last_active":"2024-02-26T10:42:00.284598+0000","last_peered":"2024-02-26T10:42:00.284598+0000","last_clean":"2024-02-26T10:42:00.284598+0000","last_became_active":"2024-02-26T10:41:59.289604+0000","last_became_peered":"2024-02-26T10:41:59.289604+0000","last_unstale":"2024-02-26T10:42:00.284598+0000","last_undegraded":"2024-02-26T10:42:00.284598+0000","last_fullsized":"2024-02-26T10:42:00.284598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:23.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147525+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:42:01.147525+0000","last_peered":"2024-02-26T10:42:01.147525+0000","last_clean":"2024-02-26T10:42:01.147525+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:42:01.147525+0000","last_undegraded":"2024-02-26T10:42:01.147525+0000","last_fullsized":"2024-02-26T10:42:01.147525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147305+0000","last_change":"2024-02-26T10:41:59.299545+0000","last_active":"2024-02-26T10:42:01.147305+0000","last_peered":"2024-02-26T10:42:01.147305+0000","last_clean":"2024-02-26T10:42:01.147305+0000","last_became_active":"2024-02-26T10:41:59.298970+0000","last_became_peered":"2024-02-26T10:41:59.298970+0000","last_unstale":"2024-02-26T10:42:01.147305+0000","last_undegraded":"2024-02-26T10:42:01.147305+0000","last_fullsized":"2024-02-26T10:42:01.147305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283786+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:42:00.283786+0000","last_peered":"2024-02-26T10:42:00.283786+0000","last_clean":"2024-02-26T10:42:00.283786+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:42:00.283786+0000","last_undegraded":"2024-02-26T10:42:00.283786+0000","last_fullsized":"2024-02-26T10:42:00.283786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079654+0000","last_change":"2024-02-26T10:41:59.283472+0000","last_active":"2024-02-26T10:42:01.079654+0000","last_peered":"2024-02-26T10:42:01.079654+0000","last_clean":"2024-02-26T10:42:01.079654+0000","last_became_active":"2024-02-26T10:41:59.282978+0000","last_became_peered":"2024-02-26T10:41:59.282978+0000","last_unstale":"2024-02-26T10:42:01.079654+0000","last_undegraded":"2024-02-26T10:42:01.079654+0000","last_fullsized":"2024-02-26T10:42:01.079654+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:50.202422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283462+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:42:00.283462+0000","last_peered":"2024-02-26T10:42:00.283462+0000","last_clean":"2024-02-26T10:42:00.283462+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:42:00.283462+0000","last_undegraded":"2024-02-26T10:42:00.283462+0000","last_fullsized":"2024-02-26T10:42:00.283462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113746+0000","last_change":"2024-02-26T10:41:59.297414+0000","last_active":"2024-02-26T10:42:01.113746+0000","last_peered":"2024-02-26T10:42:01.113746+0000","last_clean":"2024-02-26T10:42:01.113746+0000","last_became_active":"2024-02-26T10:41:59.296486+0000","last_became_peered":"2024-02-26T10:41:59.296486+0000","last_unstale":"2024-02-26T10:42:01.113746+0000","last_undegraded":"2024-02-26T10:42:01.113746+0000","last_fullsized":"2024-02-26T10:42:01.113746+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:48.071684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.281498+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:42:00.281498+0000","last_peered":"2024-02-26T10:42:00.281498+0000","last_clean":"2024-02-26T10:42:00.281498+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:42:00.281498+0000","last_undegraded":"2024-02-26T10:42:00.281498+0000","last_fullsized":"2024-02-26T10:42:00.281498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:46.825934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144660+0000","last_change":"2024-02-26T10:41:59.292484+0000","last_active":"2024-02-26T10:42:01.144660+0000","last_peered":"2024-02-26T10:42:01.144660+0000","last_clean":"2024-02-26T10:42:01.144660+0000","last_became_active":"2024-02-26T10:41:59.292140+0000","last_became_peered":"2024-02-26T10:41:59.292140+0000","last_unstale":"2024-02-26T10:42:01.144660+0000","last_undegraded":"2024-02-26T10:42:01.144660+0000","last_fullsized":"2024-02-26T10:42:01.144660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113819+0000","last_change":"2024-02-26T10:41:58.269128+0000","last_active":"2024-02-26T10:42:01.113819+0000","last_peered":"2024-02-26T10:42:01.113819+0000","last_clean":"2024-02-26T10:42:01.113819+0000","last_became_active":"2024-02-26T10:41:58.268746+0000","last_became_peered":"2024-02-26T10:41:58.268746+0000","last_unstale":"2024-02-26T10:42:01.113819+0000","last_undegraded":"2024-02-26T10:42:01.113819+0000","last_fullsized":"2024-02-26T10:42:01.113819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:46:53.257949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113664+0000","last_change":"2024-02-26T10:41:59.292780+0000","last_active":"2024-02-26T10:42:01.113664+0000","last_peered":"2024-02-26T10:42:01.113664+0000","last_clean":"2024-02-26T10:42:01.113664+0000","last_became_active":"2024-02-26T10:41:59.292526+0000","last_became_peered":"2024-02-26T10:41:59.292526+0000","last_unstale":"2024-02-26T10:42:01.113664+0000","last_undegraded":"2024-02-26T10:42:01.113664+0000","last_fullsized":"2024-02-26T10:42:01.113664+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:12.549758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.113389+0000","last_change":"2024-02-26T10:41:58.269538+0000","last_active":"2024-02-26T10:42:01.113389+0000","last_peered":"2024-02-26T10:42:01.113389+0000","last_clean":"2024-02-26T10:42:01.113389+0000","last_became_active":"2024-02-26T10:41:58.269200+0000","last_became_peered":"2024-02-26T10:41:58.269200+0000","last_unstale":"2024-02-26T10:42:01.113389+0000","last_undegraded":"2024-02-26T10:42:01.113389+0000","last_fullsized":"2024-02-26T10:42:01.113389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:27.687984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144458+0000","last_change":"2024-02-26T10:41:59.305037+0000","last_active":"2024-02-26T10:42:01.144458+0000","last_peered":"2024-02-26T10:42:01.144458+0000","last_clean":"2024-02-26T10:42:01.144458+0000","last_became_active":"2024-02-26T10:41:59.304739+0000","last_became_peered":"2024-02-26T10:41:59.304739+0000","last_unstale":"2024-02-26T10:42:01.144458+0000","last_undegraded":"2024-02-26T10:42:01.144458+0000","last_fullsized":"2024-02-26T10:42:01.144458+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.280914+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:42:00.280914+0000","last_peered":"2024-02-26T10:42:00.280914+0000","last_clean":"2024-02-26T10:42:00.280914+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:42:00.280914+0000","last_undegraded":"2024-02-26T10:42:00.280914+0000","last_fullsized":"2024-02-26T10:42:00.280914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:26.696259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079328+0000","last_change":"2024-02-26T10:41:59.289483+0000","last_active":"2024-02-26T10:42:01.079328+0000","last_peered":"2024-02-26T10:42:01.079328+0000","last_clean":"2024-02-26T10:42:01.079328+0000","last_became_active":"2024-02-26T10:41:59.288870+0000","last_became_peered":"2024-02-26T10:41:59.288870+0000","last_unstale":"2024-02-26T10:42:01.079328+0000","last_undegraded":"2024-02-26T10:42:01.079328+0000","last_fullsized":"2024-02-26T10:42:01.079328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:02.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290450+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:42:00.290450+0000","last_peered":"2024-02-26T10:42:00.290450+0000","last_clean":"2024-02-26T10:42:00.290450+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:42:00.290450+0000","last_undegraded":"2024-02-26T10:42:00.290450+0000","last_fullsized":"2024-02-26T10:42:00.290450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284373+0000","last_change":"2024-02-26T10:41:59.294199+0000","last_active":"2024-02-26T10:42:00.284373+0000","last_peered":"2024-02-26T10:42:00.284373+0000","last_clean":"2024-02-26T10:42:00.284373+0000","last_became_active":"2024-02-26T10:41:59.293906+0000","last_became_peered":"2024-02-26T10:41:59.293906+0000","last_unstale":"2024-02-26T10:42:00.284373+0000","last_undegraded":"2024-02-26T10:42:00.284373+0000","last_fullsized":"2024-02-26T10:42:00.284373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:27.711178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290377+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:42:00.290377+0000","last_peered":"2024-02-26T10:42:00.290377+0000","last_clean":"2024-02-26T10:42:00.290377+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:42:00.290377+0000","last_undegraded":"2024-02-26T10:42:00.290377+0000","last_fullsized":"2024-02-26T10:42:00.290377+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:02:34.734605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146776+0000","last_change":"2024-02-26T10:41:59.298323+0000","last_active":"2024-02-26T10:42:01.146776+0000","last_peered":"2024-02-26T10:42:01.146776+0000","last_clean":"2024-02-26T10:42:01.146776+0000","last_became_active":"2024-02-26T10:41:59.295932+0000","last_became_peered":"2024-02-26T10:41:59.295932+0000","last_unstale":"2024-02-26T10:42:01.146776+0000","last_undegraded":"2024-02-26T10:42:01.146776+0000","last_fullsized":"2024-02-26T10:42:01.146776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146596+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:42:01.146596+0000","last_peered":"2024-02-26T10:42:01.146596+0000","last_clean":"2024-02-26T10:42:01.146596+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:42:01.146596+0000","last_undegraded":"2024-02-26T10:42:01.146596+0000","last_fullsized":"2024-02-26T10:42:01.146596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.079836+0000","last_change":"2024-02-26T10:41:59.293572+0000","last_active":"2024-02-26T10:42:01.079836+0000","last_peered":"2024-02-26T10:42:01.079836+0000","last_clean":"2024-02-26T10:42:01.079836+0000","last_became_active":"2024-02-26T10:41:59.292292+0000","last_became_peered":"2024-02-26T10:41:59.292292+0000","last_unstale":"2024-02-26T10:42:01.079836+0000","last_undegraded":"2024-02-26T10:42:01.079836+0000","last_fullsized":"2024-02-26T10:42:01.079836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:26.922522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.147038+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:42:01.147038+0000","last_peered":"2024-02-26T10:42:01.147038+0000","last_clean":"2024-02-26T10:42:01.147038+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:42:01.147038+0000","last_undegraded":"2024-02-26T10:42:01.147038+0000","last_fullsized":"2024-02-26T10:42:01.147038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:14.749617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.144982+0000","last_change":"2024-02-26T10:41:59.296354+0000","last_active":"2024-02-26T10:42:01.144982+0000","last_peered":"2024-02-26T10:42:01.144982+0000","last_clean":"2024-02-26T10:42:01.144982+0000","last_became_active":"2024-02-26T10:41:59.296012+0000","last_became_peered":"2024-02-26T10:41:59.296012+0000","last_unstale":"2024-02-26T10:42:01.144982+0000","last_undegraded":"2024-02-26T10:42:01.144982+0000","last_fullsized":"2024-02-26T10:42:01.144982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284034+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:42:00.284034+0000","last_peered":"2024-02-26T10:42:00.284034+0000","last_clean":"2024-02-26T10:42:00.284034+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:42:00.284034+0000","last_undegraded":"2024-02-26T10:42:00.284034+0000","last_fullsized":"2024-02-26T10:42:00.284034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:50:59.443811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284365+0000","last_change":"2024-02-26T10:41:59.297360+0000","last_active":"2024-02-26T10:42:00.284365+0000","last_peered":"2024-02-26T10:42:00.284365+0000","last_clean":"2024-02-26T10:42:00.284365+0000","last_became_active":"2024-02-26T10:41:59.297031+0000","last_became_peered":"2024-02-26T10:41:59.297031+0000","last_unstale":"2024-02-26T10:42:00.284365+0000","last_undegraded":"2024-02-26T10:42:00.284365+0000","last_fullsized":"2024-02-26T10:42:00.284365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078834+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:42:01.078834+0000","last_peered":"2024-02-26T10:42:01.078834+0000","last_clean":"2024-02-26T10:42:01.078834+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:42:01.078834+0000","last_undegraded":"2024-02-26T10:42:01.078834+0000","last_fullsized":"2024-02-26T10:42:01.078834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:58:22.045852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.283703+0000","last_change":"2024-02-26T10:41:59.295832+0000","last_active":"2024-02-26T10:42:00.283703+0000","last_peered":"2024-02-26T10:42:00.283703+0000","last_clean":"2024-02-26T10:42:00.283703+0000","last_became_active":"2024-02-26T10:41:59.295548+0000","last_became_peered":"2024-02-26T10:41:59.295548+0000","last_unstale":"2024-02-26T10:42:00.283703+0000","last_undegraded":"2024-02-26T10:42:00.283703+0000","last_fullsized":"2024-02-26T10:42:00.283703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.146232+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:42:01.146232+0000","last_peered":"2024-02-26T10:42:01.146232+0000","last_clean":"2024-02-26T10:42:01.146232+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:42:01.146232+0000","last_undegraded":"2024-02-26T10:42:01.146232+0000","last_fullsized":"2024-02-26T10:42:01.146232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.145956+0000","last_change":"2024-02-26T10:41:59.300681+0000","last_active":"2024-02-26T10:42:01.145956+0000","last_peered":"2024-02-26T10:42:01.145956+0000","last_clean":"2024-02-26T10:42:01.145956+0000","last_became_active":"2024-02-26T10:41:59.299849+0000","last_became_peered":"2024-02-26T10:41:59.299849+0000","last_unstale":"2024-02-26T10:42:01.145956+0000","last_undegraded":"2024-02-26T10:42:01.145956+0000","last_fullsized":"2024-02-26T10:42:01.145956+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:01.078965+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:42:01.078965+0000","last_peered":"2024-02-26T10:42:01.078965+0000","last_clean":"2024-02-26T10:42:01.078965+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:42:01.078965+0000","last_undegraded":"2024-02-26T10:42:01.078965+0000","last_fullsized":"2024-02-26T10:42:01.078965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:40:26.347537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.290123+0000","last_change":"2024-02-26T10:41:59.295485+0000","last_active":"2024-02-26T10:42:00.290123+0000","last_peered":"2024-02-26T10:42:00.290123+0000","last_clean":"2024-02-26T10:42:00.290123+0000","last_became_active":"2024-02-26T10:41:59.295154+0000","last_became_peered":"2024-02-26T10:41:59.295154+0000","last_unstale":"2024-02-26T10:42:00.290123+0000","last_undegraded":"2024-02-26T10:42:00.290123+0000","last_fullsized":"2024-02-26T10:42:00.290123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:15.764536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284516+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:42:00.284516+0000","last_peered":"2024-02-26T10:42:00.284516+0000","last_clean":"2024-02-26T10:42:00.284516+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:42:00.284516+0000","last_undegraded":"2024-02-26T10:42:00.284516+0000","last_fullsized":"2024-02-26T10:42:00.284516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":274,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-02-26T10:42:00.284158+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:42:00.284158+0000","last_peered":"2024-02-26T10:42:00.284158+0000","last_clean":"2024-02-26T10:42:00.284158+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:42:00.284158+0000","last_undegraded":"2024-02-26T10:42:00.284158+0000","last_fullsized":"2024-02-26T10:42:00.284158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1004431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":414687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":415627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":413993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":415605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":414575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":414187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1527808,"data_stored":1004395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:09.429 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:42:09.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-26T10:42:09.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.707+0000 7fc818a00640 1 Processor -- start 2024-02-26T10:42:09.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.707+0000 7fc818a00640 1 -- start start 2024-02-26T10:42:09.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.709+0000 7fc818a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc814072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.709+0000 7fc818a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81406e8c0 con 0x7fc814152ea0 2024-02-26T10:42:09.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.709+0000 7fc818a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81406eb60 con 0x7fc81406e460 2024-02-26T10:42:09.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.709+0000 7fc818a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141564b0 con 0x7fc81413c000 2024-02-26T10:42:09.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.709+0000 7fc813e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc814152ea0 0x7fc814153330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34656/0 (socket says 172.21.15.67:34656) 2024-02-26T10:42:09.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.709+0000 7fc813e00640 1 -- 172.21.15.67:0/3300092789 learned_addr learned my addr 172.21.15.67:0/3300092789 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:09.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc813400640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc814072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109257547 0 0) 0x7fc81406e8c0 con 0x7fc814152ea0 2024-02-26T10:42:09.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f8003520 con 0x7fc814152ea0 2024-02-26T10:42:09.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1181543954 0 0) 0x7fc7f8003520 con 0x7fc814152ea0 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 msgr2=0x7fc814072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc814072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 >> v1:172.21.15.82:6789/0 conn(0x7fc81406e460 legacy=0x7fc8140c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814156790 con 0x7fc814152ea0 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fc80c0019f0 con 0x7fc814152ea0 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 >> v1:172.21.15.67:6789/0 conn(0x7fc814152ea0 legacy=0x7fc814153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f8003a50 0x7fc7f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8004800 0x7fc7f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.710+0000 7fc812000640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc7f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.711+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141564b0 con 0x7fc7f8004800 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.711+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8005520 con 0x7fc7f8003a50 2024-02-26T10:42:09.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.711+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f800bf00 con 0x7fc81413c000 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.711+0000 7fc812a00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8004800 0x7fc7f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc813400640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f8003a50 0x7fc7f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc813e00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc7f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc812a00640 1 -- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 msgr2=0x7fc7f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc812a00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc7f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc812a00640 1 -- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f8003a50 msgr2=0x7fc7f8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc812a00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f8003a50 0x7fc7f8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc812a00640 1 -- 172.21.15.67:0/3300092789 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8003520 con 0x7fc7f8004800 2024-02-26T10:42:09.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.712+0000 7fc813e00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc81413c000 0x7fc7f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:09.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.713+0000 7fc812a00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8004800 0x7fc7f8004c80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc800001750 tx=0x7fc7f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58d03ef03c92717 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:09.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.713+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800001370 con 0x7fc7f8004800 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.713+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000052d0 con 0x7fc7f8004800 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.713+0000 7fc812000640 1 -- 172.21.15.67:0/3300092789 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800003830 con 0x7fc7f8004800 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc818a00640 1 -- 172.21.15.67:0/3300092789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8004800 msgr2=0x7fc7f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc818a00640 1 --2- 172.21.15.67:0/3300092789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8004800 0x7fc7f8004c80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc800001750 tx=0x7fc7f8005830 comp rx=0 tx=0).stop 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc813e00640 1 -- 172.21.15.67:0/3300092789 reap_dead start 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc818a00640 1 -- 172.21.15.67:0/3300092789 shutdown_connections 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc818a00640 1 -- 172.21.15.67:0/3300092789 >> 172.21.15.67:0/3300092789 conn(0x7fc8140a84a0 msgr2=0x7fc8140a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc818a00640 1 -- 172.21.15.67:0/3300092789 shutdown_connections 2024-02-26T10:42:09.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.714+0000 7fc818a00640 1 -- 172.21.15.67:0/3300092789 wait complete. 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.716+0000 7fc818a00640 1 Processor -- start 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.716+0000 7fc818a00640 1 -- start start 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc818a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7f8004800 0x7fc8141554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc818a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 0x7fc814125800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc818a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc814134590 0x7fc814125d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc818a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814157270 con 0x7fc814134040 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc818a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814156d50 con 0x7fc814134590 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc818a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81413c3e0 con 0x7fc7f8004800 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc812a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 0x7fc814125800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc812a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 0x7fc814125800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50466/0 (socket says 172.21.15.67:50466) 2024-02-26T10:42:09.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.717+0000 7fc812a00640 1 -- 172.21.15.67:0/1753664253 learned_addr learned my addr 172.21.15.67:0/1753664253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc813e00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc814134590 0x7fc814125d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc813400640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7f8004800 0x7fc8141554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc812a00640 1 -- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7f8004800 msgr2=0x7fc8141554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc812a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7f8004800 0x7fc8141554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc812a00640 1 -- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc814134590 msgr2=0x7fc814125d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc812a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc814134590 0x7fc814125d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc812a00640 1 -- 172.21.15.67:0/1753664253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814139230 con 0x7fc814134040 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc812a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 0x7fc814125800 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc800008fd0 tx=0x7fc800001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800041070 con 0x7fc814134040 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc800005480 con 0x7fc814134040 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800001370 con 0x7fc814134040 2024-02-26T10:42:09.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.718+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8141ee2c0 con 0x7fc814134040 2024-02-26T10:42:09.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.719+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8141ee880 con 0x7fc814134040 2024-02-26T10:42:09.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.720+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8141eef40 con 0x7fc814134040 2024-02-26T10:42:09.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.721+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc800044070 con 0x7fc814134040 2024-02-26T10:42:09.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.722+0000 7fc80b400640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7f406b500 0x7fc7f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:09.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.722+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8000abcf0 con 0x7fc814134040 2024-02-26T10:42:09.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.723+0000 7fc813400640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7f406b500 0x7fc7f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:09.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.724+0000 7fc813400640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7f406b500 0x7fc7f406d9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc804009950 tx=0x7fc804008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:09.727+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8141eef40 con 0x7fc814134040 2024-02-26T10:42:10.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.018+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc8141eab70 con 0x7fc814134040 2024-02-26T10:42:10.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.023+0000 7fc80b400640 1 -- 172.21.15.67:0/1753664253 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc8141eab70 con 0x7fc814134040 2024-02-26T10:42:10.021 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-02-26T10:42:10.021 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDuadxlSlkLNRAAwgSLbOZJE8AsgjH9jNPwNw== 2024-02-26T10:42:10.021 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-02-26T10:42:10.021 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:42:10.021 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-02-26T10:42:10.022 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.0 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7f406b500 msgr2=0x7fc7f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7f406b500 0x7fc7f406d9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc804009950 tx=0x7fc804008040 comp rx=0 tx=0).stop 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 msgr2=0x7fc814125800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 0x7fc814125800 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc800008fd0 tx=0x7fc800001120 comp rx=0 tx=0).stop 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 shutdown_connections 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7f406b500 0x7fc7f406d9c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc814134590 0x7fc814125d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc814134040 0x7fc814125800 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 --2- 172.21.15.67:0/1753664253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7f8004800 0x7fc8141554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 >> 172.21.15.67:0/1753664253 conn(0x7fc8140a84a0 msgr2=0x7fc814153330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 shutdown_connections 2024-02-26T10:42:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.027+0000 7fc818a00640 1 -- 172.21.15.67:0/1753664253 wait complete. 2024-02-26T10:42:10.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:10.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.265+0000 7f4b9ee00640 1 Processor -- start 2024-02-26T10:42:10.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.265+0000 7f4b9ee00640 1 -- start start 2024-02-26T10:42:10.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.267+0000 7f4b9ee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b9806db00 0x7f4b980abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.267+0000 7f4b9ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9806e0e0 con 0x7f4b980c2710 2024-02-26T10:42:10.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.267+0000 7f4b9ee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9806e3c0 con 0x7f4b980ac230 2024-02-26T10:42:10.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.267+0000 7f4b9ee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b98070c50 con 0x7f4b9806db00 2024-02-26T10:42:10.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.267+0000 7f4b9da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4b980c2710 0x7f4b98151d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34670/0 (socket says 172.21.15.67:34670) 2024-02-26T10:42:10.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.267+0000 7f4b9da00640 1 -- 172.21.15.67:0/1915448688 learned_addr learned my addr 172.21.15.67:0/1915448688 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:10.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241025680 0 0) 0x7f4b9806e0e0 con 0x7f4b980c2710 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1915448688 >> v2:172.21.15.82:3300/0 conn(0x7f4b9806db00 0x7f4b980abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b80003440 con 0x7f4b980c2710 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3806602742 0 0) 0x7f4b80003440 con 0x7f4b980c2710 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 >> v2:172.21.15.82:3300/0 conn(0x7f4b9806db00 msgr2=0x7f4b980abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 --2- 172.21.15.67:0/1915448688 >> v2:172.21.15.82:3300/0 conn(0x7f4b9806db00 0x7f4b980abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 >> v1:172.21.15.82:6789/0 conn(0x7f4b980ac230 legacy=0x7f4b980716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b98070ea0 con 0x7f4b980c2710 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f4b8c001670 con 0x7f4b980c2710 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1915448688 >> v2:172.21.15.82:3300/0 conn(0x7f4b9806db00 0x7f4b980abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 >> v1:172.21.15.67:6789/0 conn(0x7f4b980c2710 legacy=0x7f4b98151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b80003af0 0x7f4b80003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b8000ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8000b700 0x7f4b8000dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b98070c50 con 0x7f4b800089f0 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8000e4a0 con 0x7f4b8000b700 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8000e6a0 con 0x7f4b80003af0 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.268+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b8000ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9e400640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8000b700 0x7f4b8000dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9da00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b80003af0 0x7f4b80003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9d000640 1 -- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b80003af0 msgr2=0x7f4b80003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b80003af0 0x7f4b80003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9d000640 1 -- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8000b700 msgr2=0x7f4b8000dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8000b700 0x7f4b8000dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9d000640 1 -- 172.21.15.67:0/1915448688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80003440 con 0x7f4b800089f0 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9e400640 1 -- 172.21.15.67:0/1915448688 reap_dead start 2024-02-26T10:42:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.269+0000 7f4b9da00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b80003af0 0x7f4b80003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.270+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b8000ae60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098a0 tx=0x7f4b80003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad86252fcdd422e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.270+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400f040 con 0x7f4b800089f0 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.270+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b84004230 con 0x7f4b800089f0 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.270+0000 7f4b97e00640 1 -- 172.21.15.67:0/1915448688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84003610 con 0x7f4b800089f0 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1915448688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 msgr2=0x7f4b8000ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b8000ae60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098a0 tx=0x7f4b80003730 comp rx=0 tx=0).stop 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1915448688 shutdown_connections 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1915448688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b8000ae60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1915448688 >> 172.21.15.67:0/1915448688 conn(0x7f4b980c63f0 msgr2=0x7f4b980c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1915448688 shutdown_connections 2024-02-26T10:42:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.271+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1915448688 wait complete. 2024-02-26T10:42:10.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.273+0000 7f4b9ee00640 1 Processor -- start 2024-02-26T10:42:10.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.273+0000 7f4b9ee00640 1 -- start start 2024-02-26T10:42:10.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b98147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b981dcd30 0x7f4b981eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b981ddb20 0x7f4b981eb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980abcf0 con 0x7f4b800089f0 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980c2af0 con 0x7f4b981dcd30 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b980abe70 con 0x7f4b981ddb20 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b98147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b98147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50478/0 (socket says 172.21.15.67:50478) 2024-02-26T10:42:10.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.274+0000 7f4b9da00640 1 -- 172.21.15.67:0/1621471320 learned_addr learned my addr 172.21.15.67:0/1621471320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b981dcd30 0x7f4b981eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9e400640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b981ddb20 0x7f4b981eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9da00640 1 -- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b981ddb20 msgr2=0x7f4b981eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9da00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b981ddb20 0x7f4b981eb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9da00640 1 -- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b981dcd30 msgr2=0x7f4b981eaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9da00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b981dcd30 0x7f4b981eaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9da00640 1 -- 172.21.15.67:0/1621471320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b981da980 con 0x7f4b800089f0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9da00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b98147f40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d7a0 tx=0x7f4b8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c019070 con 0x7f4b800089f0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0072d0 con 0x7f4b800089f0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0047e0 con 0x7f4b800089f0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.275+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b981dac60 con 0x7f4b800089f0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.276+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b980c6000 con 0x7f4b800089f0 2024-02-26T10:42:10.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.277+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b98074a30 con 0x7f4b800089f0 2024-02-26T10:42:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.278+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b8c004980 con 0x7f4b800089f0 2024-02-26T10:42:10.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.279+0000 7f4b95600640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4b7806b500 0x7f4b7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:10.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.279+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b8c089a40 con 0x7f4b800089f0 2024-02-26T10:42:10.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.280+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4b7806b500 0x7f4b7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.281+0000 7f4b9d000640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4b7806b500 0x7f4b7806d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4b84035000 tx=0x7f4b8403a040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.284+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b8c004c70 con 0x7f4b800089f0 2024-02-26T10:42:10.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.524+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b9806ea20 con 0x7f4b800089f0 2024-02-26T10:42:10.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.526+0000 7f4b95600640 1 -- 172.21.15.67:0/1621471320 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4b8c0588b0 con 0x7f4b800089f0 2024-02-26T10:42:10.523 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:10.523 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6447,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-02-26T10:42:10.523 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 48 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.528+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4b7806b500 msgr2=0x7f4b7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.528+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4b7806b500 0x7f4b7806d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4b84035000 tx=0x7f4b8403a040 comp rx=0 tx=0).stop 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.528+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 msgr2=0x7f4b98147f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.528+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b98147f40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d7a0 tx=0x7f4b8c00dc60 comp rx=0 tx=0).stop 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 shutdown_connections 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b981ddb20 0x7f4b981eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f4b7806b500 0x7f4b7806d9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b981dcd30 0x7f4b981eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 --2- 172.21.15.67:0/1621471320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800089f0 0x7f4b98147f40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:10.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 >> 172.21.15.67:0/1621471320 conn(0x7f4b980c63f0 msgr2=0x7f4b980c0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:10.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 shutdown_connections 2024-02-26T10:42:10.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:10.529+0000 7f4b9ee00640 1 -- 172.21.15.67:0/1621471320 wait complete. 2024-02-26T10:42:10.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-26T10:41:58.715593+0000', 'modified': '2024-02-26T10:42:01.803454+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6450}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6450': {'gid': 6450, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.67:6835/2183069114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6833', 'nonce': 2183069114}, {'type': 'v1', 'addr': '172.21.15.67:6835', 'nonce': 2183069114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-26T10:42:10.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-26T10:42:10.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-26T10:42:11.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.184+0000 7fe95ca00640 1 Processor -- start 2024-02-26T10:42:11.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.185+0000 7fe95ca00640 1 -- start start 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe9580c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe95ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe95806d290 0x7fe9580c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe95ca00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe958152760 con 0x7fe958152380 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe95ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9580c7550 con 0x7fe95806d290 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe95ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580c7830 con 0x7fe9580c7020 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe957e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe95806d290 0x7fe9580c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe957e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe95806d290 0x7fe9580c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50494/0 (socket says 172.21.15.67:50494) 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.186+0000 7fe957e00640 1 -- 172.21.15.67:0/1191206033 learned_addr learned my addr 172.21.15.67:0/1191206033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.187+0000 7fe956a00640 1 --2- 172.21.15.67:0/1191206033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe9580c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.187+0000 7fe957e00640 1 -- 172.21.15.67:0/1191206033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 msgr2=0x7fe9580c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.187+0000 7fe957e00640 1 --2- 172.21.15.67:0/1191206033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe9580c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.187+0000 7fe957e00640 1 -- 172.21.15.67:0/1191206033 >> v1:172.21.15.82:6789/0 conn(0x7fe958152380 legacy=0x7fe958155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.187+0000 7fe957e00640 1 -- 172.21.15.67:0/1191206033 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95806ca90 con 0x7fe95806d290 2024-02-26T10:42:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.188+0000 7fe957e00640 1 --2- 172.21.15.67:0/1191206033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe9580c4f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe950009e50 tx=0x7fe95002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504e2eb838830228 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.188+0000 7fe956000640 1 -- 172.21.15.67:0/1191206033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95002c9e0 con 0x7fe95806d290 2024-02-26T10:42:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.188+0000 7fe956000640 1 -- 172.21.15.67:0/1191206033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95002cb80 con 0x7fe95806d290 2024-02-26T10:42:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.188+0000 7fe956000640 1 -- 172.21.15.67:0/1191206033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500343f0 con 0x7fe95806d290 2024-02-26T10:42:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 -- 172.21.15.67:0/1191206033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 msgr2=0x7fe9580c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1191206033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe9580c4f90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe950009e50 tx=0x7fe95002be40 comp rx=0 tx=0).stop 2024-02-26T10:42:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 -- 172.21.15.67:0/1191206033 shutdown_connections 2024-02-26T10:42:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1191206033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe9580c4f90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1191206033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe9580c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 -- 172.21.15.67:0/1191206033 >> 172.21.15.67:0/1191206033 conn(0x7fe9580ac1a0 msgr2=0x7fe9580ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 -- 172.21.15.67:0/1191206033 shutdown_connections 2024-02-26T10:42:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.189+0000 7fe95ca00640 1 -- 172.21.15.67:0/1191206033 wait complete. 2024-02-26T10:42:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.191+0000 7fe95ca00640 1 Processor -- start 2024-02-26T10:42:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.191+0000 7fe95ca00640 1 -- start start 2024-02-26T10:42:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe958147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe958149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe958152380 0x7fe9581eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe95ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806d8a0 con 0x7fe95806d290 2024-02-26T10:42:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe95ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806d720 con 0x7fe958152380 2024-02-26T10:42:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe95ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806da20 con 0x7fe9580c7020 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe958147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe958147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50496/0 (socket says 172.21.15.67:50496) 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.192+0000 7fe957400640 1 -- 172.21.15.67:0/1834856714 learned_addr learned my addr 172.21.15.67:0/1834856714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe956a00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe958149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957e00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe958152380 0x7fe9581eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957400640 1 -- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 msgr2=0x7fe958149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957400640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe958149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957400640 1 -- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe958152380 msgr2=0x7fe9581eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957400640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe958152380 0x7fe9581eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957400640 1 -- 172.21.15.67:0/1834856714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9581e2020 con 0x7fe95806d290 2024-02-26T10:42:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.193+0000 7fe957400640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe958147090 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c430 tx=0x7fe94800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.194+0000 7fe94f400640 1 -- 172.21.15.67:0/1834856714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011790 con 0x7fe95806d290 2024-02-26T10:42:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.194+0000 7fe94f400640 1 -- 172.21.15.67:0/1834856714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948017070 con 0x7fe95806d290 2024-02-26T10:42:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.194+0000 7fe94f400640 1 -- 172.21.15.67:0/1834856714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011b70 con 0x7fe95806d290 2024-02-26T10:42:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.194+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe958151160 con 0x7fe95806d290 2024-02-26T10:42:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.194+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9581516e0 con 0x7fe95806d290 2024-02-26T10:42:11.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.197+0000 7fe94f400640 1 -- 172.21.15.67:0/1834856714 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe948011d10 con 0x7fe95806d290 2024-02-26T10:42:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.198+0000 7fe94f400640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe93406b4d0 0x7fe93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.198+0000 7fe94f400640 1 -- 172.21.15.67:0/1834856714 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe94808a1a0 con 0x7fe95806d290 2024-02-26T10:42:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.198+0000 7fe956a00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe93406b4d0 0x7fe93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.198+0000 7fe961014740 1 Processor -- start 2024-02-26T10:42:11.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 -- start start 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae421261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae420eb410 0x55ae42125450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae420ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4204e5d0 con 0x55ae42125070 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae41ff7900 con 0x55ae421281e0 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.199+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae4204a670 con 0x55ae420eb410 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe956a00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe93406b4d0 0x7fe93406d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae421261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae421261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50506/0 (socket says 172.21.15.67:50506) 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 -- 172.21.15.67:0/3447359809 learned_addr learned my addr 172.21.15.67:0/3447359809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe956a00640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae420eb410 0x55ae42125450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 -- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae420eb410 msgr2=0x55ae42125450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae420eb410 0x55ae42125450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957e00640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae420ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 -- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 msgr2=0x55ae420ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae420ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 -- 172.21.15.67:0/3447359809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae41d632d0 con 0x55ae42125070 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe956a00640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae420eb410 0x55ae42125450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957e00640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae420ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.200+0000 7fe957400640 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae421261e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe948054c50 tx=0x7fe948059030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d15efcfa223e73 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe947400640 1 -- 172.21.15.67:0/3447359809 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948070020 con 0x55ae42125070 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe947400640 1 -- 172.21.15.67:0/3447359809 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948060f90 con 0x55ae42125070 2024-02-26T10:42:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe947400640 1 -- 172.21.15.67:0/3447359809 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948054e60 con 0x55ae42125070 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 -- 172.21.15.67:0/3447359809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 msgr2=0x55ae421261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae421261e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe948054c50 tx=0x7fe948059030 comp rx=0 tx=0).stop 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 -- 172.21.15.67:0/3447359809 shutdown_connections 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae420ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae420eb410 0x55ae42125450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 --2- 172.21.15.67:0/3447359809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae421261e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.201+0000 7fe961014740 1 -- 172.21.15.67:0/3447359809 >> 172.21.15.67:0/3447359809 conn(0x55ae420e6a50 msgr2=0x55ae4210b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.202+0000 7fe961014740 1 -- 172.21.15.67:0/3447359809 shutdown_connections 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.202+0000 7fe961014740 1 -- 172.21.15.67:0/3447359809 wait complete. 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.202+0000 7fe961014740 1 -- start start 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.202+0000 7fe961014740 1 Processor -- start 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.203+0000 7fe961014740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae4210a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.203+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42122bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.203+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae4211d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.203+0000 7fe961014740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae42127d50 con 0x55ae42125070 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.203+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4200ad60 con 0x55ae421281e0 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.203+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae42127810 con 0x55ae42123130 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe961014740 10 client.0 _ll_register_callbacks cb 0x55ae42046320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae4210a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae4210a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50516/0 (socket says 172.21.15.67:50516) 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 -- 172.21.15.67:0/1863099635 learned_addr learned my addr 172.21.15.67:0/1863099635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 -- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 msgr2=0x55ae4211d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae4211d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 -- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 msgr2=0x55ae42122bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe956a00640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42122bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42122bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe957400640 1 -- 172.21.15.67:0/1863099635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae41e2d6b0 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.204+0000 7fe956a00640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42122bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe957400640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae4210a830 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe94801b000 tx=0x7fe94801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304afa2b53b147e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe93be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe93be00640 1 -- 172.21.15.67:0/1863099635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948075060 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe93be00640 1 -- 172.21.15.67:0/1863099635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe948054e60 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe948060f90 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe93be00640 1 -- 172.21.15.67:0/1863099635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94807c050 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe961014740 10 client.6615 fetch_fsmap learned FSMap version 48 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55ae4211eb50 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe93be00640 1 -- 172.21.15.67:0/1863099635 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948060f90 con 0x55ae42125070 2024-02-26T10:42:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe93be00640 1 -- 172.21.15.67:0/1863099635 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9480772b0 con 0x55ae42125070 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe961014740 10 client.6615 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.205+0000 7fe961014740 10 client.6615 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe961014740 20 client.6615 populate_metadata read hostname 'smithi067' 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe961014740 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42139cd0 0x55ae4213a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe961014740 4 client.6615 mds_command: new command op to 6450 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 --> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae4205bb10 con 0x55ae42139cd0 2024-02-26T10:42:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe956a00640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42139cd0 0x55ae4213a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe956a00640 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42139cd0 0x55ae4213a0b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a76085f0c9d812 server_cookie=7a385f012ceb008c in_seq=0 out_seq=0 2024-02-26T10:42:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.206+0000 7fe93be00640 10 client.6615 ms_handle_connect on v2:172.21.15.67:6833/2183069114 2024-02-26T10:42:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.207+0000 7fe93be00640 1 -- 172.21.15.67:0/1863099635 <== mds.0 v2:172.21.15.67:6833/2183069114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae4205bb10 con 0x55ae42139cd0 2024-02-26T10:42:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.207+0000 7fe93be00640 10 client.6615 handle_command_reply: tid=0 2024-02-26T10:42:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.207+0000 7fe961014740 1 client.6615 shutdown 2024-02-26T10:42:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 msgr2=0x55ae4210a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae4210a830 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe94801b000 tx=0x7fe94801e040 comp rx=0 tx=0).stop 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 shutdown_connections 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42139cd0 0x55ae4213a0b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae4211d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42122bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe93be00640 0 client.6615 ms_handle_reset on v2:172.21.15.67:6833/2183069114 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 --2- 172.21.15.67:0/1863099635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae4210a830 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 >> 172.21.15.67:0/1863099635 conn(0x55ae420eb410 msgr2=0x55ae42126770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 shutdown_connections 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 1 -- 172.21.15.67:0/1863099635 wait complete. 2024-02-26T10:42:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.208+0000 7fe961014740 20 client.6615 trim_cache size 0 max 16384 2024-02-26T10:42:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.225+0000 7fe961014740 1 Processor -- start 2024-02-26T10:42:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.225+0000 7fe961014740 1 -- start start 2024-02-26T10:42:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae42126770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe961014740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae42125450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42125990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe961014740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580bd440 con 0x55ae42125070 2024-02-26T10:42:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580bc840 con 0x55ae421281e0 2024-02-26T10:42:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580bce40 con 0x55ae42123130 2024-02-26T10:42:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe956a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae42125450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe956a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae42125450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50526/0 (socket says 172.21.15.67:50526) 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.226+0000 7fe956a00640 1 -- 172.21.15.67:0/1018794509 learned_addr learned my addr 172.21.15.67:0/1018794509 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe957400640 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae42126770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe957e00640 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42125990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe956a00640 1 -- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 msgr2=0x55ae42126770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe956a00640 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae42126770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe956a00640 1 -- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 msgr2=0x55ae42125990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe956a00640 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42125990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe956a00640 1 -- 172.21.15.67:0/1018794509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae420e6e40 con 0x55ae42125070 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe956a00640 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae42125450 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c010 tx=0x7fe940015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6633435fc6781e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe946a00640 1 -- 172.21.15.67:0/1018794509 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940014c90 con 0x55ae42125070 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe946a00640 1 -- 172.21.15.67:0/1018794509 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94002c070 con 0x55ae42125070 2024-02-26T10:42:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.227+0000 7fe946a00640 1 -- 172.21.15.67:0/1018794509 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940026370 con 0x55ae42125070 2024-02-26T10:42:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 -- 172.21.15.67:0/1018794509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 msgr2=0x55ae42125450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae42125450 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c010 tx=0x7fe940015000 comp rx=0 tx=0).stop 2024-02-26T10:42:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 -- 172.21.15.67:0/1018794509 shutdown_connections 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae421281e0 0x55ae42125990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42125070 0x55ae42125450 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 --2- 172.21.15.67:0/1018794509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae42123130 0x55ae42126770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 -- 172.21.15.67:0/1018794509 >> 172.21.15.67:0/1018794509 conn(0x55ae420eb410 msgr2=0x55ae420e9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 -- 172.21.15.67:0/1018794509 shutdown_connections 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.228+0000 7fe961014740 1 -- 172.21.15.67:0/1018794509 wait complete. 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.229+0000 7fe961014740 1 -- start start 2024-02-26T10:42:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.229+0000 7fe961014740 1 Processor -- start 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 0x55ae42125850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae42125070 0x55ae4213f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae421281e0 0x55ae4218d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae42127d50 con 0x55ae42123130 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4200ad60 con 0x55ae42125070 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae420fd5c0 con 0x55ae421281e0 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe961014740 10 client.0 _ll_register_callbacks cb 0x7fe948054b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-26T10:42:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 0x55ae42125850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe957400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 0x55ae42125850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50530/0 (socket says 172.21.15.67:50530) 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe957400640 1 -- 172.21.15.67:0/1833980993 learned_addr learned my addr 172.21.15.67:0/1833980993 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe956a00640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae42125070 0x55ae4213f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.230+0000 7fe957e00640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae421281e0 0x55ae4218d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957400640 1 -- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae421281e0 msgr2=0x55ae4218d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957400640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae421281e0 0x55ae4218d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957400640 1 -- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae42125070 msgr2=0x55ae4213f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957400640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae42125070 0x55ae4213f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957400640 1 -- 172.21.15.67:0/1833980993 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae42124150 con 0x55ae42123130 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957e00640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae421281e0 0x55ae4218d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe957400640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 0x55ae42125850 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe948054c00 tx=0x7fe948003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93b5485a76a700d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe93be00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe93be00640 1 -- 172.21.15.67:0/1833980993 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948070040 con 0x55ae42123130 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe93be00640 1 -- 172.21.15.67:0/1833980993 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9480772b0 con 0x55ae42123130 2024-02-26T10:42:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.231+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe94002c070 con 0x55ae42123130 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe93be00640 1 -- 172.21.15.67:0/1833980993 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9480585c0 con 0x55ae42123130 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 10 client.6621 fetch_fsmap learned FSMap version 48 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55ae4213fc80 con 0x55ae42123130 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe93be00640 1 -- 172.21.15.67:0/1833980993 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94002c070 con 0x55ae42123130 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe93be00640 1 -- 172.21.15.67:0/1833980993 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe948057c40 con 0x55ae42123130 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 10 client.6621 fetch_fsmap finished waiting for FSMap version 48 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 10 client.6621 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 20 client.6621 populate_metadata read hostname 'smithi067' 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42193d60 0x55ae42194140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 4 client.6621 mds_command: new command op to 6450 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 --> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae42055e60 con 0x55ae42193d60 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.232+0000 7fe956a00640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42193d60 0x55ae42194140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe956a00640 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42193d60 0x55ae42194140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7145a91de3aef91b server_cookie=41d4e34877d8d4ef in_seq=0 out_seq=0 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe93be00640 10 client.6621 ms_handle_connect on v2:172.21.15.67:6833/2183069114 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe93be00640 1 -- 172.21.15.67:0/1833980993 <== mds.0 v2:172.21.15.67:6833/2183069114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ae42055e60 con 0x55ae42193d60 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe93be00640 10 client.6621 handle_command_reply: tid=0 2024-02-26T10:42:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 client.6621 shutdown 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 msgr2=0x55ae42125850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 0x55ae42125850 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe948054c00 tx=0x7fe948003940 comp rx=0 tx=0).stop 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 shutdown_connections 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:6833/2183069114,v1:172.21.15.67:6835/2183069114] conn(0x55ae42193d60 0x55ae42194140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ae421281e0 0x55ae4218d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae42125070 0x55ae4213f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 --2- 172.21.15.67:0/1833980993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ae42123130 0x55ae42125850 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe93be00640 0 client.6621 ms_handle_reset on v2:172.21.15.67:6833/2183069114 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.233+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 >> 172.21.15.67:0/1833980993 conn(0x55ae420eb410 msgr2=0x55ae42123720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 shutdown_connections 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe961014740 1 -- 172.21.15.67:0/1833980993 wait complete. 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe961014740 20 client.6621 trim_cache size 0 max 16384 2024-02-26T10:42:11.234 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "f4a3a702-2ce8-46d3-b32d-e4f6872b8b58", 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 6450, 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 11.958707540000001, 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 48, 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 76, 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 76, 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 24.528020590000001 2024-02-26T10:42:11.235 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:42:11.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe93406b4d0 msgr2=0x7fe93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe93406b4d0 0x7fe93406d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940008040 comp rx=0 tx=0).stop 2024-02-26T10:42:11.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 msgr2=0x7fe958147090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.234+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe958147090 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c430 tx=0x7fe94800c8f0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 shutdown_connections 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe93406b4d0 0x7fe93406d990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe958152380 0x7fe9581eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9580c7020 0x7fe958149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 --2- 172.21.15.67:0/1834856714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe95806d290 0x7fe958147090 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 >> 172.21.15.67:0/1834856714 conn(0x7fe9580ac1a0 msgr2=0x7fe958152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 shutdown_connections 2024-02-26T10:42:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.235+0000 7fe95ca00640 1 -- 172.21.15.67:0/1834856714 wait complete. 2024-02-26T10:42:11.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:42:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.464+0000 7f2119800640 1 Processor -- start 2024-02-26T10:42:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.464+0000 7f2119800640 1 -- start start 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.465+0000 7f2119800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2114152380 0x7f21140c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.465+0000 7f2119800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f2114155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.465+0000 7f2119800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2114152760 con 0x7f211406d290 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.465+0000 7f2119800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2114155cd0 con 0x7f2114152380 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f2119800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21140c7550 con 0x7f21140c7020 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f2118e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f211406d290 0x7f21140c4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34674/0 (socket says 172.21.15.67:34674) 2024-02-26T10:42:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f2118e00640 1 -- 172.21.15.67:0/3350031138 learned_addr learned my addr 172.21.15.67:0/3350031138 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f2113e00640 1 --2- 172.21.15.67:0/3350031138 >> v2:172.21.15.82:3300/0 conn(0x7f2114152380 0x7f21140c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f210be00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f2114155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674089327 0 0) 0x7f2114152760 con 0x7f211406d290 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.466+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20f8003520 con 0x7f211406d290 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153293091 0 0) 0x7f20f8003520 con 0x7f211406d290 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 msgr2=0x7f2114155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f2114155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 >> v2:172.21.15.82:3300/0 conn(0x7f2114152380 msgr2=0x7f21140c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 --2- 172.21.15.67:0/3350031138 >> v2:172.21.15.82:3300/0 conn(0x7f2114152380 0x7f21140c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21140c7830 con 0x7f211406d290 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f210c0019f0 con 0x7f211406d290 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 >> v1:172.21.15.67:6789/0 conn(0x7f211406d290 legacy=0x7f21140c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f20f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f20f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8004d40 0x7f20f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114155cd0 con 0x7f20f80039b0 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21140c7550 con 0x7f20f8004d40 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f800b9f0 con 0x7f21140c7020 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.467+0000 7f2113e00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f20f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f210be00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f20f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2113e00640 1 -- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 msgr2=0x7f20f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2118e00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8004d40 0x7f20f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2113e00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f20f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2113e00640 1 -- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8004d40 msgr2=0x7f20f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2113e00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8004d40 0x7f20f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2113e00640 1 -- 172.21.15.67:0/3350031138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8003520 con 0x7f20f80039b0 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f2118e00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f8004d40 0x7f20f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.468+0000 7f210be00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21140c7020 0x7f20f8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.469+0000 7f2113e00640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f20f8003dd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2104009ac0 tx=0x7f20f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354be39cbd69dc5a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.469+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104039070 con 0x7f20f80039b0 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.469+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210402c710 con 0x7f20f80039b0 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.469+0000 7f2113400640 1 -- 172.21.15.67:0/3350031138 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104009690 con 0x7f20f80039b0 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2119800640 1 -- 172.21.15.67:0/3350031138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 msgr2=0x7f20f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2119800640 1 --2- 172.21.15.67:0/3350031138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f20f8003dd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2104009ac0 tx=0x7f20f8005830 comp rx=0 tx=0).stop 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2118e00640 1 -- 172.21.15.67:0/3350031138 reap_dead start 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2119800640 1 -- 172.21.15.67:0/3350031138 shutdown_connections 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2119800640 1 -- 172.21.15.67:0/3350031138 >> 172.21.15.67:0/3350031138 conn(0x7f21140ac1a0 msgr2=0x7f21140ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2119800640 1 -- 172.21.15.67:0/3350031138 shutdown_connections 2024-02-26T10:42:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.470+0000 7f2119800640 1 -- 172.21.15.67:0/3350031138 wait complete. 2024-02-26T10:42:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.473+0000 7f2119800640 1 Processor -- start 2024-02-26T10:42:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.473+0000 7f2119800640 1 -- start start 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2119800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f21140c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2119800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140ab2d0 0x7f2114148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2119800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2114149100 0x7f211414db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2119800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21140c56a0 con 0x7f20f80039b0 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2119800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211406cd70 con 0x7f21140ab2d0 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2119800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21140c5820 con 0x7f2114149100 2024-02-26T10:42:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2113e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f21140c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2113e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f21140c0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50552/0 (socket says 172.21.15.67:50552) 2024-02-26T10:42:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2113e00640 1 -- 172.21.15.67:0/220431825 learned_addr learned my addr 172.21.15.67:0/220431825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2118e00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2114149100 0x7f211414db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2113e00640 1 -- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2114149100 msgr2=0x7f211414db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.474+0000 7f2113e00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2114149100 0x7f211414db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2113e00640 1 -- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140ab2d0 msgr2=0x7f2114148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f210be00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140ab2d0 0x7f2114148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2113e00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140ab2d0 0x7f2114148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2113e00640 1 -- 172.21.15.67:0/220431825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211414d030 con 0x7f20f80039b0 2024-02-26T10:42:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2113e00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f21140c0b10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2104000c00 tx=0x7f2104004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104039050 con 0x7f20f80039b0 2024-02-26T10:42:11.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210403a040 con 0x7f20f80039b0 2024-02-26T10:42:11.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211414d310 con 0x7f20f80039b0 2024-02-26T10:42:11.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.475+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21040044e0 con 0x7f20f80039b0 2024-02-26T10:42:11.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.476+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211414d5f0 con 0x7f20f80039b0 2024-02-26T10:42:11.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.477+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2114074a30 con 0x7f20f80039b0 2024-02-26T10:42:11.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.478+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2104047020 con 0x7f20f80039b0 2024-02-26T10:42:11.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.479+0000 7f2110c00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f006b500 0x7f20f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:11.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.479+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21040ab810 con 0x7f20f80039b0 2024-02-26T10:42:11.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.479+0000 7f210be00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f006b500 0x7f20f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:11.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.480+0000 7f210be00640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f006b500 0x7f20f006d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f21140c9a60 tx=0x7f20fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.483+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f210404e020 con 0x7f20f80039b0 2024-02-26T10:42:11.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.723+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21141e2320 con 0x7f20f80039b0 2024-02-26T10:42:11.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.725+0000 7f2110c00640 1 -- 172.21.15.67:0/220431825 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f210407a700 con 0x7f20f80039b0 2024-02-26T10:42:11.722 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f006b500 msgr2=0x7f20f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f006b500 0x7f20f006d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f21140c9a60 tx=0x7f20fc008040 comp rx=0 tx=0).stop 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 msgr2=0x7f21140c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f21140c0b10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2104000c00 tx=0x7f2104004730 comp rx=0 tx=0).stop 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 shutdown_connections 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2114149100 0x7f211414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f20f006b500 0x7f20f006d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21140ab2d0 0x7f2114148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 --2- 172.21.15.67:0/220431825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20f80039b0 0x7f21140c0b10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 >> 172.21.15.67:0/220431825 conn(0x7f21140ac1a0 msgr2=0x7f21140c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 shutdown_connections 2024-02-26T10:42:11.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:11.727+0000 7f2119800640 1 -- 172.21.15.67:0/220431825 wait complete. 2024-02-26T10:42:11.742 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:42:11.743 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:42:11.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:11.743 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:11.792 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42318sec preferred_lft 42318sec 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:42:11.793 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::288e:93ff:fe9f:f95d/64 scope link 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::3cfc:46ff:fe71:84e3/64 scope link 2024-02-26T10:42:11.794 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:11.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:11.795 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:42:11.886 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:42:11.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:11.891 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:42:11.945 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:42:11.946 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:42:11.946 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:42:11.946 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:42:11.946 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:42:11.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:11.946 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:42:11.946 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:42:11.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:42:11.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:42:11.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:42:11.947 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:42:12.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:12.138 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:42:12.190 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:42:12.191 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:42:12.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:12.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:42:12.282 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:12.282 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:12.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:12.340 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:12.390 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-02-26T10:42:12.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:12.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:12.528 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:12.528 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:12.560 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:12.561 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:12.561 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:42:12.561 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: 0@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:42:12.561 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:42:12.563 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T10:42:12.563 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:42:12.563 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:42:12.563 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:42:12.563 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:42:12.567 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:42:12.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:12.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:12.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:12.653 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:42:12.729 INFO:teuthology.orchestra.run.smithi067.stdout:client5692 2024-02-26T10:42:12.729 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:12.729 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:12.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:42:13.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.008+0000 7fb8bee00640 1 Processor -- start 2024-02-26T10:42:13.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.008+0000 7fb8bee00640 1 -- start start 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.010+0000 7fb8bee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8b80c1c10 0x7fb8b81557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.010+0000 7fb8bee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806e180 0x7fb8b806e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.010+0000 7fb8bee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b8155cf0 con 0x7fb8b80c0850 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.010+0000 7fb8bee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b806dca0 con 0x7fb8b80c1c10 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.010+0000 7fb8bee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80c2a70 con 0x7fb8b806e180 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8b80c1c10 0x7fb8b81557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb8b80c1c10 0x7fb8b81557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50564/0 (socket says 172.21.15.67:50564) 2024-02-26T10:42:13.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 -- 172.21.15.67:0/1721335640 learned_addr learned my addr 172.21.15.67:0/1721335640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:13.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8be400640 1 --2- 172.21.15.67:0/1721335640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806e180 0x7fb8b806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:13.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 -- 172.21.15.67:0/1721335640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806e180 msgr2=0x7fb8b806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:13.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 --2- 172.21.15.67:0/1721335640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806e180 0x7fb8b806e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 -- 172.21.15.67:0/1721335640 >> v1:172.21.15.82:6789/0 conn(0x7fb8b80c0850 legacy=0x7fb8b806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:13.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.011+0000 7fb8bda00640 1 -- 172.21.15.67:0/1721335640 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80c21d0 con 0x7fb8b80c1c10 2024-02-26T10:42:13.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.013+0000 7fb8bda00640 1 --2- 172.21.15.67:0/1721335640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b80c1c10 0x7fb8b81557b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009870 tx=0x7fb8ac02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a544628fc436a2b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:13.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.013+0000 7fb8b7e00640 1 -- 172.21.15.67:0/1721335640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0317c0 con 0x7fb8b80c1c10 2024-02-26T10:42:13.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.013+0000 7fb8b7e00640 1 -- 172.21.15.67:0/1721335640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac037070 con 0x7fb8b80c1c10 2024-02-26T10:42:13.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.013+0000 7fb8b7e00640 1 -- 172.21.15.67:0/1721335640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac031b80 con 0x7fb8b80c1c10 2024-02-26T10:42:13.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 -- 172.21.15.67:0/1721335640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b80c1c10 msgr2=0x7fb8b81557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:13.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 --2- 172.21.15.67:0/1721335640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b80c1c10 0x7fb8b81557b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009870 tx=0x7fb8ac02b860 comp rx=0 tx=0).stop 2024-02-26T10:42:13.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 -- 172.21.15.67:0/1721335640 shutdown_connections 2024-02-26T10:42:13.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 --2- 172.21.15.67:0/1721335640 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b806e180 0x7fb8b806e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 --2- 172.21.15.67:0/1721335640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b80c1c10 0x7fb8b81557b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 -- 172.21.15.67:0/1721335640 >> 172.21.15.67:0/1721335640 conn(0x7fb8b80c63f0 msgr2=0x7fb8b80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:13.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 -- 172.21.15.67:0/1721335640 shutdown_connections 2024-02-26T10:42:13.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.014+0000 7fb8bee00640 1 -- 172.21.15.67:0/1721335640 wait complete. 2024-02-26T10:42:13.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.016+0000 7fb8bee00640 1 Processor -- start 2024-02-26T10:42:13.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.016+0000 7fb8bee00640 1 -- start start 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 0x7fb8b81eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b80c0850 0x7fb8b814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b80c1c10 0x7fb8b81581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b806b330 con 0x7fb8b806e180 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80c2790 con 0x7fb8b80c0850 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b806b4b0 con 0x7fb8b80c1c10 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.017+0000 7fb8bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 0x7fb8b81eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 0x7fb8b81eb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50566/0 (socket says 172.21.15.67:50566) 2024-02-26T10:42:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 -- 172.21.15.67:0/669887143 learned_addr learned my addr 172.21.15.67:0/669887143 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bd000640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b80c0850 0x7fb8b814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 -- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b80c1c10 msgr2=0x7fb8b81581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b80c1c10 0x7fb8b81581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 -- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b80c0850 msgr2=0x7fb8b814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b80c0850 0x7fb8b814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 -- 172.21.15.67:0/669887143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b81561f0 con 0x7fb8b806e180 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.018+0000 7fb8bda00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 0x7fb8b81eb560 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac000c00 tx=0x7fb8ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.019+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac032860 con 0x7fb8b806e180 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.019+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac032a20 con 0x7fb8b806e180 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.019+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac032e10 con 0x7fb8b806e180 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.019+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b8157910 con 0x7fb8b806e180 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.019+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b8157ed0 con 0x7fb8b806e180 2024-02-26T10:42:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.020+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b8074a30 con 0x7fb8b806e180 2024-02-26T10:42:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.021+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8ac04c060 con 0x7fb8b806e180 2024-02-26T10:42:13.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.022+0000 7fb8b5600640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:13.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.023+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8ac0aba00 con 0x7fb8b806e180 2024-02-26T10:42:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.023+0000 7fb8bd000640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:13.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.024+0000 7fb8bd000640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b4d0 0x7fb89806d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80c38c0 tx=0x7fb8a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.027+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8ac040070 con 0x7fb8b806e180 2024-02-26T10:42:13.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.264+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb8b80bfca0 con 0x7fb8b806e180 2024-02-26T10:42:13.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.265+0000 7fb8b5600640 1 -- 172.21.15.67:0/669887143 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb8ac04c350 con 0x7fb8b806e180 2024-02-26T10:42:13.264 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b4d0 msgr2=0x7fb89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b4d0 0x7fb89806d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80c38c0 tx=0x7fb8a4009040 comp rx=0 tx=0).stop 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 msgr2=0x7fb8b81eb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 0x7fb8b81eb560 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac000c00 tx=0x7fb8ac0046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 shutdown_connections 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8b80c1c10 0x7fb8b81581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8b80c0850 0x7fb8b814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 --2- 172.21.15.67:0/669887143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8b806e180 0x7fb8b81eb560 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 >> 172.21.15.67:0/669887143 conn(0x7fb8b80c63f0 msgr2=0x7fb8b80c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:13.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.268+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 shutdown_connections 2024-02-26T10:42:13.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:13.269+0000 7fb8bee00640 1 -- 172.21.15.67:0/669887143 wait complete. 2024-02-26T10:42:13.287 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:42:13.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:42:13.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:13.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.288 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:42:13.346 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:42:13.347 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:13.348 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:42:13.348 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:13.348 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:42:13.348 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:42:13.348 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:42:13.348 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:13.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.349 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:42:13.435 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:42:13.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.439 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:42:13.489 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:42:13.489 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:42:13.489 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:42:13.489 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:42:13.489 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:42:13.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.490 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:42:13.490 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:42:13.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:42:13.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:42:13.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:42:13.490 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:42:13.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.666 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:42:13.712 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:42:13.713 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:13.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.713 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:42:13.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:13.793 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:13.840 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:13.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:13.841 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:13.937 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-02-26T10:42:13.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:13.937 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:14.079 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:14.079 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:14.116 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:14.116 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:14.116 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:42:14.116 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:42:14.116 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:42:14.119 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:42:14.119 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:42:14.119 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:42:14.119 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:42:14.119 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:42:14.124 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:42:14.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:14.125 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:14.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:14.210 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:42:14.271 INFO:teuthology.orchestra.run.smithi082.stdout:client6636 2024-02-26T10:42:14.272 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:14.272 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:14.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-02-26T10:42:14.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.560+0000 7efc94400640 1 Processor -- start 2024-02-26T10:42:14.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.561+0000 7efc94400640 1 -- start start 2024-02-26T10:42:14.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.562+0000 7efc94400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ea30 0x7efc8c06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:14.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.562+0000 7efc94400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc8c0c1070 0x7efc8c072e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:14.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.562+0000 7efc94400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c06b100 con 0x7efc8c06ac20 2024-02-26T10:42:14.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.562+0000 7efc94400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8c151670 con 0x7efc8c0c1070 2024-02-26T10:42:14.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.562+0000 7efc94400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c06b3e0 con 0x7efc8c06ea30 2024-02-26T10:42:14.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc8c0c1070 0x7efc8c072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:14.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc8c0c1070 0x7efc8c072e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50580/0 (socket says 172.21.15.67:50580) 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc92400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efc8c06ac20 0x7efc8c151b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34690/0 (socket says 172.21.15.67:34690) 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 -- 172.21.15.67:0/2145388269 learned_addr learned my addr 172.21.15.67:0/2145388269 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91a00640 1 --2- 172.21.15.67:0/2145388269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ea30 0x7efc8c06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 -- 172.21.15.67:0/2145388269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ea30 msgr2=0x7efc8c06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 --2- 172.21.15.67:0/2145388269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ea30 0x7efc8c06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 -- 172.21.15.67:0/2145388269 >> v1:172.21.15.67:6789/0 conn(0x7efc8c06ac20 legacy=0x7efc8c151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:14.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.563+0000 7efc91000640 1 -- 172.21.15.67:0/2145388269 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c06b6c0 con 0x7efc8c0c1070 2024-02-26T10:42:14.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.564+0000 7efc91000640 1 --2- 172.21.15.67:0/2145388269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c0c1070 0x7efc8c072e60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efc78009870 tx=0x7efc7802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe3cfad8b23eaea server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:14.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.565+0000 7efc8be00640 1 -- 172.21.15.67:0/2145388269 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78037070 con 0x7efc8c0c1070 2024-02-26T10:42:14.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.565+0000 7efc8be00640 1 -- 172.21.15.67:0/2145388269 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc78030710 con 0x7efc8c0c1070 2024-02-26T10:42:14.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.565+0000 7efc8be00640 1 -- 172.21.15.67:0/2145388269 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78030ae0 con 0x7efc8c0c1070 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.565+0000 7efc94400640 1 -- 172.21.15.67:0/2145388269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c0c1070 msgr2=0x7efc8c072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.565+0000 7efc94400640 1 --2- 172.21.15.67:0/2145388269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c0c1070 0x7efc8c072e60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efc78009870 tx=0x7efc7802b7c0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.565+0000 7efc94400640 1 -- 172.21.15.67:0/2145388269 shutdown_connections 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.566+0000 7efc94400640 1 --2- 172.21.15.67:0/2145388269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c0c1070 0x7efc8c072e60 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.566+0000 7efc94400640 1 --2- 172.21.15.67:0/2145388269 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ea30 0x7efc8c06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.566+0000 7efc94400640 1 -- 172.21.15.67:0/2145388269 >> 172.21.15.67:0/2145388269 conn(0x7efc8c0a84a0 msgr2=0x7efc8c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.566+0000 7efc94400640 1 -- 172.21.15.67:0/2145388269 shutdown_connections 2024-02-26T10:42:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.566+0000 7efc94400640 1 -- 172.21.15.67:0/2145388269 wait complete. 2024-02-26T10:42:14.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.567+0000 7efc94400640 1 Processor -- start 2024-02-26T10:42:14.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 -- start start 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ac20 0x7efc8c147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 0x7efc8c0bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc8c1dbb10 0x7efc8c1ea880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0c3100 con 0x7efc8c06ea30 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c151290 con 0x7efc8c1dbb10 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.568+0000 7efc94400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0c3280 con 0x7efc8c06ac20 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 0x7efc8c0bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 0x7efc8c0bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50588/0 (socket says 172.21.15.67:50588) 2024-02-26T10:42:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 -- 172.21.15.67:0/1926528598 learned_addr learned my addr 172.21.15.67:0/1926528598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc92400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc8c1dbb10 0x7efc8c1ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91a00640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ac20 0x7efc8c147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 -- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ac20 msgr2=0x7efc8c147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ac20 0x7efc8c147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 -- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc8c1dbb10 msgr2=0x7efc8c1ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc8c1dbb10 0x7efc8c1ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc91000640 1 -- 172.21.15.67:0/1926528598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c1ede40 con 0x7efc8c06ea30 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.569+0000 7efc92400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc8c1dbb10 0x7efc8c1ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.570+0000 7efc91000640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 0x7efc8c0bfa30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efc78000c00 tx=0x7efc780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.570+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78041020 con 0x7efc8c06ea30 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.570+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7803c950 con 0x7efc8c06ea30 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.570+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78037040 con 0x7efc8c06ea30 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.570+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c1ee0e0 con 0x7efc8c06ea30 2024-02-26T10:42:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.570+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c1ee6a0 con 0x7efc8c06ea30 2024-02-26T10:42:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.571+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c074a30 con 0x7efc8c06ea30 2024-02-26T10:42:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.572+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc78044020 con 0x7efc8c06ea30 2024-02-26T10:42:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.573+0000 7efc89600640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc6c06b4d0 0x7efc6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.574+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc780ab940 con 0x7efc8c06ea30 2024-02-26T10:42:14.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.574+0000 7efc91a00640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc6c06b4d0 0x7efc6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:14.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.575+0000 7efc91a00640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc6c06b4d0 0x7efc6c06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009f30 tx=0x7efc7c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.578+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc7804b070 con 0x7efc8c06ea30 2024-02-26T10:42:14.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.849+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7efc8c0ab710 con 0x7efc8c06ea30 2024-02-26T10:42:14.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.850+0000 7efc89600640 1 -- 172.21.15.67:0/1926528598 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v17) v1 ==== 84+0+0 (secure 0 0 0) 0x7efc7807a7b0 con 0x7efc8c06ea30 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc6c06b4d0 msgr2=0x7efc6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc6c06b4d0 0x7efc6c06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009f30 tx=0x7efc7c00b040 comp rx=0 tx=0).stop 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 msgr2=0x7efc8c0bfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 0x7efc8c0bfa30 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efc78000c00 tx=0x7efc780046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 shutdown_connections 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc6c06b4d0 0x7efc6c06d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc8c1dbb10 0x7efc8c1ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc8c06ea30 0x7efc8c0bfa30 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 --2- 172.21.15.67:0/1926528598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc8c06ac20 0x7efc8c147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:14.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.852+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 >> 172.21.15.67:0/1926528598 conn(0x7efc8c0a84a0 msgr2=0x7efc8c151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:14.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.853+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 shutdown_connections 2024-02-26T10:42:14.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:14.853+0000 7efc94400640 1 -- 172.21.15.67:0/1926528598 wait complete. 2024-02-26T10:42:14.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:15.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.098+0000 7f268ac00640 1 Processor -- start 2024-02-26T10:42:15.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.099+0000 7f268ac00640 1 -- start start 2024-02-26T10:42:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.100+0000 7f268ac00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f268406b000 0x7f268406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.100+0000 7f268ac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26840a9a80 con 0x7f268406ac20 2024-02-26T10:42:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.100+0000 7f268ac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26840a9d60 con 0x7f26840a94a0 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.100+0000 7f268ac00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268406b470 con 0x7f268406b000 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f2689800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f268406ac20 0x7f2684153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34692/0 (socket says 172.21.15.67:34692) 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f2689800640 1 -- 172.21.15.67:0/1384557490 learned_addr learned my addr 172.21.15.67:0/1384557490 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f268a200640 1 --2- 172.21.15.67:0/1384557490 >> v2:172.21.15.67:3300/0 conn(0x7f268406b000 0x7f268406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f2683e00640 1 -- 172.21.15.67:0/1384557490 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208981052 0 0) 0x7f26840a9a80 con 0x7f268406ac20 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f2683e00640 1 -- 172.21.15.67:0/1384557490 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f266c003520 con 0x7f268406ac20 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f268a200640 1 -- 172.21.15.67:0/1384557490 >> v1:172.21.15.82:6789/0 conn(0x7f26840a94a0 legacy=0x7f26840abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f268a200640 1 -- 172.21.15.67:0/1384557490 >> v1:172.21.15.67:6789/0 conn(0x7f268406ac20 legacy=0x7f2684153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.101+0000 7f268a200640 1 -- 172.21.15.67:0/1384557490 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840ab750 con 0x7f268406b000 2024-02-26T10:42:15.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.103+0000 7f268a200640 1 --2- 172.21.15.67:0/1384557490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f268406db00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f267c009d90 tx=0x7f267c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69a72c6d164d19d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.103+0000 7f2683e00640 1 -- 172.21.15.67:0/1384557490 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c039070 con 0x7f268406b000 2024-02-26T10:42:15.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.103+0000 7f2683e00640 1 -- 172.21.15.67:0/1384557490 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c02cc60 con 0x7f268406b000 2024-02-26T10:42:15.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.103+0000 7f2683e00640 1 -- 172.21.15.67:0/1384557490 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c034660 con 0x7f268406b000 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 -- 172.21.15.67:0/1384557490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 msgr2=0x7f268406db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 --2- 172.21.15.67:0/1384557490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f268406db00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f267c009d90 tx=0x7f267c02beb0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 -- 172.21.15.67:0/1384557490 shutdown_connections 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 --2- 172.21.15.67:0/1384557490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f268406db00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 -- 172.21.15.67:0/1384557490 >> 172.21.15.67:0/1384557490 conn(0x7f26840c63f0 msgr2=0x7f26840c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 -- 172.21.15.67:0/1384557490 shutdown_connections 2024-02-26T10:42:15.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.104+0000 7f268ac00640 1 -- 172.21.15.67:0/1384557490 wait complete. 2024-02-26T10:42:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.106+0000 7f268ac00640 1 Processor -- start 2024-02-26T10:42:15.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.106+0000 7f268ac00640 1 -- start start 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.107+0000 7f268ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f268406ac20 0x7f26841ec590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.107+0000 7f268ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f26840ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.107+0000 7f268ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26840a94a0 0x7f26841e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.107+0000 7f268ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684150590 con 0x7f268406b000 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.107+0000 7f268ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684150410 con 0x7f26840a94a0 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.107+0000 7f268ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268415e520 con 0x7f268406ac20 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f26840ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f26840ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50606/0 (socket says 172.21.15.67:50606) 2024-02-26T10:42:15.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 -- 172.21.15.67:0/1659883051 learned_addr learned my addr 172.21.15.67:0/1659883051 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2689800640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f268406ac20 0x7f26841ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f268a200640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26840a94a0 0x7f26841e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 -- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f268406ac20 msgr2=0x7f26841ec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f268406ac20 0x7f26841ec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 -- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26840a94a0 msgr2=0x7f26841e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26840a94a0 0x7f26841e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f2688e00640 1 -- 172.21.15.67:0/1659883051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268414bca0 con 0x7f268406b000 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.108+0000 7f268a200640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26840a94a0 0x7f26841e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.109+0000 7f2688e00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f26840ac1c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f267000b3f0 tx=0x7f267000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.109+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670015020 con 0x7f268406b000 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.109+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670011830 con 0x7f268406b000 2024-02-26T10:42:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.109+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670011c20 con 0x7f268406b000 2024-02-26T10:42:15.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.109+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2684153d50 con 0x7f268406b000 2024-02-26T10:42:15.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.109+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684154310 con 0x7f268406b000 2024-02-26T10:42:15.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.110+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2684074a30 con 0x7f268406b000 2024-02-26T10:42:15.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.111+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2670018020 con 0x7f268406b000 2024-02-26T10:42:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.112+0000 7f2681600640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.112+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2670089ea0 con 0x7f268406b000 2024-02-26T10:42:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.113+0000 7f2689800640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.114+0000 7f2689800640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2678007fd0 tx=0x7f2678009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.117+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f267001f030 con 0x7f268406b000 2024-02-26T10:42:15.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.324+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26840c0df0 con 0x7f268406b000 2024-02-26T10:42:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.326+0000 7f2681600640 1 -- 172.21.15.67:0/1659883051 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16588 (secure 0 0 0) 0x7f26700172c0 con 0x7f268406b000 2024-02-26T10:42:15.325 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:15.325 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":76,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:00.271058+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 msgr2=0x7f266406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2678007fd0 tx=0x7f2678009040 comp rx=0 tx=0).stop 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 msgr2=0x7f26840ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f26840ac1c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f267000b3f0 tx=0x7f267000b8b0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 shutdown_connections 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26840a94a0 0x7f26841e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268406b000 0x7f26840ac1c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 --2- 172.21.15.67:0/1659883051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f268406ac20 0x7f26841ec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 >> 172.21.15.67:0/1659883051 conn(0x7f26840c63f0 msgr2=0x7f2684152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.329+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 shutdown_connections 2024-02-26T10:42:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.330+0000 7f268ac00640 1 -- 172.21.15.67:0/1659883051 wait complete. 2024-02-26T10:42:15.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:15.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:15.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.573+0000 7f851e600640 1 Processor -- start 2024-02-26T10:42:15.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.574+0000 7f851e600640 1 -- start start 2024-02-26T10:42:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.575+0000 7f851e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85180c0ef0 0x7f8518158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.575+0000 7f851e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8518072ce0 con 0x7f85180c1400 2024-02-26T10:42:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.575+0000 7f851e600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8518155bf0 con 0x7f85181591b0 2024-02-26T10:42:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.575+0000 7f851e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8518073010 con 0x7f85180c0ef0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.575+0000 7f8516a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85180c0ef0 0x7f8518158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.576+0000 7f8516a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85180c0ef0 0x7f8518158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50622/0 (socket says 172.21.15.67:50622) 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.576+0000 7f8516a00640 1 -- 172.21.15.67:0/1037431231 learned_addr learned my addr 172.21.15.67:0/1037431231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.576+0000 7f8516a00640 1 -- 172.21.15.67:0/1037431231 >> v1:172.21.15.82:6789/0 conn(0x7f85181591b0 legacy=0x7f8518070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.576+0000 7f8516a00640 1 -- 172.21.15.67:0/1037431231 >> v1:172.21.15.67:6789/0 conn(0x7f85180c1400 legacy=0x7f851806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.576+0000 7f8516a00640 1 -- 172.21.15.67:0/1037431231 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518155ed0 con 0x7f85180c0ef0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.576+0000 7f8516000640 1 -- 172.21.15.67:0/1037431231 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414480391 0 0) 0x7f8518072ce0 con 0x7f85180c1400 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.577+0000 7f8516a00640 1 --2- 172.21.15.67:0/1037431231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85180c0ef0 0x7f8518158580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8504009870 tx=0x7f850402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75dc2169b6efafe server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.578+0000 7f8516000640 1 -- 172.21.15.67:0/1037431231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504035020 con 0x7f85180c0ef0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.578+0000 7f8516000640 1 -- 172.21.15.67:0/1037431231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8504031870 con 0x7f85180c0ef0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.578+0000 7f8516000640 1 -- 172.21.15.67:0/1037431231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504031c40 con 0x7f85180c0ef0 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.578+0000 7f851e600640 1 -- 172.21.15.67:0/1037431231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85180c0ef0 msgr2=0x7f8518158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.578+0000 7f851e600640 1 --2- 172.21.15.67:0/1037431231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85180c0ef0 0x7f8518158580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8504009870 tx=0x7f850402b910 comp rx=0 tx=0).stop 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.579+0000 7f851e600640 1 -- 172.21.15.67:0/1037431231 shutdown_connections 2024-02-26T10:42:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.579+0000 7f851e600640 1 --2- 172.21.15.67:0/1037431231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85180c0ef0 0x7f8518158580 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.579+0000 7f851e600640 1 -- 172.21.15.67:0/1037431231 >> 172.21.15.67:0/1037431231 conn(0x7f85180c6da0 msgr2=0x7f85180c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:15.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.579+0000 7f851e600640 1 -- 172.21.15.67:0/1037431231 shutdown_connections 2024-02-26T10:42:15.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.579+0000 7f851e600640 1 -- 172.21.15.67:0/1037431231 wait complete. 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.580+0000 7f851e600640 1 Processor -- start 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.580+0000 7f851e600640 1 -- start start 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.581+0000 7f851e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85180c0ef0 0x7f851814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.581+0000 7f851e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85180c1400 0x7f85181ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.581+0000 7f851e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 0x7f8518157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.581+0000 7f851e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851806e560 con 0x7f85181591b0 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.581+0000 7f851e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851814f490 con 0x7f85180c1400 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.581+0000 7f851e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851806e6e0 con 0x7f85180c0ef0 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 0x7f8518157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 0x7f8518157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50628/0 (socket says 172.21.15.67:50628) 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 -- 172.21.15.67:0/2610492521 learned_addr learned my addr 172.21.15.67:0/2610492521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:15.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517400640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85180c0ef0 0x7f851814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 -- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85180c0ef0 msgr2=0x7f851814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85180c0ef0 0x7f851814b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 -- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85180c1400 msgr2=0x7f85181ebae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85180c1400 0x7f85181ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.582+0000 7f8517e00640 1 -- 172.21.15.67:0/2610492521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180ac2e0 con 0x7f85181591b0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.583+0000 7f8517e00640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 0x7f8518157310 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f851000c760 tx=0x7f851000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.583+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510003b90 con 0x7f85181591b0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.583+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510003d50 con 0x7f85181591b0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.583+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000aa90 con 0x7f85181591b0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.583+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851814bb20 con 0x7f85181591b0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.583+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518067e00 con 0x7f85181591b0 2024-02-26T10:42:15.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.584+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518074a30 con 0x7f85181591b0 2024-02-26T10:42:15.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.585+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f851000ac30 con 0x7f85181591b0 2024-02-26T10:42:15.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.586+0000 7f850f400640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f84f806b4d0 0x7f84f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:15.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.586+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f851008a160 con 0x7f85181591b0 2024-02-26T10:42:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.587+0000 7f8517400640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f84f806b4d0 0x7f84f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:15.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.588+0000 7f8517400640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f84f806b4d0 0x7f84f806d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8508006fd0 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.591+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8510058fd0 con 0x7f85181591b0 2024-02-26T10:42:15.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.787+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f851806eb10 con 0x7f85181591b0 2024-02-26T10:42:15.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.787+0000 7f850f400640 1 -- 172.21.15.67:0/2610492521 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85100588d0 con 0x7f85181591b0 2024-02-26T10:42:15.786 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f84f806b4d0 msgr2=0x7f84f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f84f806b4d0 0x7f84f806d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8508006fd0 tx=0x7f8508008040 comp rx=0 tx=0).stop 2024-02-26T10:42:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 msgr2=0x7f8518157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 0x7f8518157310 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f851000c760 tx=0x7f851000cc20 comp rx=0 tx=0).stop 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 shutdown_connections 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85181591b0 0x7f8518157310 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f84f806b4d0 0x7f84f806d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85180c1400 0x7f85181ebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 --2- 172.21.15.67:0/2610492521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85180c0ef0 0x7f851814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 >> 172.21.15.67:0/2610492521 conn(0x7f85180c6da0 msgr2=0x7f851806de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 shutdown_connections 2024-02-26T10:42:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:15.790+0000 7f851e600640 1 -- 172.21.15.67:0/2610492521 wait complete. 2024-02-26T10:42:15.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.030+0000 7f7a36400640 1 Processor -- start 2024-02-26T10:42:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.031+0000 7f7a36400640 1 -- start start 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a36400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a300c5290 0x7f7a301562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a36400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a30155880 0x7f7a3006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a36400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a3006ce30 0x7f7a300af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a36400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3006c110 con 0x7f7a3006ce30 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a36400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300c72c0 con 0x7f7a30155880 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a36400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300c75a0 con 0x7f7a300c5290 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a35a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a3006ce30 0x7f7a300af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a35a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a3006ce30 0x7f7a300af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50638/0 (socket says 172.21.15.67:50638) 2024-02-26T10:42:16.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a35a00640 1 -- 172.21.15.67:0/3372755905 learned_addr learned my addr 172.21.15.67:0/3372755905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a2fe00640 1 --2- 172.21.15.67:0/3372755905 >> v2:172.21.15.82:3300/0 conn(0x7f7a30155880 0x7f7a3006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.032+0000 7f7a35000640 1 --2- 172.21.15.67:0/3372755905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a300c5290 0x7f7a301562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.033+0000 7f7a35a00640 1 -- 172.21.15.67:0/3372755905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a300c5290 msgr2=0x7f7a301562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.033+0000 7f7a35a00640 1 --2- 172.21.15.67:0/3372755905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a300c5290 0x7f7a301562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.033+0000 7f7a35a00640 1 -- 172.21.15.67:0/3372755905 >> v2:172.21.15.82:3300/0 conn(0x7f7a30155880 msgr2=0x7f7a3006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.033+0000 7f7a35a00640 1 --2- 172.21.15.67:0/3372755905 >> v2:172.21.15.82:3300/0 conn(0x7f7a30155880 0x7f7a3006c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.033+0000 7f7a35a00640 1 -- 172.21.15.67:0/3372755905 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300c7880 con 0x7f7a3006ce30 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.033+0000 7f7a2fe00640 1 --2- 172.21.15.67:0/3372755905 >> v2:172.21.15.82:3300/0 conn(0x7f7a30155880 0x7f7a3006c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.034+0000 7f7a35a00640 1 --2- 172.21.15.67:0/3372755905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3006ce30 0x7f7a300af8a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7a300cb7a0 tx=0x7f7a2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd1200b97da2927 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.034+0000 7f7a2f400640 1 -- 172.21.15.67:0/3372755905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2802e070 con 0x7f7a3006ce30 2024-02-26T10:42:16.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.034+0000 7f7a2f400640 1 -- 172.21.15.67:0/3372755905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a28037070 con 0x7f7a3006ce30 2024-02-26T10:42:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.034+0000 7f7a2f400640 1 -- 172.21.15.67:0/3372755905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28031a00 con 0x7f7a3006ce30 2024-02-26T10:42:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 -- 172.21.15.67:0/3372755905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3006ce30 msgr2=0x7f7a300af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 --2- 172.21.15.67:0/3372755905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3006ce30 0x7f7a300af8a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7a300cb7a0 tx=0x7f7a2802b880 comp rx=0 tx=0).stop 2024-02-26T10:42:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 -- 172.21.15.67:0/3372755905 shutdown_connections 2024-02-26T10:42:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 --2- 172.21.15.67:0/3372755905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3006ce30 0x7f7a300af8a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 --2- 172.21.15.67:0/3372755905 >> v2:172.21.15.82:3300/0 conn(0x7f7a30155880 0x7f7a3006c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 --2- 172.21.15.67:0/3372755905 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a300c5290 0x7f7a301562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 -- 172.21.15.67:0/3372755905 >> 172.21.15.67:0/3372755905 conn(0x7f7a300ac230 msgr2=0x7f7a300ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 -- 172.21.15.67:0/3372755905 shutdown_connections 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.035+0000 7f7a36400640 1 -- 172.21.15.67:0/3372755905 wait complete. 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.036+0000 7f7a36400640 1 Processor -- start 2024-02-26T10:42:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 -- start start 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a300c5290 0x7f7a300c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a30155880 0x7f7a300c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 0x7f7a301e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300c5670 con 0x7f7a301e7720 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30155cc0 con 0x7f7a300c5290 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.037+0000 7f7a36400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300c57f0 con 0x7f7a30155880 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 0x7f7a301e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 0x7f7a301e7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50644/0 (socket says 172.21.15.67:50644) 2024-02-26T10:42:16.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 -- 172.21.15.67:0/1608667001 learned_addr learned my addr 172.21.15.67:0/1608667001 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a2fe00640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a30155880 0x7f7a300c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35000640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a300c5290 0x7f7a300c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 -- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a30155880 msgr2=0x7f7a300c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a30155880 0x7f7a300c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 -- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a300c5290 msgr2=0x7f7a300c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a300c5290 0x7f7a300c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.038+0000 7f7a35a00640 1 -- 172.21.15.67:0/1608667001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a301e8110 con 0x7f7a301e7720 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.039+0000 7f7a35a00640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 0x7f7a301e7bd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a280090b0 tx=0x7f7a280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.039+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28032670 con 0x7f7a301e7720 2024-02-26T10:42:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.039+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a28032830 con 0x7f7a301e7720 2024-02-26T10:42:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.039+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28032c20 con 0x7f7a301e7720 2024-02-26T10:42:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.039+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a301ed9f0 con 0x7f7a301e7720 2024-02-26T10:42:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.039+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a301edfb0 con 0x7f7a301e7720 2024-02-26T10:42:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.040+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30074a30 con 0x7f7a301e7720 2024-02-26T10:42:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.041+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a280453f0 con 0x7f7a301e7720 2024-02-26T10:42:16.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.042+0000 7f7a2cc00640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.042+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a280ac100 con 0x7f7a301e7720 2024-02-26T10:42:16.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.043+0000 7f7a35000640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.044+0000 7f7a35000640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7a20004450 tx=0x7f7a20003a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.047+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a301e8110 con 0x7f7a301e7720 2024-02-26T10:42:16.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.246+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a301591d0 con 0x7f7a301e7720 2024-02-26T10:42:16.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.247+0000 7f7a2cc00640 1 -- 172.21.15.67:0/1608667001 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a28040070 con 0x7f7a301e7720 2024-02-26T10:42:16.245 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.249+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7a0c06b4d0 msgr2=0x7f7a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.249+0000 7f7a36400640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7a20004450 tx=0x7f7a20003a30 comp rx=0 tx=0).stop 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 msgr2=0x7f7a301e7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 0x7f7a301e7bd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a280090b0 tx=0x7f7a280046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 shutdown_connections 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a301e7720 0x7f7a301e7bd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a30155880 0x7f7a300c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 --2- 172.21.15.67:0/1608667001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a300c5290 0x7f7a300c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 >> 172.21.15.67:0/1608667001 conn(0x7f7a300ac230 msgr2=0x7f7a30156110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:16.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 shutdown_connections 2024-02-26T10:42:16.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.250+0000 7f7a36400640 1 -- 172.21.15.67:0/1608667001 wait complete. 2024-02-26T10:42:16.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.502+0000 7f1df6800640 1 Processor -- start 2024-02-26T10:42:16.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.502+0000 7f1df6800640 1 -- start start 2024-02-26T10:42:16.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1df006ec60 0x7f1df00c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df6800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df00c3370 0x7f1df0070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df6800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df00c4930 con 0x7f1df0150250 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df6800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df00c46e0 con 0x7f1df006ec60 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df6800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00c4c10 con 0x7f1df00c3370 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1df006ec60 0x7f1df00c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1df006ec60 0x7f1df00c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50660/0 (socket says 172.21.15.67:50660) 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.504+0000 7f1df4a00640 1 -- 172.21.15.67:0/218621922 learned_addr learned my addr 172.21.15.67:0/218621922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.505+0000 7f1df5e00640 1 --2- 172.21.15.67:0/218621922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df00c3370 0x7f1df0070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.505+0000 7f1df4a00640 1 -- 172.21.15.67:0/218621922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df00c3370 msgr2=0x7f1df0070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.505+0000 7f1df4a00640 1 --2- 172.21.15.67:0/218621922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df00c3370 0x7f1df0070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.505+0000 7f1df4a00640 1 -- 172.21.15.67:0/218621922 >> v1:172.21.15.82:6789/0 conn(0x7f1df0150250 legacy=0x7f1df0156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.505+0000 7f1df4a00640 1 -- 172.21.15.67:0/218621922 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00c6440 con 0x7f1df006ec60 2024-02-26T10:42:16.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.506+0000 7f1df4a00640 1 --2- 172.21.15.67:0/218621922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df006ec60 0x7f1df00c6f20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009870 tx=0x7f1ddc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fb567cd9e9aeba server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.506+0000 7f1defe00640 1 -- 172.21.15.67:0/218621922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc035020 con 0x7f1df006ec60 2024-02-26T10:42:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.506+0000 7f1defe00640 1 -- 172.21.15.67:0/218621922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc031640 con 0x7f1df006ec60 2024-02-26T10:42:16.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.506+0000 7f1defe00640 1 -- 172.21.15.67:0/218621922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc02d040 con 0x7f1df006ec60 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 -- 172.21.15.67:0/218621922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df006ec60 msgr2=0x7f1df00c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 --2- 172.21.15.67:0/218621922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df006ec60 0x7f1df00c6f20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009870 tx=0x7f1ddc02b8f0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 -- 172.21.15.67:0/218621922 shutdown_connections 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 --2- 172.21.15.67:0/218621922 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df00c3370 0x7f1df0070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 --2- 172.21.15.67:0/218621922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df006ec60 0x7f1df00c6f20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 -- 172.21.15.67:0/218621922 >> 172.21.15.67:0/218621922 conn(0x7f1df00aa160 msgr2=0x7f1df00aa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 -- 172.21.15.67:0/218621922 shutdown_connections 2024-02-26T10:42:16.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.507+0000 7f1df6800640 1 -- 172.21.15.67:0/218621922 wait complete. 2024-02-26T10:42:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.509+0000 7f1df6800640 1 Processor -- start 2024-02-26T10:42:16.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.509+0000 7f1df6800640 1 -- start start 2024-02-26T10:42:16.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df6800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df006ec60 0x7f1df01eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df00c3370 0x7f1df00ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 0x7f1df0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0150630 con 0x7f1df0150250 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df6800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00c6a00 con 0x7f1df00c3370 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df6800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01507b0 con 0x7f1df006ec60 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 0x7f1df0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 0x7f1df0154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50676/0 (socket says 172.21.15.67:50676) 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.510+0000 7f1df5e00640 1 -- 172.21.15.67:0/2792795567 learned_addr learned my addr 172.21.15.67:0/2792795567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df4a00640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df00c3370 0x7f1df00ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5400640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df006ec60 0x7f1df01eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5e00640 1 -- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df006ec60 msgr2=0x7f1df01eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5e00640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df006ec60 0x7f1df01eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5e00640 1 -- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df00c3370 msgr2=0x7f1df00ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5e00640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df00c3370 0x7f1df00ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5e00640 1 -- 172.21.15.67:0/2792795567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0155fc0 con 0x7f1df0150250 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df5e00640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 0x7f1df0154be0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007e70 tx=0x7f1de800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8014620 con 0x7f1df0150250 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8019070 con 0x7f1df0150250 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8014a20 con 0x7f1df0150250 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.511+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df0154100 con 0x7f1df0150250 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.512+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df01546c0 con 0x7f1df0150250 2024-02-26T10:42:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.513+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df0074a30 con 0x7f1df0150250 2024-02-26T10:42:16.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.514+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1de8014bc0 con 0x7f1df0150250 2024-02-26T10:42:16.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.515+0000 7f1ded600640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1dd006b4d0 0x7f1dd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.516+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1de808aac0 con 0x7f1df0150250 2024-02-26T10:42:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.516+0000 7f1df5400640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1dd006b4d0 0x7f1dd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.517+0000 7f1df5400640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1dd006b4d0 0x7f1dd006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000f60 tx=0x7f1de4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.520+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1de8059930 con 0x7f1df0150250 2024-02-26T10:42:16.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.717+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1df014f3c0 con 0x7f1df0150250 2024-02-26T10:42:16.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.718+0000 7f1ded600640 1 -- 172.21.15.67:0/2792795567 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1de8059230 con 0x7f1df0150250 2024-02-26T10:42:16.716 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:16.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1dd006b4d0 msgr2=0x7f1dd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1dd006b4d0 0x7f1dd006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000f60 tx=0x7f1de4009040 comp rx=0 tx=0).stop 2024-02-26T10:42:16.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 msgr2=0x7f1df0154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 0x7f1df0154be0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007e70 tx=0x7f1de800e4b0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 shutdown_connections 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1df0150250 0x7f1df0154be0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1dd006b4d0 0x7f1dd006d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1df00c3370 0x7f1df00ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 --2- 172.21.15.67:0/2792795567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1df006ec60 0x7f1df01eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 >> 172.21.15.67:0/2792795567 conn(0x7f1df00aa160 msgr2=0x7f1df006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 shutdown_connections 2024-02-26T10:42:16.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.721+0000 7f1df6800640 1 -- 172.21.15.67:0/2792795567 wait complete. 2024-02-26T10:42:16.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-26T10:42:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.955+0000 7fe9c2800640 1 Processor -- start 2024-02-26T10:42:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.956+0000 7fe9c2800640 1 -- start start 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.957+0000 7fe9c2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0c5820 0x7fe9bc155a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.957+0000 7fe9c2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9bc0c3730 0x7fe9bc158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.957+0000 7fe9c2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc155fb0 con 0x7fe9bc1591e0 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.957+0000 7fe9c2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc0c7100 con 0x7fe9bc0c3730 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.957+0000 7fe9c2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc158810 con 0x7fe9bc0c5820 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9bc0c3730 0x7fe9bc158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9bc0c3730 0x7fe9bc158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50690/0 (socket says 172.21.15.67:50690) 2024-02-26T10:42:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 -- 172.21.15.67:0/536298303 learned_addr learned my addr 172.21.15.67:0/536298303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9bb400640 1 --2- 172.21.15.67:0/536298303 >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0c5820 0x7fe9bc155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 -- 172.21.15.67:0/536298303 >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0c5820 msgr2=0x7fe9bc155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 --2- 172.21.15.67:0/536298303 >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0c5820 0x7fe9bc155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 -- 172.21.15.67:0/536298303 >> v1:172.21.15.82:6789/0 conn(0x7fe9bc1591e0 legacy=0x7fe9bc14ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:16.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.958+0000 7fe9baa00640 1 -- 172.21.15.67:0/536298303 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0c6860 con 0x7fe9bc0c3730 2024-02-26T10:42:16.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.959+0000 7fe9baa00640 1 --2- 172.21.15.67:0/536298303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc158ca0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0c85b0 tx=0x7fe9a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63cfc1002a06875 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.959+0000 7fe9ba000640 1 -- 172.21.15.67:0/536298303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4035020 con 0x7fe9bc0c3730 2024-02-26T10:42:16.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9ba000640 1 -- 172.21.15.67:0/536298303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4031870 con 0x7fe9bc0c3730 2024-02-26T10:42:16.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9ba000640 1 -- 172.21.15.67:0/536298303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4031c40 con 0x7fe9bc0c3730 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 -- 172.21.15.67:0/536298303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 msgr2=0x7fe9bc158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 --2- 172.21.15.67:0/536298303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc158ca0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0c85b0 tx=0x7fe9a402b910 comp rx=0 tx=0).stop 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 -- 172.21.15.67:0/536298303 shutdown_connections 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 --2- 172.21.15.67:0/536298303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc158ca0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 --2- 172.21.15.67:0/536298303 >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0c5820 0x7fe9bc155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 -- 172.21.15.67:0/536298303 >> 172.21.15.67:0/536298303 conn(0x7fe9bc06b040 msgr2=0x7fe9bc06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 -- 172.21.15.67:0/536298303 shutdown_connections 2024-02-26T10:42:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.960+0000 7fe9c2800640 1 -- 172.21.15.67:0/536298303 wait complete. 2024-02-26T10:42:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.962+0000 7fe9c2800640 1 Processor -- start 2024-02-26T10:42:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.962+0000 7fe9c2800640 1 -- start start 2024-02-26T10:42:16.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9c2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0c5820 0x7fe9bc149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9c2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc1591e0 0x7fe9bc1e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9c2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0c6530 con 0x7fe9bc0c3730 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9c2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0c6e20 con 0x7fe9bc0c5820 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9c2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0a7fc0 con 0x7fe9bc1591e0 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9bb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc14dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50698/0 (socket says 172.21.15.67:50698) 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.963+0000 7fe9bb400640 1 -- 172.21.15.67:0/446881398 learned_addr learned my addr 172.21.15.67:0/446881398 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bbe00640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc1591e0 0x7fe9bc1e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9baa00640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0c5820 0x7fe9bc149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bb400640 1 -- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc1591e0 msgr2=0x7fe9bc1e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bb400640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc1591e0 0x7fe9bc1e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bb400640 1 -- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0c5820 msgr2=0x7fe9bc149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bb400640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0c5820 0x7fe9bc149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bb400640 1 -- 172.21.15.67:0/446881398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc1ea990 con 0x7fe9bc0c3730 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9baa00640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0c5820 0x7fe9bc149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9bb400640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc14dbf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00ce10 tx=0x7fe9ac00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac003b90 con 0x7fe9bc0c3730 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac003d30 con 0x7fe9bc0c3730 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac004260 con 0x7fe9bc0c3730 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.964+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc1eac70 con 0x7fe9bc0c3730 2024-02-26T10:42:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.965+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc14e130 con 0x7fe9bc0c3730 2024-02-26T10:42:16.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.966+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc1eb730 con 0x7fe9bc0c3730 2024-02-26T10:42:16.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.967+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9ac004400 con 0x7fe9bc0c3730 2024-02-26T10:42:16.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.968+0000 7fe9b3400640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b500 0x7fe99806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:16.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.968+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9ac08ac80 con 0x7fe9bc0c3730 2024-02-26T10:42:16.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.968+0000 7fe9baa00640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b500 0x7fe99806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:16.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.969+0000 7fe9baa00640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b500 0x7fe99806d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4036000 tx=0x7fe9a4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:16.972+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9ac059af0 con 0x7fe9bc0c3730 2024-02-26T10:42:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.172+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe9bc074a30 con 0x7fe9bc0c3730 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.173+0000 7fe9b3400640 1 -- 172.21.15.67:0/446881398 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+209 (secure 0 0 0) 0x7fe9ac0593f0 con 0x7fe9bc0c3730 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 12, 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "cephfs_metadata" 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-02-26T10:42:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-02-26T10:42:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 13, 2024-02-26T10:42:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "cephfs_data" 2024-02-26T10:42:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-02-26T10:42:17.172 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b500 msgr2=0x7fe99806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b500 0x7fe99806d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4036000 tx=0x7fe9a4008030 comp rx=0 tx=0).stop 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 msgr2=0x7fe9bc14dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc14dbf0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00ce10 tx=0x7fe9ac00a660 comp rx=0 tx=0).stop 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 shutdown_connections 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc1591e0 0x7fe9bc1e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe99806b500 0x7fe99806d9c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0c5820 0x7fe9bc149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 --2- 172.21.15.67:0/446881398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9bc0c3730 0x7fe9bc14dbf0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.175+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 >> 172.21.15.67:0/446881398 conn(0x7fe9bc06b040 msgr2=0x7fe9bc0a92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:17.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.176+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 shutdown_connections 2024-02-26T10:42:17.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.176+0000 7fe9c2800640 1 -- 172.21.15.67:0/446881398 wait complete. 2024-02-26T10:42:17.193 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-02-26T10:42:17.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-02-26T10:42:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.418+0000 7efd4e800640 1 Processor -- start 2024-02-26T10:42:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.418+0000 7efd4e800640 1 -- start start 2024-02-26T10:42:17.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd480c8090 0x7efd48158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4e800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efd4815d790 0x7efd4815fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd48154ea0 con 0x7efd480c8470 2024-02-26T10:42:17.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd480c8cd0 con 0x7efd480c8090 2024-02-26T10:42:17.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4e800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd48155d90 con 0x7efd4815d790 2024-02-26T10:42:17.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd480c8090 0x7efd48158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efd480c8090 0x7efd48158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50702/0 (socket says 172.21.15.67:50702) 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4d400640 1 -- 172.21.15.67:0/1233317706 learned_addr learned my addr 172.21.15.67:0/1233317706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4ca00640 1 --1- 172.21.15.67:0/1233317706 >> v1:172.21.15.67:6789/0 conn(0x7efd480c8470 0x7efd48155680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34698/0 (socket says 172.21.15.67:34698) 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.420+0000 7efd4de00640 1 --2- 172.21.15.67:0/1233317706 >> v2:172.21.15.82:3300/0 conn(0x7efd4815d790 0x7efd4815fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.421+0000 7efd4d400640 1 -- 172.21.15.67:0/1233317706 >> v2:172.21.15.82:3300/0 conn(0x7efd4815d790 msgr2=0x7efd4815fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.421+0000 7efd4d400640 1 --2- 172.21.15.67:0/1233317706 >> v2:172.21.15.82:3300/0 conn(0x7efd4815d790 0x7efd4815fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.421+0000 7efd4d400640 1 -- 172.21.15.67:0/1233317706 >> v1:172.21.15.67:6789/0 conn(0x7efd480c8470 legacy=0x7efd48155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.421+0000 7efd4d400640 1 -- 172.21.15.67:0/1233317706 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48070870 con 0x7efd480c8090 2024-02-26T10:42:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.422+0000 7efd4d400640 1 --2- 172.21.15.67:0/1233317706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd48158410 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7efd480c8e50 tx=0x7efd3c02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98de90f814873809 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.422+0000 7efd47e00640 1 -- 172.21.15.67:0/1233317706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0317f0 con 0x7efd480c8090 2024-02-26T10:42:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd47e00640 1 -- 172.21.15.67:0/1233317706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c037070 con 0x7efd480c8090 2024-02-26T10:42:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd47e00640 1 -- 172.21.15.67:0/1233317706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c031bb0 con 0x7efd480c8090 2024-02-26T10:42:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 -- 172.21.15.67:0/1233317706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 msgr2=0x7efd48158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 --2- 172.21.15.67:0/1233317706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd48158410 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7efd480c8e50 tx=0x7efd3c02b890 comp rx=0 tx=0).stop 2024-02-26T10:42:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 -- 172.21.15.67:0/1233317706 shutdown_connections 2024-02-26T10:42:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 --2- 172.21.15.67:0/1233317706 >> v2:172.21.15.82:3300/0 conn(0x7efd4815d790 0x7efd4815fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 --2- 172.21.15.67:0/1233317706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd48158410 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 -- 172.21.15.67:0/1233317706 >> 172.21.15.67:0/1233317706 conn(0x7efd4806ac20 msgr2=0x7efd4806b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 -- 172.21.15.67:0/1233317706 shutdown_connections 2024-02-26T10:42:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.423+0000 7efd4e800640 1 -- 172.21.15.67:0/1233317706 wait complete. 2024-02-26T10:42:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.425+0000 7efd4e800640 1 Processor -- start 2024-02-26T10:42:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.425+0000 7efd4e800640 1 -- start start 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd4815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd480c8470 0x7efd481561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4815d790 0x7efd48156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c88d0 con 0x7efd480c8090 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48155160 con 0x7efd4815d790 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd48152770 con 0x7efd480c8470 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd4815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd4815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50716/0 (socket says 172.21.15.67:50716) 2024-02-26T10:42:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.426+0000 7efd4d400640 1 -- 172.21.15.67:0/117931316 learned_addr learned my addr 172.21.15.67:0/117931316 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4ca00640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd480c8470 0x7efd481561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4de00640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4815d790 0x7efd48156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4d400640 1 -- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd480c8470 msgr2=0x7efd481561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4d400640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd480c8470 0x7efd481561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4d400640 1 -- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4815d790 msgr2=0x7efd48156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4d400640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4815d790 0x7efd48156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4d400640 1 -- 172.21.15.67:0/117931316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48159ce0 con 0x7efd480c8090 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4ca00640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd480c8470 0x7efd481561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4de00640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4815d790 0x7efd48156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.427+0000 7efd4d400640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd4815c000 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7efd3c02b8d0 tx=0x7efd3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.428+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c032890 con 0x7efd480c8090 2024-02-26T10:42:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.428+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c02e070 con 0x7efd480c8090 2024-02-26T10:42:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.428+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c032c90 con 0x7efd480c8090 2024-02-26T10:42:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.428+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4815ae90 con 0x7efd480c8090 2024-02-26T10:42:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.428+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4815b450 con 0x7efd480c8090 2024-02-26T10:42:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.429+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd48074a30 con 0x7efd480c8090 2024-02-26T10:42:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.430+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd3c04c060 con 0x7efd480c8090 2024-02-26T10:42:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.431+0000 7efd45600640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd2806b4d0 0x7efd2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.431+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd3c0abda0 con 0x7efd480c8090 2024-02-26T10:42:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.432+0000 7efd4ca00640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd2806b4d0 0x7efd2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:17.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.433+0000 7efd4ca00640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd2806b4d0 0x7efd2806d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd480ab190 tx=0x7efd34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:17.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.436+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd3c040070 con 0x7efd480c8090 2024-02-26T10:42:17.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:17.638+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efd480a7bb0 con 0x7efd480c8090 2024-02-26T10:42:18.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.520+0000 7efd45600640 1 -- 172.21.15.67:0/117931316 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v77) v1 ==== 140+0+0 (secure 0 0 0) 0x7efd3c07acb0 con 0x7efd480c8090 2024-02-26T10:42:18.518 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_metadata' created 2024-02-26T10:42:18.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.524+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd2806b4d0 msgr2=0x7efd2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:18.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.524+0000 7efd4e800640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd2806b4d0 0x7efd2806d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd480ab190 tx=0x7efd34009040 comp rx=0 tx=0).stop 2024-02-26T10:42:18.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.524+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 msgr2=0x7efd4815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.524+0000 7efd4e800640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd4815c000 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7efd3c02b8d0 tx=0x7efd3c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 shutdown_connections 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efd2806b4d0 0x7efd2806d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4815d790 0x7efd48156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd480c8470 0x7efd481561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 --2- 172.21.15.67:0/117931316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd480c8090 0x7efd4815c000 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 >> 172.21.15.67:0/117931316 conn(0x7efd4806ac20 msgr2=0x7efd480c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:18.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.525+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 shutdown_connections 2024-02-26T10:42:18.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.526+0000 7efd4e800640 1 -- 172.21.15.67:0/117931316 wait complete. 2024-02-26T10:42:18.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-26T10:42:18.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.764+0000 7f6396000640 1 Processor -- start 2024-02-26T10:42:18.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.764+0000 7f6396000640 1 -- start start 2024-02-26T10:42:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.766+0000 7f6396000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6390072dd0 0x7f639006e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.766+0000 7f6396000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 0x7f6390153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.766+0000 7f6396000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639006eb60 con 0x7f639013c000 2024-02-26T10:42:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.766+0000 7f6396000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6390073260 con 0x7f6390072dd0 2024-02-26T10:42:18.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.766+0000 7f6396000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63901564b0 con 0x7f6390152ea0 2024-02-26T10:42:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.767+0000 7f6395600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 0x7f6390153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.767+0000 7f638fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6390072dd0 0x7f639006e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.767+0000 7f6395600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 0x7f6390153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:36426/0 (socket says 172.21.15.67:36426) 2024-02-26T10:42:18.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.767+0000 7f6395600640 1 -- 172.21.15.67:0/3010868436 learned_addr learned my addr 172.21.15.67:0/3010868436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:18.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.768+0000 7f6395600640 1 -- 172.21.15.67:0/3010868436 >> v2:172.21.15.82:3300/0 conn(0x7f6390072dd0 msgr2=0x7f639006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:18.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.768+0000 7f6395600640 1 --2- 172.21.15.67:0/3010868436 >> v2:172.21.15.82:3300/0 conn(0x7f6390072dd0 0x7f639006e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.768+0000 7f6395600640 1 -- 172.21.15.67:0/3010868436 >> v1:172.21.15.82:6789/0 conn(0x7f639013c000 legacy=0x7f63900c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:18.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.768+0000 7f6395600640 1 -- 172.21.15.67:0/3010868436 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390156790 con 0x7f6390152ea0 2024-02-26T10:42:18.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.768+0000 7f638fe00640 1 --2- 172.21.15.67:0/3010868436 >> v2:172.21.15.82:3300/0 conn(0x7f6390072dd0 0x7f639006e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.770+0000 7f6395600640 1 --2- 172.21.15.67:0/3010868436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 0x7f6390153330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6388009e50 tx=0x7f638802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f1129c7437bb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.770+0000 7f638f400640 1 -- 172.21.15.67:0/3010868436 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638802ca80 con 0x7f6390152ea0 2024-02-26T10:42:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.770+0000 7f638f400640 1 -- 172.21.15.67:0/3010868436 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638802cc20 con 0x7f6390152ea0 2024-02-26T10:42:18.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.770+0000 7f638f400640 1 -- 172.21.15.67:0/3010868436 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880343f0 con 0x7f6390152ea0 2024-02-26T10:42:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 -- 172.21.15.67:0/3010868436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 msgr2=0x7f6390153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 --2- 172.21.15.67:0/3010868436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 0x7f6390153330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6388009e50 tx=0x7f638802bee0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 -- 172.21.15.67:0/3010868436 shutdown_connections 2024-02-26T10:42:18.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 --2- 172.21.15.67:0/3010868436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390152ea0 0x7f6390153330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 --2- 172.21.15.67:0/3010868436 >> v2:172.21.15.82:3300/0 conn(0x7f6390072dd0 0x7f639006e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 -- 172.21.15.67:0/3010868436 >> 172.21.15.67:0/3010868436 conn(0x7f63900a84a0 msgr2=0x7f63900a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 -- 172.21.15.67:0/3010868436 shutdown_connections 2024-02-26T10:42:18.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.771+0000 7f6396000640 1 -- 172.21.15.67:0/3010868436 wait complete. 2024-02-26T10:42:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.773+0000 7f6396000640 1 Processor -- start 2024-02-26T10:42:18.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.773+0000 7f6396000640 1 -- start start 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.774+0000 7f6396000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390072dd0 0x7f63901e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6396000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 0x7f63901554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6396000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6390152ea0 0x7f63901331b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6396000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390157270 con 0x7f639013c000 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6396000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390156d50 con 0x7f6390152ea0 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6396000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639013c3e0 con 0x7f6390072dd0 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f638fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 0x7f63901554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:18.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f638fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 0x7f63901554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50718/0 (socket says 172.21.15.67:50718) 2024-02-26T10:42:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f638fe00640 1 -- 172.21.15.67:0/770897749 learned_addr learned my addr 172.21.15.67:0/770897749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6394c00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390072dd0 0x7f63901e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:18.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.775+0000 7f6395600640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6390152ea0 0x7f63901331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638fe00640 1 -- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390072dd0 msgr2=0x7f63901e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638fe00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390072dd0 0x7f63901e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638fe00640 1 -- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6390152ea0 msgr2=0x7f63901331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638fe00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6390152ea0 0x7f63901331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638fe00640 1 -- 172.21.15.67:0/770897749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63901e95a0 con 0x7f639013c000 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638fe00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 0x7f63901554d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f638000e960 tx=0x7f638000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380017070 con 0x7f639013c000 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638000c9a0 con 0x7f639013c000 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000cd70 con 0x7f639013c000 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.776+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63900c6740 con 0x7f639013c000 2024-02-26T10:42:18.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.777+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63900c6d00 con 0x7f639013c000 2024-02-26T10:42:18.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.778+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6390074a30 con 0x7f639013c000 2024-02-26T10:42:18.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.779+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f638001a070 con 0x7f639013c000 2024-02-26T10:42:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.780+0000 7f638cc00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:18.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.780+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6080+0+0 (secure 0 0 0) 0x7f638008a4c0 con 0x7f639013c000 2024-02-26T10:42:18.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.781+0000 7f6394c00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:18.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.782+0000 7f6394c00640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f637406b4d0 0x7f637406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63900a8ca0 tx=0x7f6384009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:18.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.785+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63901e95a0 con 0x7f639013c000 2024-02-26T10:42:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:18.983+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f63900c0a40 con 0x7f639013c000 2024-02-26T10:42:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.527+0000 7f638cc00640 1 -- 172.21.15.67:0/770897749 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v78) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6380058b90 con 0x7f639013c000 2024-02-26T10:42:19.525 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_data' created 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f637406b4d0 msgr2=0x7f637406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f637406b4d0 0x7f637406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63900a8ca0 tx=0x7f6384009040 comp rx=0 tx=0).stop 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 msgr2=0x7f63901554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 0x7f63901554d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f638000e960 tx=0x7f638000ee20 comp rx=0 tx=0).stop 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 shutdown_connections 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6390152ea0 0x7f63901331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f639013c000 0x7f63901554d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 --2- 172.21.15.67:0/770897749 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6390072dd0 0x7f63901e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 >> 172.21.15.67:0/770897749 conn(0x7f63900a84a0 msgr2=0x7f639013a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.531+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 shutdown_connections 2024-02-26T10:42:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.532+0000 7f6396000640 1 -- 172.21.15.67:0/770897749 wait complete. 2024-02-26T10:42:19.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-02-26T10:42:19.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.754+0000 7f2d57200640 1 Processor -- start 2024-02-26T10:42:19.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.755+0000 7f2d57200640 1 -- start start 2024-02-26T10:42:19.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d57200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d501503d0 0x7f2d5006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d57200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2d5006e390 0x7f2d501533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d57200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d500a6eb0 con 0x7f2d500c2950 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d57200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d500a7190 con 0x7f2d5006e390 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d57200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500a7470 con 0x7f2d501503d0 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d4f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2d5006e390 0x7f2d501533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d4f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2d5006e390 0x7f2d501533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50486/0 (socket says 172.21.15.67:50486) 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.756+0000 7f2d4f400640 1 -- 172.21.15.67:0/2675820257 learned_addr learned my addr 172.21.15.67:0/2675820257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.757+0000 7f2d4fe00640 1 --2- 172.21.15.67:0/2675820257 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d501503d0 0x7f2d5006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.757+0000 7f2d4f400640 1 -- 172.21.15.67:0/2675820257 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d501503d0 msgr2=0x7f2d5006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.757+0000 7f2d4f400640 1 --2- 172.21.15.67:0/2675820257 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d501503d0 0x7f2d5006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.757+0000 7f2d4f400640 1 -- 172.21.15.67:0/2675820257 >> v1:172.21.15.82:6789/0 conn(0x7f2d500c2950 legacy=0x7f2d50156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:19.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.757+0000 7f2d4f400640 1 -- 172.21.15.67:0/2675820257 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d500a7750 con 0x7f2d5006e390 2024-02-26T10:42:19.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.758+0000 7f2d4f400640 1 --2- 172.21.15.67:0/2675820257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5006e390 0x7f2d501533e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009e50 tx=0x7f2d4002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4087d59429b7c2c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:19.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.758+0000 7f2d4ea00640 1 -- 172.21.15.67:0/2675820257 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4002c950 con 0x7f2d5006e390 2024-02-26T10:42:19.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.758+0000 7f2d4ea00640 1 -- 172.21.15.67:0/2675820257 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4002caf0 con 0x7f2d5006e390 2024-02-26T10:42:19.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.758+0000 7f2d4ea00640 1 -- 172.21.15.67:0/2675820257 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400343f0 con 0x7f2d5006e390 2024-02-26T10:42:19.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 -- 172.21.15.67:0/2675820257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5006e390 msgr2=0x7f2d501533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:19.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 --2- 172.21.15.67:0/2675820257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5006e390 0x7f2d501533e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009e50 tx=0x7f2d4002bdb0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 -- 172.21.15.67:0/2675820257 shutdown_connections 2024-02-26T10:42:19.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 --2- 172.21.15.67:0/2675820257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d5006e390 0x7f2d501533e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 --2- 172.21.15.67:0/2675820257 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d501503d0 0x7f2d5006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 -- 172.21.15.67:0/2675820257 >> 172.21.15.67:0/2675820257 conn(0x7f2d500c6da0 msgr2=0x7f2d500c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:19.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 -- 172.21.15.67:0/2675820257 shutdown_connections 2024-02-26T10:42:19.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.759+0000 7f2d57200640 1 -- 172.21.15.67:0/2675820257 wait complete. 2024-02-26T10:42:19.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.761+0000 7f2d57200640 1 Processor -- start 2024-02-26T10:42:19.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.761+0000 7f2d57200640 1 -- start start 2024-02-26T10:42:19.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d57200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5006e390 0x7f2d501476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d57200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d500c2950 0x7f2d501edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d57200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 0x7f2d500c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d57200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501507b0 con 0x7f2d501503d0 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d57200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5006ef00 con 0x7f2d5006e390 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d57200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c0ef0 con 0x7f2d500c2950 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d55200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 0x7f2d500c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d55200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 0x7f2d500c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50498/0 (socket says 172.21.15.67:50498) 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d55200640 1 -- 172.21.15.67:0/145763009 learned_addr learned my addr 172.21.15.67:0/145763009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.762+0000 7f2d4fe00640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5006e390 0x7f2d501476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d4f400640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d500c2950 0x7f2d501edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d55200640 1 -- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d500c2950 msgr2=0x7f2d501edcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:19.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d55200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d500c2950 0x7f2d501edcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d55200640 1 -- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5006e390 msgr2=0x7f2d501476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d55200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5006e390 0x7f2d501476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d55200640 1 -- 172.21.15.67:0/145763009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d501ee6a0 con 0x7f2d501503d0 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d55200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 0x7f2d500c4450 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800c410 tx=0x7f2d4800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48011790 con 0x7f2d501503d0 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d48017070 con 0x7f2d501503d0 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48011bb0 con 0x7f2d501503d0 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.763+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d50154250 con 0x7f2d501503d0 2024-02-26T10:42:19.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.764+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5006e020 con 0x7f2d501503d0 2024-02-26T10:42:19.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.765+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d50074a30 con 0x7f2d501503d0 2024-02-26T10:42:19.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.766+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d48007020 con 0x7f2d501503d0 2024-02-26T10:42:19.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.767+0000 7f2d47e00640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2d2c06b4d0 0x7f2d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:19.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.767+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f2d4808aaa0 con 0x7f2d501503d0 2024-02-26T10:42:19.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.768+0000 7f2d4fe00640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2d2c06b4d0 0x7f2d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:19.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.769+0000 7f2d4fe00640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2d2c06b4d0 0x7f2d2c06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006fd0 tx=0x7f2d38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:19.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:19.771+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d48021020 con 0x7f2d501503d0 2024-02-26T10:42:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.010+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f2d5006ec50 con 0x7f2d501503d0 2024-02-26T10:42:20.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.584+0000 7f2d47e00640 1 -- 172.21.15.67:0/145763009 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 14 and data pool 15 v49) v1 ==== 177+0+0 (secure 0 0 0) 0x7f2d480596a0 con 0x7f2d501503d0 2024-02-26T10:42:20.586 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 14 and data pool 15 2024-02-26T10:42:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2d2c06b4d0 msgr2=0x7f2d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2d2c06b4d0 0x7f2d2c06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006fd0 tx=0x7f2d38008040 comp rx=0 tx=0).stop 2024-02-26T10:42:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 msgr2=0x7f2d500c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 0x7f2d500c4450 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800c410 tx=0x7f2d4800c8d0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 shutdown_connections 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d501503d0 0x7f2d500c4450 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d500c2950 0x7f2d501edcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2d2c06b4d0 0x7f2d2c06d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 --2- 172.21.15.67:0/145763009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d5006e390 0x7f2d501476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.589+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 >> 172.21.15.67:0/145763009 conn(0x7f2d500c6da0 msgr2=0x7f2d500c02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.590+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 shutdown_connections 2024-02-26T10:42:20.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.590+0000 7f2d57200640 1 -- 172.21.15.67:0/145763009 wait complete. 2024-02-26T10:42:20.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:20.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.809+0000 7efc38200640 1 Processor -- start 2024-02-26T10:42:20.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.810+0000 7efc38200640 1 -- start start 2024-02-26T10:42:20.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc38200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc300bfa30 0x7efc30158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc38200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc300c1090 0x7efc30158e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc38200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300c15e0 con 0x7efc30152420 2024-02-26T10:42:20.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc38200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc301593a0 con 0x7efc300c1090 2024-02-26T10:42:20.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc38200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3006e660 con 0x7efc300bfa30 2024-02-26T10:42:20.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc35800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7efc30152420 0x7efc300c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47014/0 (socket says 172.21.15.67:47014) 2024-02-26T10:42:20.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc35800640 1 -- 172.21.15.67:0/1720021146 learned_addr learned my addr 172.21.15.67:0/1720021146 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:20.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.811+0000 7efc36200640 1 --2- 172.21.15.67:0/1720021146 >> v2:172.21.15.67:3300/0 conn(0x7efc300c1090 0x7efc30158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc34e00640 1 --2- 172.21.15.67:0/1720021146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc300bfa30 0x7efc30158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc2fe00640 1 -- 172.21.15.67:0/1720021146 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432983234 0 0) 0x7efc300c15e0 con 0x7efc30152420 2024-02-26T10:42:20.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc2fe00640 1 -- 172.21.15.67:0/1720021146 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc18003520 con 0x7efc30152420 2024-02-26T10:42:20.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc36200640 1 -- 172.21.15.67:0/1720021146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc300bfa30 msgr2=0x7efc30158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:20.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc36200640 1 --2- 172.21.15.67:0/1720021146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc300bfa30 0x7efc30158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc36200640 1 -- 172.21.15.67:0/1720021146 >> v1:172.21.15.67:6789/0 conn(0x7efc30152420 legacy=0x7efc300c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.812+0000 7efc36200640 1 -- 172.21.15.67:0/1720021146 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3006e920 con 0x7efc300c1090 2024-02-26T10:42:20.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.813+0000 7efc36200640 1 --2- 172.21.15.67:0/1720021146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300c1090 0x7efc30158e60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efc280097a0 tx=0x7efc2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d4bc6b95ed387c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:20.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.813+0000 7efc2fe00640 1 -- 172.21.15.67:0/1720021146 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc280315a0 con 0x7efc300c1090 2024-02-26T10:42:20.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc2fe00640 1 -- 172.21.15.67:0/1720021146 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28037070 con 0x7efc300c1090 2024-02-26T10:42:20.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc2fe00640 1 -- 172.21.15.67:0/1720021146 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28031960 con 0x7efc300c1090 2024-02-26T10:42:20.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 -- 172.21.15.67:0/1720021146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300c1090 msgr2=0x7efc30158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 --2- 172.21.15.67:0/1720021146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300c1090 0x7efc30158e60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efc280097a0 tx=0x7efc2802b840 comp rx=0 tx=0).stop 2024-02-26T10:42:20.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 -- 172.21.15.67:0/1720021146 shutdown_connections 2024-02-26T10:42:20.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 --2- 172.21.15.67:0/1720021146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300c1090 0x7efc30158e60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 --2- 172.21.15.67:0/1720021146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc300bfa30 0x7efc30158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 -- 172.21.15.67:0/1720021146 >> 172.21.15.67:0/1720021146 conn(0x7efc300a84a0 msgr2=0x7efc300a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:20.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 -- 172.21.15.67:0/1720021146 shutdown_connections 2024-02-26T10:42:20.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.814+0000 7efc38200640 1 -- 172.21.15.67:0/1720021146 wait complete. 2024-02-26T10:42:20.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.816+0000 7efc38200640 1 Processor -- start 2024-02-26T10:42:20.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.816+0000 7efc38200640 1 -- start start 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc38200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 0x7efc300aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc38200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc300c1090 0x7efc30143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc38200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc30152420 0x7efc301443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc38200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc30158400 con 0x7efc300bfa30 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc38200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c1b60 con 0x7efc300c1090 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc38200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc30152800 con 0x7efc30152420 2024-02-26T10:42:20.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc35800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 0x7efc300aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc35800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 0x7efc300aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50508/0 (socket says 172.21.15.67:50508) 2024-02-26T10:42:20.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.817+0000 7efc35800640 1 -- 172.21.15.67:0/3017541780 learned_addr learned my addr 172.21.15.67:0/3017541780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:20.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc36200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc30152420 0x7efc301443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc34e00640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc300c1090 0x7efc30143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc35800640 1 -- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc30152420 msgr2=0x7efc301443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc35800640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc30152420 0x7efc301443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc35800640 1 -- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc300c1090 msgr2=0x7efc30143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc35800640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc300c1090 0x7efc30143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc35800640 1 -- 172.21.15.67:0/3017541780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc301ee350 con 0x7efc300bfa30 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc35800640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 0x7efc300aa060 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c430 tx=0x7efc2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24017070 con 0x7efc300bfa30 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24005ce0 con 0x7efc300bfa30 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240072f0 con 0x7efc300bfa30 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.818+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc301ee630 con 0x7efc300bfa30 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.819+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc301eebf0 con 0x7efc300bfa30 2024-02-26T10:42:20.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.820+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc301eee30 con 0x7efc300bfa30 2024-02-26T10:42:20.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.822+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc2401a070 con 0x7efc300bfa30 2024-02-26T10:42:20.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.822+0000 7efc2d600640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc1406b4d0 0x7efc1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:20.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.823+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6565+0+0 (secure 0 0 0) 0x7efc24089980 con 0x7efc300bfa30 2024-02-26T10:42:20.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.823+0000 7efc34e00640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc1406b4d0 0x7efc1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:20.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.824+0000 7efc34e00640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc1406b4d0 0x7efc1406d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efc301e8840 tx=0x7efc200083c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:20.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:20.827+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc301eee30 con 0x7efc300bfa30 2024-02-26T10:42:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.026+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc30074a30 con 0x7efc300bfa30 2024-02-26T10:42:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.030+0000 7efc2d600640 1 -- 172.21.15.67:0/3017541780 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+20321 (secure 0 0 0) 0x7efc24058530 con 0x7efc300bfa30 2024-02-26T10:42:21.028 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:21.029 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":79,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:20.012624+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.032+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc1406b4d0 msgr2=0x7efc1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.032+0000 7efc38200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc1406b4d0 0x7efc1406d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efc301e8840 tx=0x7efc200083c0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.032+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 msgr2=0x7efc300aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.032+0000 7efc38200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 0x7efc300aa060 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c430 tx=0x7efc2400c8f0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 shutdown_connections 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc30152420 0x7efc301443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efc1406b4d0 0x7efc1406d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc300c1090 0x7efc30143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 --2- 172.21.15.67:0/3017541780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc300bfa30 0x7efc300aa060 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 >> 172.21.15.67:0/3017541780 conn(0x7efc300a84a0 msgr2=0x7efc300c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 shutdown_connections 2024-02-26T10:42:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.033+0000 7efc38200640 1 -- 172.21.15.67:0/3017541780 wait complete. 2024-02-26T10:42:21.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:21.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.255+0000 7f7e9e200640 1 Processor -- start 2024-02-26T10:42:21.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.255+0000 7f7e9e200640 1 -- start start 2024-02-26T10:42:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e9e200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e98150300 0x7f7e980c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e9e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e9806e410 0x7f7e980c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e98156230 0x7f7e980aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e9e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e981506e0 con 0x7f7e9806e410 2024-02-26T10:42:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e9e200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e98153a60 con 0x7f7e98150300 2024-02-26T10:42:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e9e200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98152e50 con 0x7f7e98156230 2024-02-26T10:42:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.257+0000 7f7e96a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e9806e410 0x7f7e980c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e9806e410 0x7f7e980c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50518/0 (socket says 172.21.15.67:50518) 2024-02-26T10:42:21.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 -- 172.21.15.67:0/1237261656 learned_addr learned my addr 172.21.15.67:0/1237261656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e97400640 1 --2- 172.21.15.67:0/1237261656 >> v2:172.21.15.82:3300/0 conn(0x7f7e98150300 0x7f7e980c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e97e00640 1 --2- 172.21.15.67:0/1237261656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e98156230 0x7f7e980aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 -- 172.21.15.67:0/1237261656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e98156230 msgr2=0x7f7e980aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 --2- 172.21.15.67:0/1237261656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e98156230 0x7f7e980aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 -- 172.21.15.67:0/1237261656 >> v2:172.21.15.82:3300/0 conn(0x7f7e98150300 msgr2=0x7f7e980c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 --2- 172.21.15.67:0/1237261656 >> v2:172.21.15.82:3300/0 conn(0x7f7e98150300 0x7f7e980c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.258+0000 7f7e96a00640 1 -- 172.21.15.67:0/1237261656 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98153130 con 0x7f7e9806e410 2024-02-26T10:42:21.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.260+0000 7f7e96a00640 1 --2- 172.21.15.67:0/1237261656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e9806e410 0x7f7e980c3530 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7e9806bcc0 tx=0x7f7e8402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cafa8780dfd6ff2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:21.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.260+0000 7f7e96000640 1 -- 172.21.15.67:0/1237261656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84038070 con 0x7f7e9806e410 2024-02-26T10:42:21.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.260+0000 7f7e96000640 1 -- 172.21.15.67:0/1237261656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8402f050 con 0x7f7e9806e410 2024-02-26T10:42:21.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.260+0000 7f7e96000640 1 -- 172.21.15.67:0/1237261656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84031970 con 0x7f7e9806e410 2024-02-26T10:42:21.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 -- 172.21.15.67:0/1237261656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e9806e410 msgr2=0x7f7e980c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 --2- 172.21.15.67:0/1237261656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e9806e410 0x7f7e980c3530 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7e9806bcc0 tx=0x7f7e8402b7f0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 -- 172.21.15.67:0/1237261656 shutdown_connections 2024-02-26T10:42:21.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 --2- 172.21.15.67:0/1237261656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e98156230 0x7f7e980aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 --2- 172.21.15.67:0/1237261656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e9806e410 0x7f7e980c3530 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 --2- 172.21.15.67:0/1237261656 >> v2:172.21.15.82:3300/0 conn(0x7f7e98150300 0x7f7e980c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 -- 172.21.15.67:0/1237261656 >> 172.21.15.67:0/1237261656 conn(0x7f7e980a84a0 msgr2=0x7f7e980a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 -- 172.21.15.67:0/1237261656 shutdown_connections 2024-02-26T10:42:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.261+0000 7f7e9e200640 1 -- 172.21.15.67:0/1237261656 wait complete. 2024-02-26T10:42:21.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.263+0000 7f7e9e200640 1 Processor -- start 2024-02-26T10:42:21.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.263+0000 7f7e9e200640 1 -- start start 2024-02-26T10:42:21.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e9806e410 0x7f7e98071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 0x7f7e980bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e9e200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98156230 0x7f7e98148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e9e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9806dd80 con 0x7f7e98150300 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e9e200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e981536f0 con 0x7f7e98156230 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e9e200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980ab0e0 con 0x7f7e9806e410 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e96a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 0x7f7e980bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e96a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 0x7f7e980bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50524/0 (socket says 172.21.15.67:50524) 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e96a00640 1 -- 172.21.15.67:0/3576391668 learned_addr learned my addr 172.21.15.67:0/3576391668 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.264+0000 7f7e97400640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e9806e410 0x7f7e98071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e97e00640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98156230 0x7f7e98148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e96a00640 1 -- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e9806e410 msgr2=0x7f7e98071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e96a00640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e9806e410 0x7f7e98071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e96a00640 1 -- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98156230 msgr2=0x7f7e98148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e96a00640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98156230 0x7f7e98148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e96a00640 1 -- 172.21.15.67:0/3576391668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98151c80 con 0x7f7e98150300 2024-02-26T10:42:21.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e97e00640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98156230 0x7f7e98148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e96a00640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 0x7f7e980bfca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7e84000c00 tx=0x7f7e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84043070 con 0x7f7e98150300 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8403f870 con 0x7f7e98150300 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84038040 con 0x7f7e98150300 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.265+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e98071f70 con 0x7f7e98150300 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.266+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e981521f0 con 0x7f7e98150300 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.267+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e98074a30 con 0x7f7e98150300 2024-02-26T10:42:21.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.268+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e84046070 con 0x7f7e98150300 2024-02-26T10:42:21.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.268+0000 7f7e8f400640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7e7806b4d0 0x7f7e7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.269+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f7e840ac570 con 0x7f7e98150300 2024-02-26T10:42:21.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.269+0000 7f7e97400640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7e7806b4d0 0x7f7e7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.270+0000 7f7e97400640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7e7806b4d0 0x7f7e7806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006fd0 tx=0x7f7e88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:21.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.273+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e8407b110 con 0x7f7e98150300 2024-02-26T10:42:21.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.469+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e9806e0d0 con 0x7f7e98150300 2024-02-26T10:42:21.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.472+0000 7f7e8f400640 1 -- 172.21.15.67:0/3576391668 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+20321 (secure 0 0 0) 0x7f7e8407aa10 con 0x7f7e98150300 2024-02-26T10:42:21.471 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:21.471 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":79,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:20.012624+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:21.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7e7806b4d0 msgr2=0x7f7e7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7e7806b4d0 0x7f7e7806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006fd0 tx=0x7f7e88008040 comp rx=0 tx=0).stop 2024-02-26T10:42:21.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 msgr2=0x7f7e980bfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 0x7f7e980bfca0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7e84000c00 tx=0x7f7e840046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 shutdown_connections 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7e7806b4d0 0x7f7e7806d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e98156230 0x7f7e98148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e98150300 0x7f7e980bfca0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 --2- 172.21.15.67:0/3576391668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e9806e410 0x7f7e98071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 >> 172.21.15.67:0/3576391668 conn(0x7f7e980a84a0 msgr2=0x7f7e980a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 shutdown_connections 2024-02-26T10:42:21.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.475+0000 7f7e9e200640 1 -- 172.21.15.67:0/3576391668 wait complete. 2024-02-26T10:42:21.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-02-26T10:42:21.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.697+0000 7f85ea000640 1 Processor -- start 2024-02-26T10:42:21.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.698+0000 7f85ea000640 1 -- start start 2024-02-26T10:42:21.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.699+0000 7f85ea000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85e40ab480 0x7f85e40c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.699+0000 7f85ea000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e406e9e0 con 0x7f85e41503a0 2024-02-26T10:42:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.699+0000 7f85ea000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e4150780 con 0x7f85e40a9430 2024-02-26T10:42:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.699+0000 7f85ea000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40ab8d0 con 0x7f85e40ab480 2024-02-26T10:42:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e8c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f85e41503a0 0x7f85e41535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47030/0 (socket says 172.21.15.67:47030) 2024-02-26T10:42:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e8c00640 1 -- 172.21.15.67:0/2616069845 learned_addr learned my addr 172.21.15.67:0/2616069845 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276295622 0 0) 0x7f85e406e9e0 con 0x7f85e41503a0 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85cc003520 con 0x7f85e41503a0 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e9600640 1 --2- 172.21.15.67:0/2616069845 >> v2:172.21.15.82:3300/0 conn(0x7f85e40ab480 0x7f85e40c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749198339 0 0) 0x7f85cc003520 con 0x7f85e41503a0 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 >> v2:172.21.15.82:3300/0 conn(0x7f85e40ab480 msgr2=0x7f85e40c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 --2- 172.21.15.67:0/2616069845 >> v2:172.21.15.82:3300/0 conn(0x7f85e40ab480 0x7f85e40c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 >> v1:172.21.15.82:6789/0 conn(0x7f85e40a9430 legacy=0x7f85e406e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e40a98c0 con 0x7f85e41503a0 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.700+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f85d8001670 con 0x7f85e41503a0 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 >> v1:172.21.15.67:6789/0 conn(0x7f85e41503a0 legacy=0x7f85e41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc003a00 0x7f85cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc004800 0x7f85cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85cc005210 0x7f85cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40ab8d0 con 0x7f85cc004800 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc00c140 con 0x7f85cc005210 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc00c340 con 0x7f85cc003a00 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3e00640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc004800 0x7f85cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3e00640 1 -- 172.21.15.67:0/2616069845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc003a00 msgr2=0x7f85cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3e00640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc003a00 0x7f85cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3e00640 1 -- 172.21.15.67:0/2616069845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85cc005210 msgr2=0x7f85cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3e00640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85cc005210 0x7f85cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e3e00640 1 -- 172.21.15.67:0/2616069845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc003520 con 0x7f85cc004800 2024-02-26T10:42:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.701+0000 7f85e9600640 1 -- 172.21.15.67:0/2616069845 reap_dead start 2024-02-26T10:42:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.702+0000 7f85e3e00640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc004800 0x7f85cc004c80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f85d4003d80 tx=0x7f85cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc73244e0519bf5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.702+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4037070 con 0x7f85cc004800 2024-02-26T10:42:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.702+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d4032600 con 0x7f85cc004800 2024-02-26T10:42:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.702+0000 7f85e3400640 1 -- 172.21.15.67:0/2616069845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4003980 con 0x7f85cc004800 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 -- 172.21.15.67:0/2616069845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc004800 msgr2=0x7f85cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc004800 0x7f85cc004c80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f85d4003d80 tx=0x7f85cc005830 comp rx=0 tx=0).stop 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 -- 172.21.15.67:0/2616069845 shutdown_connections 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 --2- 172.21.15.67:0/2616069845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85cc004800 0x7f85cc004c80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 -- 172.21.15.67:0/2616069845 >> 172.21.15.67:0/2616069845 conn(0x7f85e40c6380 msgr2=0x7f85e40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 -- 172.21.15.67:0/2616069845 shutdown_connections 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.703+0000 7f85ea000640 1 -- 172.21.15.67:0/2616069845 wait complete. 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 Processor -- start 2024-02-26T10:42:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 -- start start 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc004800 0x7f85e41eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 0x7f85e406b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e41eb1e0 0x7f85e41eb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e406dad0 con 0x7f85e4070c20 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40a9e80 con 0x7f85e41eb1e0 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.705+0000 7f85ea000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e406ddb0 con 0x7f85cc004800 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 0x7f85e406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 0x7f85e406b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50542/0 (socket says 172.21.15.67:50542) 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 -- 172.21.15.67:0/1619687947 learned_addr learned my addr 172.21.15.67:0/1619687947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e9600640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e41eb1e0 0x7f85e41eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 -- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc004800 msgr2=0x7f85e41eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e8c00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc004800 0x7f85e41eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc004800 0x7f85e41eea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 -- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e41eb1e0 msgr2=0x7f85e41eb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e41eb1e0 0x7f85e41eb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.706+0000 7f85e3e00640 1 -- 172.21.15.67:0/1619687947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e40bfaa0 con 0x7f85e4070c20 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.707+0000 7f85e3e00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 0x7f85e406b140 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f85d4003d80 tx=0x7f85d40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.707+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4042030 con 0x7f85e4070c20 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.707+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d4032400 con 0x7f85e4070c20 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.707+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4037040 con 0x7f85e4070c20 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.707+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e40bfd40 con 0x7f85e4070c20 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.707+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e41eacc0 con 0x7f85e4070c20 2024-02-26T10:42:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.708+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e40678e0 con 0x7f85e4070c20 2024-02-26T10:42:21.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.709+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85d4045070 con 0x7f85e4070c20 2024-02-26T10:42:21.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.710+0000 7f85e0c00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c806b500 0x7f85c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:21.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.710+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f85d40abbf0 con 0x7f85e4070c20 2024-02-26T10:42:21.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.711+0000 7f85e8c00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c806b500 0x7f85c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:21.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.712+0000 7f85e8c00640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c806b500 0x7f85c806d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f85e4068340 tx=0x7f85d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:21.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.715+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85d404c070 con 0x7f85e4070c20 2024-02-26T10:42:21.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:21.953+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f85e41eaab0 con 0x7f85e4070c20 2024-02-26T10:42:22.157 INFO:tasks.ceph.mds.c.smithi067.stderr:2024-02-26T10:42:22.159+0000 7f1462a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:42:22.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.595+0000 7f85e0c00640 1 -- 172.21.15.67:0/1619687947 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 119+0+0 (secure 0 0 0) 0x7f85d407a7a0 con 0x7f85e4070c20 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c806b500 msgr2=0x7f85c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c806b500 0x7f85c806d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f85e4068340 tx=0x7f85d8009040 comp rx=0 tx=0).stop 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 msgr2=0x7f85e406b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 0x7f85e406b140 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f85d4003d80 tx=0x7f85d40036f0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 shutdown_connections 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c806b500 0x7f85c806d9c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e41eb1e0 0x7f85e41eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e4070c20 0x7f85e406b140 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 --2- 172.21.15.67:0/1619687947 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85cc004800 0x7f85e41eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 >> 172.21.15.67:0/1619687947 conn(0x7f85e40c6380 msgr2=0x7f85e414d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 shutdown_connections 2024-02-26T10:42:22.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.599+0000 7f85ea000640 1 -- 172.21.15.67:0/1619687947 wait complete. 2024-02-26T10:42:22.619 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-26T10:42:22.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-02-26T10:42:22.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.845+0000 7f75e4a00640 1 Processor -- start 2024-02-26T10:42:22.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.846+0000 7f75e4a00640 1 -- start start 2024-02-26T10:42:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.847+0000 7f75e4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00c5290 0x7f75e01562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.847+0000 7f75e4a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e006c110 con 0x7f75e00af8a0 2024-02-26T10:42:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.847+0000 7f75e4a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e006d000 con 0x7f75e0155880 2024-02-26T10:42:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.847+0000 7f75e4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c72c0 con 0x7f75e00c5290 2024-02-26T10:42:22.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.848+0000 7f75dfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f75e00af8a0 0x7f75e014e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47032/0 (socket says 172.21.15.67:47032) 2024-02-26T10:42:22.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.848+0000 7f75dfe00640 1 -- 172.21.15.67:0/3822072714 learned_addr learned my addr 172.21.15.67:0/3822072714 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:22.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.848+0000 7f75df400640 1 --2- 172.21.15.67:0/3822072714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00c5290 0x7f75e01562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.848+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351233158 0 0) 0x7f75e006c110 con 0x7f75e00af8a0 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.848+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c4003520 con 0x7f75e00af8a0 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.849+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86896664 0 0) 0x7f75e006d000 con 0x7f75e0155880 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.849+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e006c110 con 0x7f75e0155880 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.849+0000 7f75df400640 1 -- 172.21.15.67:0/3822072714 >> v1:172.21.15.82:6789/0 conn(0x7f75e0155880 legacy=0x7f75e006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.849+0000 7f75df400640 1 -- 172.21.15.67:0/3822072714 >> v1:172.21.15.67:6789/0 conn(0x7f75e00af8a0 legacy=0x7f75e014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.849+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553590140 0 0) 0x7f75c4003520 con 0x7f75e00af8a0 2024-02-26T10:42:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.849+0000 7f75df400640 1 -- 172.21.15.67:0/3822072714 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00c75a0 con 0x7f75e00c5290 2024-02-26T10:42:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.850+0000 7f75df400640 1 --2- 172.21.15.67:0/3822072714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00c5290 0x7f75e01562f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75d00097d0 tx=0x7f75d002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ebd2aa46f824b4c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.850+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0037070 con 0x7f75e00c5290 2024-02-26T10:42:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.850+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0030510 con 0x7f75e00c5290 2024-02-26T10:42:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.850+0000 7f75de000640 1 -- 172.21.15.67:0/3822072714 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80019f0 con 0x7f75e00c5290 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 -- 172.21.15.67:0/3822072714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00c5290 msgr2=0x7f75e01562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 --2- 172.21.15.67:0/3822072714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00c5290 0x7f75e01562f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75d00097d0 tx=0x7f75d002b770 comp rx=0 tx=0).stop 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 -- 172.21.15.67:0/3822072714 shutdown_connections 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 --2- 172.21.15.67:0/3822072714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00c5290 0x7f75e01562f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 -- 172.21.15.67:0/3822072714 >> 172.21.15.67:0/3822072714 conn(0x7f75e00ac230 msgr2=0x7f75e00ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 -- 172.21.15.67:0/3822072714 shutdown_connections 2024-02-26T10:42:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.851+0000 7f75e4a00640 1 -- 172.21.15.67:0/3822072714 wait complete. 2024-02-26T10:42:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.853+0000 7f75e4a00640 1 Processor -- start 2024-02-26T10:42:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.853+0000 7f75e4a00640 1 -- start start 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75e4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00af8a0 0x7f75e0156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75e4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 0x7f75e00c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75e4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0155880 0x7f75e014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75e4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0155cc0 con 0x7f75e00c5290 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75e4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e006c3d0 con 0x7f75e0155880 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75e4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c5670 con 0x7f75e00af8a0 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 0x7f75e00c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 0x7f75e00c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50554/0 (socket says 172.21.15.67:50554) 2024-02-26T10:42:22.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.854+0000 7f75dea00640 1 -- 172.21.15.67:0/436275352 learned_addr learned my addr 172.21.15.67:0/436275352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75df400640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00af8a0 0x7f75e0156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dfe00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0155880 0x7f75e014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dea00640 1 -- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00af8a0 msgr2=0x7f75e0156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dea00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00af8a0 0x7f75e0156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dea00640 1 -- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0155880 msgr2=0x7f75e014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dea00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0155880 0x7f75e014db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dea00640 1 -- 172.21.15.67:0/436275352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e01ed940 con 0x7f75e00c5290 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75dea00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 0x7f75e00c1ff0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c4c0 tx=0x7f75c800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.855+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c80052d0 con 0x7f75e00c5290 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.856+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8005470 con 0x7f75e00c5290 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.856+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8012870 con 0x7f75e00c5290 2024-02-26T10:42:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.856+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e01edc20 con 0x7f75e00c5290 2024-02-26T10:42:22.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.856+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e01ee1e0 con 0x7f75e00c5290 2024-02-26T10:42:22.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.857+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e0074a30 con 0x7f75e00c5290 2024-02-26T10:42:22.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.858+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75c8012a10 con 0x7f75e00c5290 2024-02-26T10:42:22.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.859+0000 7f75d7400640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75bc06b4d0 0x7f75bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:22.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.859+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f75c808a220 con 0x7f75e00c5290 2024-02-26T10:42:22.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.859+0000 7f75df400640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75bc06b4d0 0x7f75bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:22.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.861+0000 7f75df400640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75bc06b4d0 0x7f75bc06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75d0034000 tx=0x7f75d0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:22.863+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75c8058d30 con 0x7f75e00c5290 2024-02-26T10:42:23.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.106+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f75e006d950 con 0x7f75e00c5290 2024-02-26T10:42:23.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.607+0000 7f75d7400640 1 -- 172.21.15.67:0/436275352 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 123+0+0 (secure 0 0 0) 0x7f75c8058630 con 0x7f75e00c5290 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75bc06b4d0 msgr2=0x7f75bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75bc06b4d0 0x7f75bc06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75d0034000 tx=0x7f75d0008030 comp rx=0 tx=0).stop 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 msgr2=0x7f75e00c1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 0x7f75e00c1ff0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c4c0 tx=0x7f75c800c980 comp rx=0 tx=0).stop 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 shutdown_connections 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75bc06b4d0 0x7f75bc06d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0155880 0x7f75e014db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75e00c5290 0x7f75e00c1ff0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 --2- 172.21.15.67:0/436275352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75e00af8a0 0x7f75e0156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 >> 172.21.15.67:0/436275352 conn(0x7f75e00ac230 msgr2=0x7f75e00aa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 shutdown_connections 2024-02-26T10:42:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.611+0000 7f75e4a00640 1 -- 172.21.15.67:0/436275352 wait complete. 2024-02-26T10:42:23.630 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-02-26T10:42:23.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.845+0000 7f814b200640 1 Processor -- start 2024-02-26T10:42:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.845+0000 7f814b200640 1 -- start start 2024-02-26T10:42:23.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.846+0000 7f814b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c0a94a0 0x7f814c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:23.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.846+0000 7f814b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f814c06db00 0x7f814c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:23.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.846+0000 7f814b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c06df90 con 0x7f814c06ac20 2024-02-26T10:42:23.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.846+0000 7f814b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c0a9a80 con 0x7f814c06db00 2024-02-26T10:42:23.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.846+0000 7f814b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0a9d60 con 0x7f814c0a94a0 2024-02-26T10:42:23.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.847+0000 7f8149e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f814c06ac20 0x7f814c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:44310/0 (socket says 172.21.15.67:44310) 2024-02-26T10:42:23.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.847+0000 7f8149e00640 1 -- 172.21.15.67:0/541893192 learned_addr learned my addr 172.21.15.67:0/541893192 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:23.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.847+0000 7f8149400640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c0a94a0 0x7f814c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:23.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.847+0000 7f814a800640 1 --2- 172.21.15.67:0/541893192 >> v2:172.21.15.82:3300/0 conn(0x7f814c06db00 0x7f814c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.847+0000 7f8148a00640 1 -- 172.21.15.67:0/541893192 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637552220 0 0) 0x7f814c06df90 con 0x7f814c06ac20 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.848+0000 7f8148a00640 1 -- 172.21.15.67:0/541893192 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8130003520 con 0x7f814c06ac20 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.848+0000 7f814a800640 1 -- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c0a94a0 msgr2=0x7f814c06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.848+0000 7f814a800640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c0a94a0 0x7f814c06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.848+0000 7f814a800640 1 -- 172.21.15.67:0/541893192 >> v1:172.21.15.82:6789/0 conn(0x7f814c06ac20 legacy=0x7f814c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.848+0000 7f814a800640 1 -- 172.21.15.67:0/541893192 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0ab750 con 0x7f814c06db00 2024-02-26T10:42:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.848+0000 7f8149400640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c0a94a0 0x7f814c06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.849+0000 7f814a800640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c06db00 0x7f814c0abf50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8144009d90 tx=0x7f814402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51c115cf70306f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f8148a00640 1 -- 172.21.15.67:0/541893192 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144039070 con 0x7f814c06db00 2024-02-26T10:42:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f8148a00640 1 -- 172.21.15.67:0/541893192 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814402cc50 con 0x7f814c06db00 2024-02-26T10:42:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f8148a00640 1 -- 172.21.15.67:0/541893192 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144034440 con 0x7f814c06db00 2024-02-26T10:42:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 -- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c06db00 msgr2=0x7f814c0abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c06db00 0x7f814c0abf50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8144009d90 tx=0x7f814402bea0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 -- 172.21.15.67:0/541893192 shutdown_connections 2024-02-26T10:42:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c06db00 0x7f814c0abf50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 --2- 172.21.15.67:0/541893192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c0a94a0 0x7f814c06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 -- 172.21.15.67:0/541893192 >> 172.21.15.67:0/541893192 conn(0x7f814c0c63f0 msgr2=0x7f814c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 -- 172.21.15.67:0/541893192 shutdown_connections 2024-02-26T10:42:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.850+0000 7f814b200640 1 -- 172.21.15.67:0/541893192 wait complete. 2024-02-26T10:42:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.852+0000 7f814b200640 1 Processor -- start 2024-02-26T10:42:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.852+0000 7f814b200640 1 -- start start 2024-02-26T10:42:23.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f814b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c06ac20 0x7f814c0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:23.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f814b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 0x7f814c151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:23.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f814b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c0a94a0 0x7f814c1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:23.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f814b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c150410 con 0x7f814c06db00 2024-02-26T10:42:23.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f814b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0ac490 con 0x7f814c0a94a0 2024-02-26T10:42:23.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f814b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c150590 con 0x7f814c06ac20 2024-02-26T10:42:23.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f8149400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 0x7f814c151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:23.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.853+0000 7f8149400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 0x7f814c151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50562/0 (socket says 172.21.15.67:50562) 2024-02-26T10:42:23.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 -- 172.21.15.67:0/3537373186 learned_addr learned my addr 172.21.15.67:0/3537373186 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:23.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149e00640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c06ac20 0x7f814c0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f814a800640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c0a94a0 0x7f814c1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 -- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c06ac20 msgr2=0x7f814c0bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c06ac20 0x7f814c0bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 -- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c0a94a0 msgr2=0x7f814c1521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c0a94a0 0x7f814c1521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 -- 172.21.15.67:0/3537373186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c071660 con 0x7f814c06db00 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.854+0000 7f8149400640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 0x7f814c151c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e990 tx=0x7f813c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.855+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c017070 con 0x7f814c06db00 2024-02-26T10:42:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.855+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c00c980 con 0x7f814c06db00 2024-02-26T10:42:23.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.855+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c00cd50 con 0x7f814c06db00 2024-02-26T10:42:23.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.855+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c071940 con 0x7f814c06db00 2024-02-26T10:42:23.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.855+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c071f00 con 0x7f814c06db00 2024-02-26T10:42:23.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.856+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c074a30 con 0x7f814c06db00 2024-02-26T10:42:23.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.857+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f813c01a070 con 0x7f814c06db00 2024-02-26T10:42:23.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.858+0000 7f8142000640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f812806b4d0 0x7f812806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:23.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.858+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f813c08a770 con 0x7f814c06db00 2024-02-26T10:42:23.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.859+0000 7f8149e00640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f812806b4d0 0x7f812806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:23.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.860+0000 7f8149e00640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f812806b4d0 0x7f812806d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8134005880 tx=0x7f8134009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:23.863+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f813c059280 con 0x7f814c06db00 2024-02-26T10:42:24.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.098+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f814c06b9b0 con 0x7f814c06db00 2024-02-26T10:42:24.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.099+0000 7f8142000640 1 -- 172.21.15.67:0/3537373186 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6185 (secure 0 0 0) 0x7f813c058b80 con 0x7f814c06db00 2024-02-26T10:42:24.097 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:24.097 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:42:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 53 2024-02-26T10:42:24.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.101+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f812806b4d0 msgr2=0x7f812806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.101+0000 7f814b200640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f812806b4d0 0x7f812806d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8134005880 tx=0x7f8134009040 comp rx=0 tx=0).stop 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.101+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 msgr2=0x7f814c151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.101+0000 7f814b200640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 0x7f814c151c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e990 tx=0x7f813c00ee50 comp rx=0 tx=0).stop 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 shutdown_connections 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f812806b4d0 0x7f812806d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f814c0a94a0 0x7f814c1521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c06db00 0x7f814c151c80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 --2- 172.21.15.67:0/3537373186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c06ac20 0x7f814c0bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 >> 172.21.15.67:0/3537373186 conn(0x7f814c0c63f0 msgr2=0x7f814c14ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 shutdown_connections 2024-02-26T10:42:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.102+0000 7f814b200640 1 -- 172.21.15.67:0/3537373186 wait complete. 2024-02-26T10:42:24.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:24.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.335+0000 7f3abac00640 1 Processor -- start 2024-02-26T10:42:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.336+0000 7f3abac00640 1 -- start start 2024-02-26T10:42:24.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.337+0000 7f3abac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 0x7f3ab406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.337+0000 7f3abac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3ab406db00 0x7f3ab40abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.337+0000 7f3abac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab406df90 con 0x7f3ab406ac20 2024-02-26T10:42:24.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.337+0000 7f3abac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab40a9a80 con 0x7f3ab406db00 2024-02-26T10:42:24.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.337+0000 7f3abac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40a9d60 con 0x7f3ab40a94a0 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.338+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 0x7f3ab406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.338+0000 7f3aba200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3ab406db00 0x7f3ab40abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.338+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 0x7f3ab406b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:58508/0 (socket says 172.21.15.67:58508) 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.338+0000 7f3ab8e00640 1 -- 172.21.15.67:0/2625614946 learned_addr learned my addr 172.21.15.67:0/2625614946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.339+0000 7f3aba200640 1 -- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 msgr2=0x7f3ab406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.339+0000 7f3aba200640 1 --2- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 0x7f3ab406b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.339+0000 7f3aba200640 1 -- 172.21.15.67:0/2625614946 >> v1:172.21.15.82:6789/0 conn(0x7f3ab406ac20 legacy=0x7f3ab4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.339+0000 7f3aba200640 1 -- 172.21.15.67:0/2625614946 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40ab750 con 0x7f3ab406db00 2024-02-26T10:42:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.339+0000 7f3ab8e00640 1 --2- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 0x7f3ab406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:24.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.340+0000 7f3aba200640 1 --2- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab406db00 0x7f3ab40abf50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009d90 tx=0x7f3aac02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db3be7f9070f11f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:24.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.340+0000 7f3ab3e00640 1 -- 172.21.15.67:0/2625614946 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac039070 con 0x7f3ab406db00 2024-02-26T10:42:24.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3ab3e00640 1 -- 172.21.15.67:0/2625614946 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac02cc60 con 0x7f3ab406db00 2024-02-26T10:42:24.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3ab3e00640 1 -- 172.21.15.67:0/2625614946 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac034660 con 0x7f3ab406db00 2024-02-26T10:42:24.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 -- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab406db00 msgr2=0x7f3ab40abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 --2- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab406db00 0x7f3ab40abf50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009d90 tx=0x7f3aac02beb0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 -- 172.21.15.67:0/2625614946 shutdown_connections 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 --2- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab406db00 0x7f3ab40abf50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 --2- 172.21.15.67:0/2625614946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab40a94a0 0x7f3ab406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 -- 172.21.15.67:0/2625614946 >> 172.21.15.67:0/2625614946 conn(0x7f3ab40c63f0 msgr2=0x7f3ab40c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 -- 172.21.15.67:0/2625614946 shutdown_connections 2024-02-26T10:42:24.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.341+0000 7f3abac00640 1 -- 172.21.15.67:0/2625614946 wait complete. 2024-02-26T10:42:24.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.344+0000 7f3abac00640 1 Processor -- start 2024-02-26T10:42:24.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.344+0000 7f3abac00640 1 -- start start 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3abac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab406ac20 0x7f3ab40bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3abac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 0x7f3ab41ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3abac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab40a94a0 0x7f3ab41eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3abac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4150410 con 0x7f3ab406db00 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3abac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40ac490 con 0x7f3ab40a94a0 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3abac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4150590 con 0x7f3ab406ac20 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 0x7f3ab41ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 0x7f3ab41ea9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50578/0 (socket says 172.21.15.67:50578) 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3ab8e00640 1 -- 172.21.15.67:0/2073629121 learned_addr learned my addr 172.21.15.67:0/2073629121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.345+0000 7f3ab9800640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab406ac20 0x7f3ab40bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3aba200640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab40a94a0 0x7f3ab41eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab8e00640 1 -- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab406ac20 msgr2=0x7f3ab40bffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab8e00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab406ac20 0x7f3ab40bffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab8e00640 1 -- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab40a94a0 msgr2=0x7f3ab41eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab8e00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab40a94a0 0x7f3ab41eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab8e00640 1 -- 172.21.15.67:0/2073629121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab41eb460 con 0x7f3ab406db00 2024-02-26T10:42:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab9800640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab406ac20 0x7f3ab40bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab8e00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 0x7f3ab41ea9e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9b0 tx=0x7f3aa000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.346+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab406db00 2024-02-26T10:42:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.347+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa000c9a0 con 0x7f3ab406db00 2024-02-26T10:42:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.347+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000cd50 con 0x7f3ab406db00 2024-02-26T10:42:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.347+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab41ea4e0 con 0x7f3ab406db00 2024-02-26T10:42:24.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.347+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab41ee230 con 0x7f3ab406db00 2024-02-26T10:42:24.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.348+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab4074a30 con 0x7f3ab406db00 2024-02-26T10:42:24.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.349+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3aa001a070 con 0x7f3ab406db00 2024-02-26T10:42:24.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.350+0000 7f3ab1600640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3a9406b550 0x7f3a9406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.350+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3aa008a290 con 0x7f3ab406db00 2024-02-26T10:42:24.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.350+0000 7f3ab9800640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3a9406b550 0x7f3a9406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.352+0000 7f3ab9800640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3a9406b550 0x7f3a9406da10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8001700 tx=0x7f3aa8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:24.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.354+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ab41eb460 con 0x7f3ab406db00 2024-02-26T10:42:24.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.559+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ab406b9b0 con 0x7f3ab406db00 2024-02-26T10:42:24.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.561+0000 7f3ab1600640 1 -- 172.21.15.67:0/2073629121 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7f3aa0058f20 con 0x7f3ab406db00 2024-02-26T10:42:24.560 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:24.560 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:24.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3a9406b550 msgr2=0x7f3a9406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3a9406b550 0x7f3a9406da10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8001700 tx=0x7f3aa8009040 comp rx=0 tx=0).stop 2024-02-26T10:42:24.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 msgr2=0x7f3ab41ea9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 0x7f3ab41ea9e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000e9b0 tx=0x7f3aa000ee70 comp rx=0 tx=0).stop 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 shutdown_connections 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3a9406b550 0x7f3a9406da10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab40a94a0 0x7f3ab41eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab406db00 0x7f3ab41ea9e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 --2- 172.21.15.67:0/2073629121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ab406ac20 0x7f3ab40bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 >> 172.21.15.67:0/2073629121 conn(0x7f3ab40c63f0 msgr2=0x7f3ab414e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 shutdown_connections 2024-02-26T10:42:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.564+0000 7f3abac00640 1 -- 172.21.15.67:0/2073629121 wait complete. 2024-02-26T10:42:24.582 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-26T10:42:24.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:24.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.795+0000 7fc20ba00640 1 Processor -- start 2024-02-26T10:42:24.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.796+0000 7fc20ba00640 1 -- start start 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc20ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc20406e410 0x7fc2040aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc20ba00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc204156230 0x7fc2040ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc20ba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc204152e50 con 0x7fc204150300 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc20ba00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2041506e0 con 0x7fc204156230 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc20ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc204153a60 con 0x7fc20406e410 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc209a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc204156230 0x7fc2040ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc209a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc204156230 0x7fc2040ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50584/0 (socket says 172.21.15.67:50584) 2024-02-26T10:42:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.797+0000 7fc209a00640 1 -- 172.21.15.67:0/4289564736 learned_addr learned my addr 172.21.15.67:0/4289564736 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.798+0000 7fc203e00640 1 --2- 172.21.15.67:0/4289564736 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc20406e410 0x7fc2040aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.798+0000 7fc209a00640 1 -- 172.21.15.67:0/4289564736 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc20406e410 msgr2=0x7fc2040aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.798+0000 7fc209a00640 1 --2- 172.21.15.67:0/4289564736 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc20406e410 0x7fc2040aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.798+0000 7fc209a00640 1 -- 172.21.15.67:0/4289564736 >> v1:172.21.15.82:6789/0 conn(0x7fc204150300 legacy=0x7fc2040c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.798+0000 7fc209a00640 1 -- 172.21.15.67:0/4289564736 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc204153130 con 0x7fc204156230 2024-02-26T10:42:24.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.799+0000 7fc209a00640 1 --2- 172.21.15.67:0/4289564736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc2040ab0e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009e50 tx=0x7fc1fc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b381129eefec7e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:24.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.799+0000 7fc203400640 1 -- 172.21.15.67:0/4289564736 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc02c950 con 0x7fc204156230 2024-02-26T10:42:24.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.799+0000 7fc203400640 1 -- 172.21.15.67:0/4289564736 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc02caf0 con 0x7fc204156230 2024-02-26T10:42:24.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.799+0000 7fc203400640 1 -- 172.21.15.67:0/4289564736 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0343f0 con 0x7fc204156230 2024-02-26T10:42:24.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 -- 172.21.15.67:0/4289564736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 msgr2=0x7fc2040ab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 --2- 172.21.15.67:0/4289564736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc2040ab0e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009e50 tx=0x7fc1fc02bdb0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 -- 172.21.15.67:0/4289564736 shutdown_connections 2024-02-26T10:42:24.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 --2- 172.21.15.67:0/4289564736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc2040ab0e0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 --2- 172.21.15.67:0/4289564736 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc20406e410 0x7fc2040aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 -- 172.21.15.67:0/4289564736 >> 172.21.15.67:0/4289564736 conn(0x7fc2040a84a0 msgr2=0x7fc2040a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:24.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 -- 172.21.15.67:0/4289564736 shutdown_connections 2024-02-26T10:42:24.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.800+0000 7fc20ba00640 1 -- 172.21.15.67:0/4289564736 wait complete. 2024-02-26T10:42:24.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.802+0000 7fc20ba00640 1 Processor -- start 2024-02-26T10:42:24.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.802+0000 7fc20ba00640 1 -- start start 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc20ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20406e410 0x7fc2041ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc20ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc204150300 0x7fc20414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc20ba00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc204148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc20ba00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20406dd80 con 0x7fc204156230 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc20ba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2041536f0 con 0x7fc20406e410 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc20ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040c3700 con 0x7fc204150300 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc209a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc204148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc209a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc204148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50596/0 (socket says 172.21.15.67:50596) 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc209a00640 1 -- 172.21.15.67:0/231654282 learned_addr learned my addr 172.21.15.67:0/231654282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc209000640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20406e410 0x7fc2041ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc203e00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc204150300 0x7fc20414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc209a00640 1 -- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc204150300 msgr2=0x7fc20414a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.803+0000 7fc209a00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc204150300 0x7fc20414a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc209a00640 1 -- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20406e410 msgr2=0x7fc2041ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:24.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc209a00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20406e410 0x7fc2041ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:24.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc209a00640 1 -- 172.21.15.67:0/231654282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2041eea90 con 0x7fc204156230 2024-02-26T10:42:24.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc203e00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc204150300 0x7fc20414a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:24.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc209a00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc204148560 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009e50 tx=0x7fc1fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc043020 con 0x7fc204156230 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc03e450 con 0x7fc204156230 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc03e820 con 0x7fc204156230 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc204072c60 con 0x7fc204156230 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.804+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc204071980 con 0x7fc204156230 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.806+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc204074a30 con 0x7fc204156230 2024-02-26T10:42:24.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.806+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1fc046020 con 0x7fc204156230 2024-02-26T10:42:24.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.807+0000 7fc200c00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc1e406b500 0x7fc1e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:24.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.807+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc1fc0acb50 con 0x7fc204156230 2024-02-26T10:42:24.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.808+0000 7fc209000640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc1e406b500 0x7fc1e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:24.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.809+0000 7fc209000640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc1e406b500 0x7fc1e406d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4004ea0 tx=0x7fc1f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:24.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:24.812+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1fc04d070 con 0x7fc204156230 2024-02-26T10:42:25.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.006+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2040c4170 con 0x7fc1e406b500 2024-02-26T10:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.044+0000 7fc200c00640 1 -- 172.21.15.67:0/231654282 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490654 (secure 0 0 0) 0x7fc2040c4170 con 0x7fc1e406b500 2024-02-26T10:42:25.046 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:25.052 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:25.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc1e406b500 msgr2=0x7fc1e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc1e406b500 0x7fc1e406d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4004ea0 tx=0x7fc1f4009040 comp rx=0 tx=0).stop 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 msgr2=0x7fc204148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc204148560 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009e50 tx=0x7fc1fc004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 shutdown_connections 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc204156230 0x7fc204148560 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc204150300 0x7fc20414a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc1e406b500 0x7fc1e406d9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 --2- 172.21.15.67:0/231654282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20406e410 0x7fc2041ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 >> 172.21.15.67:0/231654282 conn(0x7fc2040a84a0 msgr2=0x7fc2040a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.054+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 shutdown_connections 2024-02-26T10:42:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.055+0000 7fc20ba00640 1 -- 172.21.15.67:0/231654282 wait complete. 2024-02-26T10:42:25.076 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-02-26T10:42:24.412072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":367,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258404,"kb_used_data":8936,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716316,"statfs":{"total":773094113280,"available":772829507584,"internally_reserved":0,"allocated":9150464,"data_stored":4734054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.889059"},"pg_stats":[{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593602+0000","last_change":"2024-02-26T10:42:20.608978+0000","last_active":"2024-02-26T10:42:21.593602+0000","last_peered":"2024-02-26T10:42:21.593602+0000","last_clean":"2024-02-26T10:42:21.593602+0000","last_became_active":"2024-02-26T10:42:20.608613+0000","last_became_peered":"2024-02-26T10:42:20.608613+0000","last_unstale":"2024-02-26T10:42:21.593602+0000","last_undegraded":"2024-02-26T10:42:21.593602+0000","last_fullsized":"2024-02-26T10:42:21.593602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:48.452971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923082+0000","last_change":"2024-02-26T10:41:59.308476+0000","last_active":"2024-02-26T10:42:21.923082+0000","last_peered":"2024-02-26T10:42:21.923082+0000","last_clean":"2024-02-26T10:42:21.923082+0000","last_became_active":"2024-02-26T10:41:59.307995+0000","last_became_peered":"2024-02-26T10:41:59.307995+0000","last_unstale":"2024-02-26T10:42:21.923082+0000","last_undegraded":"2024-02-26T10:42:21.923082+0000","last_fullsized":"2024-02-26T10:42:21.923082+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919927+0000","last_change":"2024-02-26T10:42:20.593534+0000","last_active":"2024-02-26T10:42:21.919927+0000","last_peered":"2024-02-26T10:42:21.919927+0000","last_clean":"2024-02-26T10:42:21.919927+0000","last_became_active":"2024-02-26T10:42:20.593129+0000","last_became_peered":"2024-02-26T10:42:20.593129+0000","last_unstale":"2024-02-26T10:42:21.919927+0000","last_undegraded":"2024-02-26T10:42:21.919927+0000","last_fullsized":"2024-02-26T10:42:21.919927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:39:31.051666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594484+0000","last_change":"2024-02-26T10:41:59.300851+0000","last_active":"2024-02-26T10:42:21.594484+0000","last_peered":"2024-02-26T10:42:21.594484+0000","last_clean":"2024-02-26T10:42:21.594484+0000","last_became_active":"2024-02-26T10:41:59.300574+0000","last_became_peered":"2024-02-26T10:41:59.300574+0000","last_unstale":"2024-02-26T10:42:21.594484+0000","last_undegraded":"2024-02-26T10:42:21.594484+0000","last_fullsized":"2024-02-26T10:42:21.594484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:04.528547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593696+0000","last_change":"2024-02-26T10:42:20.605522+0000","last_active":"2024-02-26T10:42:21.593696+0000","last_peered":"2024-02-26T10:42:21.593696+0000","last_clean":"2024-02-26T10:42:21.593696+0000","last_became_active":"2024-02-26T10:42:20.605209+0000","last_became_peered":"2024-02-26T10:42:20.605209+0000","last_unstale":"2024-02-26T10:42:21.593696+0000","last_undegraded":"2024-02-26T10:42:21.593696+0000","last_fullsized":"2024-02-26T10:42:21.593696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:31.467164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920373+0000","last_change":"2024-02-26T10:41:59.293058+0000","last_active":"2024-02-26T10:42:21.920373+0000","last_peered":"2024-02-26T10:42:21.920373+0000","last_clean":"2024-02-26T10:42:21.920373+0000","last_became_active":"2024-02-26T10:41:59.292789+0000","last_became_peered":"2024-02-26T10:41:59.292789+0000","last_unstale":"2024-02-26T10:42:21.920373+0000","last_undegraded":"2024-02-26T10:42:21.920373+0000","last_fullsized":"2024-02-26T10:42:21.920373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597521+0000","last_change":"2024-02-26T10:42:20.613590+0000","last_active":"2024-02-26T10:42:21.597521+0000","last_peered":"2024-02-26T10:42:21.597521+0000","last_clean":"2024-02-26T10:42:21.597521+0000","last_became_active":"2024-02-26T10:42:20.612995+0000","last_became_peered":"2024-02-26T10:42:20.612995+0000","last_unstale":"2024-02-26T10:42:21.597521+0000","last_undegraded":"2024-02-26T10:42:21.597521+0000","last_fullsized":"2024-02-26T10:42:21.597521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:37:10.632162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606436+0000","last_change":"2024-02-26T10:41:59.296577+0000","last_active":"2024-02-26T10:42:21.606436+0000","last_peered":"2024-02-26T10:42:21.606436+0000","last_clean":"2024-02-26T10:42:21.606436+0000","last_became_active":"2024-02-26T10:41:59.296264+0000","last_became_peered":"2024-02-26T10:41:59.296264+0000","last_unstale":"2024-02-26T10:42:21.606436+0000","last_undegraded":"2024-02-26T10:42:21.606436+0000","last_fullsized":"2024-02-26T10:42:21.606436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:13.257965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918739+0000","last_change":"2024-02-26T10:42:20.600877+0000","last_active":"2024-02-26T10:42:21.918739+0000","last_peered":"2024-02-26T10:42:21.918739+0000","last_clean":"2024-02-26T10:42:21.918739+0000","last_became_active":"2024-02-26T10:42:20.599771+0000","last_became_peered":"2024-02-26T10:42:20.599771+0000","last_unstale":"2024-02-26T10:42:21.918739+0000","last_undegraded":"2024-02-26T10:42:21.918739+0000","last_fullsized":"2024-02-26T10:42:21.918739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:53.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598460+0000","last_change":"2024-02-26T10:41:59.298038+0000","last_active":"2024-02-26T10:42:21.598460+0000","last_peered":"2024-02-26T10:42:21.598460+0000","last_clean":"2024-02-26T10:42:21.598460+0000","last_became_active":"2024-02-26T10:41:59.297699+0000","last_became_peered":"2024-02-26T10:41:59.297699+0000","last_unstale":"2024-02-26T10:42:21.598460+0000","last_undegraded":"2024-02-26T10:42:21.598460+0000","last_fullsized":"2024-02-26T10:42:21.598460+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:30.879707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595462+0000","last_change":"2024-02-26T10:42:20.615279+0000","last_active":"2024-02-26T10:42:21.595462+0000","last_peered":"2024-02-26T10:42:21.595462+0000","last_clean":"2024-02-26T10:42:21.595462+0000","last_became_active":"2024-02-26T10:42:20.610374+0000","last_became_peered":"2024-02-26T10:42:20.610374+0000","last_unstale":"2024-02-26T10:42:21.595462+0000","last_undegraded":"2024-02-26T10:42:21.595462+0000","last_fullsized":"2024-02-26T10:42:21.595462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731746+0000","last_change":"2024-02-26T10:41:59.295626+0000","last_active":"2024-02-26T10:42:21.731746+0000","last_peered":"2024-02-26T10:42:21.731746+0000","last_clean":"2024-02-26T10:42:21.731746+0000","last_became_active":"2024-02-26T10:41:59.295266+0000","last_became_peered":"2024-02-26T10:41:59.295266+0000","last_unstale":"2024-02-26T10:42:21.731746+0000","last_undegraded":"2024-02-26T10:42:21.731746+0000","last_fullsized":"2024-02-26T10:42:21.731746+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921041+0000","last_change":"2024-02-26T10:42:20.594928+0000","last_active":"2024-02-26T10:42:21.921041+0000","last_peered":"2024-02-26T10:42:21.921041+0000","last_clean":"2024-02-26T10:42:21.921041+0000","last_became_active":"2024-02-26T10:42:20.594651+0000","last_became_peered":"2024-02-26T10:42:20.594651+0000","last_unstale":"2024-02-26T10:42:21.921041+0000","last_undegraded":"2024-02-26T10:42:21.921041+0000","last_fullsized":"2024-02-26T10:42:21.921041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:53.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.605914+0000","last_change":"2024-02-26T10:41:59.287398+0000","last_active":"2024-02-26T10:42:21.605914+0000","last_peered":"2024-02-26T10:42:21.605914+0000","last_clean":"2024-02-26T10:42:21.605914+0000","last_became_active":"2024-02-26T10:41:59.286305+0000","last_became_peered":"2024-02-26T10:41:59.286305+0000","last_unstale":"2024-02-26T10:42:21.605914+0000","last_undegraded":"2024-02-26T10:42:21.605914+0000","last_fullsized":"2024-02-26T10:42:21.605914+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:54.865254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920442+0000","last_change":"2024-02-26T10:42:20.600066+0000","last_active":"2024-02-26T10:42:21.920442+0000","last_peered":"2024-02-26T10:42:21.920442+0000","last_clean":"2024-02-26T10:42:21.920442+0000","last_became_active":"2024-02-26T10:42:20.599736+0000","last_became_peered":"2024-02-26T10:42:20.599736+0000","last_unstale":"2024-02-26T10:42:21.920442+0000","last_undegraded":"2024-02-26T10:42:21.920442+0000","last_fullsized":"2024-02-26T10:42:21.920442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:08:23.198928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.730946+0000","last_change":"2024-02-26T10:41:59.307667+0000","last_active":"2024-02-26T10:42:21.730946+0000","last_peered":"2024-02-26T10:42:21.730946+0000","last_clean":"2024-02-26T10:42:21.730946+0000","last_became_active":"2024-02-26T10:41:59.307319+0000","last_became_peered":"2024-02-26T10:41:59.307319+0000","last_unstale":"2024-02-26T10:42:21.730946+0000","last_undegraded":"2024-02-26T10:42:21.730946+0000","last_fullsized":"2024-02-26T10:42:21.730946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919840+0000","last_change":"2024-02-26T10:42:20.605813+0000","last_active":"2024-02-26T10:42:21.919840+0000","last_peered":"2024-02-26T10:42:21.919840+0000","last_clean":"2024-02-26T10:42:21.919840+0000","last_became_active":"2024-02-26T10:42:20.605522+0000","last_became_peered":"2024-02-26T10:42:20.605522+0000","last_unstale":"2024-02-26T10:42:21.919840+0000","last_undegraded":"2024-02-26T10:42:21.919840+0000","last_fullsized":"2024-02-26T10:42:21.919840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:28:05.289716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732630+0000","last_change":"2024-02-26T10:41:59.304927+0000","last_active":"2024-02-26T10:42:21.732630+0000","last_peered":"2024-02-26T10:42:21.732630+0000","last_clean":"2024-02-26T10:42:21.732630+0000","last_became_active":"2024-02-26T10:41:59.304641+0000","last_became_peered":"2024-02-26T10:41:59.304641+0000","last_unstale":"2024-02-26T10:42:21.732630+0000","last_undegraded":"2024-02-26T10:42:21.732630+0000","last_fullsized":"2024-02-26T10:42:21.732630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599031+0000","last_change":"2024-02-26T10:42:20.612287+0000","last_active":"2024-02-26T10:42:21.599031+0000","last_peered":"2024-02-26T10:42:21.599031+0000","last_clean":"2024-02-26T10:42:21.599031+0000","last_became_active":"2024-02-26T10:42:20.611835+0000","last_became_peered":"2024-02-26T10:42:20.611835+0000","last_unstale":"2024-02-26T10:42:21.599031+0000","last_undegraded":"2024-02-26T10:42:21.599031+0000","last_fullsized":"2024-02-26T10:42:21.599031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:25:22.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922537+0000","last_change":"2024-02-26T10:41:59.293335+0000","last_active":"2024-02-26T10:42:21.922537+0000","last_peered":"2024-02-26T10:42:21.922537+0000","last_clean":"2024-02-26T10:42:21.922537+0000","last_became_active":"2024-02-26T10:41:59.292937+0000","last_became_peered":"2024-02-26T10:41:59.292937+0000","last_unstale":"2024-02-26T10:42:21.922537+0000","last_undegraded":"2024-02-26T10:42:21.922537+0000","last_fullsized":"2024-02-26T10:42:21.922537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599864+0000","last_change":"2024-02-26T10:42:20.606711+0000","last_active":"2024-02-26T10:42:21.599864+0000","last_peered":"2024-02-26T10:42:21.599864+0000","last_clean":"2024-02-26T10:42:21.599864+0000","last_became_active":"2024-02-26T10:42:20.606355+0000","last_became_peered":"2024-02-26T10:42:20.606355+0000","last_unstale":"2024-02-26T10:42:21.599864+0000","last_undegraded":"2024-02-26T10:42:21.599864+0000","last_fullsized":"2024-02-26T10:42:21.599864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:21.722907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593945+0000","last_change":"2024-02-26T10:41:59.282216+0000","last_active":"2024-02-26T10:42:21.593945+0000","last_peered":"2024-02-26T10:42:21.593945+0000","last_clean":"2024-02-26T10:42:21.593945+0000","last_became_active":"2024-02-26T10:41:59.281891+0000","last_became_peered":"2024-02-26T10:41:59.281891+0000","last_unstale":"2024-02-26T10:42:21.593945+0000","last_undegraded":"2024-02-26T10:42:21.593945+0000","last_fullsized":"2024-02-26T10:42:21.593945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:07.272911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921936+0000","last_change":"2024-02-26T10:42:20.600140+0000","last_active":"2024-02-26T10:42:21.921936+0000","last_peered":"2024-02-26T10:42:21.921936+0000","last_clean":"2024-02-26T10:42:21.921936+0000","last_became_active":"2024-02-26T10:42:20.599862+0000","last_became_peered":"2024-02-26T10:42:20.599862+0000","last_unstale":"2024-02-26T10:42:21.921936+0000","last_undegraded":"2024-02-26T10:42:21.921936+0000","last_fullsized":"2024-02-26T10:42:21.921936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:22.708651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922096+0000","last_change":"2024-02-26T10:41:59.290942+0000","last_active":"2024-02-26T10:42:21.922096+0000","last_peered":"2024-02-26T10:42:21.922096+0000","last_clean":"2024-02-26T10:42:21.922096+0000","last_became_active":"2024-02-26T10:41:59.288616+0000","last_became_peered":"2024-02-26T10:41:59.288616+0000","last_unstale":"2024-02-26T10:42:21.922096+0000","last_undegraded":"2024-02-26T10:42:21.922096+0000","last_fullsized":"2024-02-26T10:42:21.922096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919033+0000","last_change":"2024-02-26T10:42:20.598690+0000","last_active":"2024-02-26T10:42:21.919033+0000","last_peered":"2024-02-26T10:42:21.919033+0000","last_clean":"2024-02-26T10:42:21.919033+0000","last_became_active":"2024-02-26T10:42:20.598374+0000","last_became_peered":"2024-02-26T10:42:20.598374+0000","last_unstale":"2024-02-26T10:42:21.919033+0000","last_undegraded":"2024-02-26T10:42:21.919033+0000","last_fullsized":"2024-02-26T10:42:21.919033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:53.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919776+0000","last_change":"2024-02-26T10:41:59.281911+0000","last_active":"2024-02-26T10:42:21.919776+0000","last_peered":"2024-02-26T10:42:21.919776+0000","last_clean":"2024-02-26T10:42:21.919776+0000","last_became_active":"2024-02-26T10:41:59.281270+0000","last_became_peered":"2024-02-26T10:41:59.281270+0000","last_unstale":"2024-02-26T10:42:21.919776+0000","last_undegraded":"2024-02-26T10:42:21.919776+0000","last_fullsized":"2024-02-26T10:42:21.919776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918527+0000","last_change":"2024-02-26T10:42:20.613254+0000","last_active":"2024-02-26T10:42:21.918527+0000","last_peered":"2024-02-26T10:42:21.918527+0000","last_clean":"2024-02-26T10:42:21.918527+0000","last_became_active":"2024-02-26T10:42:20.612937+0000","last_became_peered":"2024-02-26T10:42:20.612937+0000","last_unstale":"2024-02-26T10:42:21.918527+0000","last_undegraded":"2024-02-26T10:42:21.918527+0000","last_fullsized":"2024-02-26T10:42:21.918527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:00:26.977878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607737+0000","last_change":"2024-02-26T10:41:59.310387+0000","last_active":"2024-02-26T10:42:21.607737+0000","last_peered":"2024-02-26T10:42:21.607737+0000","last_clean":"2024-02-26T10:42:21.607737+0000","last_became_active":"2024-02-26T10:41:59.310111+0000","last_became_peered":"2024-02-26T10:41:59.310111+0000","last_unstale":"2024-02-26T10:42:21.607737+0000","last_undegraded":"2024-02-26T10:42:21.607737+0000","last_fullsized":"2024-02-26T10:42:21.607737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:53:52.167978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920180+0000","last_change":"2024-02-26T10:42:20.605674+0000","last_active":"2024-02-26T10:42:21.920180+0000","last_peered":"2024-02-26T10:42:21.920180+0000","last_clean":"2024-02-26T10:42:21.920180+0000","last_became_active":"2024-02-26T10:42:20.605420+0000","last_became_peered":"2024-02-26T10:42:20.605420+0000","last_unstale":"2024-02-26T10:42:21.920180+0000","last_undegraded":"2024-02-26T10:42:21.920180+0000","last_fullsized":"2024-02-26T10:42:21.920180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:08:23.198928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919181+0000","last_change":"2024-02-26T10:41:59.289328+0000","last_active":"2024-02-26T10:42:21.919181+0000","last_peered":"2024-02-26T10:42:21.919181+0000","last_clean":"2024-02-26T10:42:21.919181+0000","last_became_active":"2024-02-26T10:41:59.288951+0000","last_became_peered":"2024-02-26T10:41:59.288951+0000","last_unstale":"2024-02-26T10:42:21.919181+0000","last_undegraded":"2024-02-26T10:42:21.919181+0000","last_fullsized":"2024-02-26T10:42:21.919181+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:00.613487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607467+0000","last_change":"2024-02-26T10:42:20.592320+0000","last_active":"2024-02-26T10:42:21.607467+0000","last_peered":"2024-02-26T10:42:21.607467+0000","last_clean":"2024-02-26T10:42:21.607467+0000","last_became_active":"2024-02-26T10:42:20.592031+0000","last_became_peered":"2024-02-26T10:42:20.592031+0000","last_unstale":"2024-02-26T10:42:21.607467+0000","last_undegraded":"2024-02-26T10:42:21.607467+0000","last_fullsized":"2024-02-26T10:42:21.607467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:46:48.209177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597600+0000","last_change":"2024-02-26T10:41:59.290536+0000","last_active":"2024-02-26T10:42:21.597600+0000","last_peered":"2024-02-26T10:42:21.597600+0000","last_clean":"2024-02-26T10:42:21.597600+0000","last_became_active":"2024-02-26T10:41:59.289661+0000","last_became_peered":"2024-02-26T10:41:59.289661+0000","last_unstale":"2024-02-26T10:42:21.597600+0000","last_undegraded":"2024-02-26T10:42:21.597600+0000","last_fullsized":"2024-02-26T10:42:21.597600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:56.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606888+0000","last_change":"2024-02-26T10:42:20.600988+0000","last_active":"2024-02-26T10:42:21.606888+0000","last_peered":"2024-02-26T10:42:21.606888+0000","last_clean":"2024-02-26T10:42:21.606888+0000","last_became_active":"2024-02-26T10:42:20.600715+0000","last_became_peered":"2024-02-26T10:42:20.600715+0000","last_unstale":"2024-02-26T10:42:21.606888+0000","last_undegraded":"2024-02-26T10:42:21.606888+0000","last_fullsized":"2024-02-26T10:42:21.606888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:21:38.693700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.600013+0000","last_change":"2024-02-26T10:41:59.291203+0000","last_active":"2024-02-26T10:42:21.600013+0000","last_peered":"2024-02-26T10:42:21.600013+0000","last_clean":"2024-02-26T10:42:21.600013+0000","last_became_active":"2024-02-26T10:41:59.290916+0000","last_became_peered":"2024-02-26T10:41:59.290916+0000","last_unstale":"2024-02-26T10:42:21.600013+0000","last_undegraded":"2024-02-26T10:42:21.600013+0000","last_fullsized":"2024-02-26T10:42:21.600013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:09.397334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598998+0000","last_change":"2024-02-26T10:42:20.598910+0000","last_active":"2024-02-26T10:42:21.598998+0000","last_peered":"2024-02-26T10:42:21.598998+0000","last_clean":"2024-02-26T10:42:21.598998+0000","last_became_active":"2024-02-26T10:42:20.598544+0000","last_became_peered":"2024-02-26T10:42:20.598544+0000","last_unstale":"2024-02-26T10:42:21.598998+0000","last_undegraded":"2024-02-26T10:42:21.598998+0000","last_fullsized":"2024-02-26T10:42:21.598998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:29:17.248481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607437+0000","last_change":"2024-02-26T10:41:59.291282+0000","last_active":"2024-02-26T10:42:21.607437+0000","last_peered":"2024-02-26T10:42:21.607437+0000","last_clean":"2024-02-26T10:42:21.607437+0000","last_became_active":"2024-02-26T10:41:59.291008+0000","last_became_peered":"2024-02-26T10:41:59.291008+0000","last_unstale":"2024-02-26T10:42:21.607437+0000","last_undegraded":"2024-02-26T10:42:21.607437+0000","last_fullsized":"2024-02-26T10:42:21.607437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:55:38.972893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597561+0000","last_change":"2024-02-26T10:42:20.609153+0000","last_active":"2024-02-26T10:42:21.597561+0000","last_peered":"2024-02-26T10:42:21.597561+0000","last_clean":"2024-02-26T10:42:21.597561+0000","last_became_active":"2024-02-26T10:42:20.608837+0000","last_became_peered":"2024-02-26T10:42:20.608837+0000","last_unstale":"2024-02-26T10:42:21.597561+0000","last_undegraded":"2024-02-26T10:42:21.597561+0000","last_fullsized":"2024-02-26T10:42:21.597561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:19:34.963516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597240+0000","last_change":"2024-02-26T10:41:59.289830+0000","last_active":"2024-02-26T10:42:21.597240+0000","last_peered":"2024-02-26T10:42:21.597240+0000","last_clean":"2024-02-26T10:42:21.597240+0000","last_became_active":"2024-02-26T10:41:59.289466+0000","last_became_peered":"2024-02-26T10:41:59.289466+0000","last_unstale":"2024-02-26T10:42:21.597240+0000","last_undegraded":"2024-02-26T10:42:21.597240+0000","last_fullsized":"2024-02-26T10:42:21.597240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920343+0000","last_change":"2024-02-26T10:42:20.596858+0000","last_active":"2024-02-26T10:42:21.920343+0000","last_peered":"2024-02-26T10:42:21.920343+0000","last_clean":"2024-02-26T10:42:21.920343+0000","last_became_active":"2024-02-26T10:42:20.596571+0000","last_became_peered":"2024-02-26T10:42:20.596571+0000","last_unstale":"2024-02-26T10:42:21.920343+0000","last_undegraded":"2024-02-26T10:42:21.920343+0000","last_fullsized":"2024-02-26T10:42:21.920343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606502+0000","last_change":"2024-02-26T10:41:59.294466+0000","last_active":"2024-02-26T10:42:21.606502+0000","last_peered":"2024-02-26T10:42:21.606502+0000","last_clean":"2024-02-26T10:42:21.606502+0000","last_became_active":"2024-02-26T10:41:59.294199+0000","last_became_peered":"2024-02-26T10:41:59.294199+0000","last_unstale":"2024-02-26T10:42:21.606502+0000","last_undegraded":"2024-02-26T10:42:21.606502+0000","last_fullsized":"2024-02-26T10:42:21.606502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:06.679617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595594+0000","last_change":"2024-02-26T10:42:20.609952+0000","last_active":"2024-02-26T10:42:21.595594+0000","last_peered":"2024-02-26T10:42:21.595594+0000","last_clean":"2024-02-26T10:42:21.595594+0000","last_became_active":"2024-02-26T10:42:20.609624+0000","last_became_peered":"2024-02-26T10:42:20.609624+0000","last_unstale":"2024-02-26T10:42:21.595594+0000","last_undegraded":"2024-02-26T10:42:21.595594+0000","last_fullsized":"2024-02-26T10:42:21.595594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:00.537776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606992+0000","last_change":"2024-02-26T10:41:59.302058+0000","last_active":"2024-02-26T10:42:21.606992+0000","last_peered":"2024-02-26T10:42:21.606992+0000","last_clean":"2024-02-26T10:42:21.606992+0000","last_became_active":"2024-02-26T10:41:59.301773+0000","last_became_peered":"2024-02-26T10:41:59.301773+0000","last_unstale":"2024-02-26T10:42:21.606992+0000","last_undegraded":"2024-02-26T10:42:21.606992+0000","last_fullsized":"2024-02-26T10:42:21.606992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:58.489330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919011+0000","last_change":"2024-02-26T10:42:20.605841+0000","last_active":"2024-02-26T10:42:21.919011+0000","last_peered":"2024-02-26T10:42:21.919011+0000","last_clean":"2024-02-26T10:42:21.919011+0000","last_became_active":"2024-02-26T10:42:20.605501+0000","last_became_peered":"2024-02-26T10:42:20.605501+0000","last_unstale":"2024-02-26T10:42:21.919011+0000","last_undegraded":"2024-02-26T10:42:21.919011+0000","last_fullsized":"2024-02-26T10:42:21.919011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:43:19.105024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593520+0000","last_change":"2024-02-26T10:41:59.306729+0000","last_active":"2024-02-26T10:42:21.593520+0000","last_peered":"2024-02-26T10:42:21.593520+0000","last_clean":"2024-02-26T10:42:21.593520+0000","last_became_active":"2024-02-26T10:41:59.306464+0000","last_became_peered":"2024-02-26T10:41:59.306464+0000","last_unstale":"2024-02-26T10:42:21.593520+0000","last_undegraded":"2024-02-26T10:42:21.593520+0000","last_fullsized":"2024-02-26T10:42:21.593520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:56.409926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606312+0000","last_change":"2024-02-26T10:42:20.593902+0000","last_active":"2024-02-26T10:42:21.606312+0000","last_peered":"2024-02-26T10:42:21.606312+0000","last_clean":"2024-02-26T10:42:21.606312+0000","last_became_active":"2024-02-26T10:42:20.592385+0000","last_became_peered":"2024-02-26T10:42:20.592385+0000","last_unstale":"2024-02-26T10:42:21.606312+0000","last_undegraded":"2024-02-26T10:42:21.606312+0000","last_fullsized":"2024-02-26T10:42:21.606312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:07:57.945835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920232+0000","last_change":"2024-02-26T10:41:59.304328+0000","last_active":"2024-02-26T10:42:21.920232+0000","last_peered":"2024-02-26T10:42:21.920232+0000","last_clean":"2024-02-26T10:42:21.920232+0000","last_became_active":"2024-02-26T10:41:59.304025+0000","last_became_peered":"2024-02-26T10:41:59.304025+0000","last_unstale":"2024-02-26T10:42:21.920232+0000","last_undegraded":"2024-02-26T10:42:21.920232+0000","last_fullsized":"2024-02-26T10:42:21.920232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595614+0000","last_change":"2024-02-26T10:42:20.612258+0000","last_active":"2024-02-26T10:42:21.595614+0000","last_peered":"2024-02-26T10:42:21.595614+0000","last_clean":"2024-02-26T10:42:21.595614+0000","last_became_active":"2024-02-26T10:42:20.610215+0000","last_became_peered":"2024-02-26T10:42:20.610215+0000","last_unstale":"2024-02-26T10:42:21.595614+0000","last_undegraded":"2024-02-26T10:42:21.595614+0000","last_fullsized":"2024-02-26T10:42:21.595614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:48.385871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606580+0000","last_change":"2024-02-26T10:41:59.303897+0000","last_active":"2024-02-26T10:42:21.606580+0000","last_peered":"2024-02-26T10:42:21.606580+0000","last_clean":"2024-02-26T10:42:21.606580+0000","last_became_active":"2024-02-26T10:41:59.301812+0000","last_became_peered":"2024-02-26T10:41:59.301812+0000","last_unstale":"2024-02-26T10:42:21.606580+0000","last_undegraded":"2024-02-26T10:42:21.606580+0000","last_fullsized":"2024-02-26T10:42:21.606580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:03:57.549048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920949+0000","last_change":"2024-02-26T10:42:20.608517+0000","last_active":"2024-02-26T10:42:21.920949+0000","last_peered":"2024-02-26T10:42:21.920949+0000","last_clean":"2024-02-26T10:42:21.920949+0000","last_became_active":"2024-02-26T10:42:20.606683+0000","last_became_peered":"2024-02-26T10:42:20.606683+0000","last_unstale":"2024-02-26T10:42:21.920949+0000","last_undegraded":"2024-02-26T10:42:21.920949+0000","last_fullsized":"2024-02-26T10:42:21.920949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:00.537776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594217+0000","last_change":"2024-02-26T10:41:59.297092+0000","last_active":"2024-02-26T10:42:21.594217+0000","last_peered":"2024-02-26T10:42:21.594217+0000","last_clean":"2024-02-26T10:42:21.594217+0000","last_became_active":"2024-02-26T10:41:59.295816+0000","last_became_peered":"2024-02-26T10:41:59.295816+0000","last_unstale":"2024-02-26T10:42:21.594217+0000","last_undegraded":"2024-02-26T10:42:21.594217+0000","last_fullsized":"2024-02-26T10:42:21.594217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:05.502778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922832+0000","last_change":"2024-02-26T10:42:20.594955+0000","last_active":"2024-02-26T10:42:21.922832+0000","last_peered":"2024-02-26T10:42:21.922832+0000","last_clean":"2024-02-26T10:42:21.922832+0000","last_became_active":"2024-02-26T10:42:20.592013+0000","last_became_peered":"2024-02-26T10:42:20.592013+0000","last_unstale":"2024-02-26T10:42:21.922832+0000","last_undegraded":"2024-02-26T10:42:21.922832+0000","last_fullsized":"2024-02-26T10:42:21.922832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:39:31.051666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920890+0000","last_change":"2024-02-26T10:41:59.307675+0000","last_active":"2024-02-26T10:42:21.920890+0000","last_peered":"2024-02-26T10:42:21.920890+0000","last_clean":"2024-02-26T10:42:21.920890+0000","last_became_active":"2024-02-26T10:41:59.307360+0000","last_became_peered":"2024-02-26T10:41:59.307360+0000","last_unstale":"2024-02-26T10:42:21.920890+0000","last_undegraded":"2024-02-26T10:42:21.920890+0000","last_fullsized":"2024-02-26T10:42:21.920890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923737+0000","last_change":"2024-02-26T10:42:20.613331+0000","last_active":"2024-02-26T10:42:21.923737+0000","last_peered":"2024-02-26T10:42:21.923737+0000","last_clean":"2024-02-26T10:42:21.923737+0000","last_became_active":"2024-02-26T10:42:20.613039+0000","last_became_peered":"2024-02-26T10:42:20.613039+0000","last_unstale":"2024-02-26T10:42:21.923737+0000","last_undegraded":"2024-02-26T10:42:21.923737+0000","last_fullsized":"2024-02-26T10:42:21.923737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599200+0000","last_change":"2024-02-26T10:41:59.287990+0000","last_active":"2024-02-26T10:42:21.599200+0000","last_peered":"2024-02-26T10:42:21.599200+0000","last_clean":"2024-02-26T10:42:21.599200+0000","last_became_active":"2024-02-26T10:41:59.286925+0000","last_became_peered":"2024-02-26T10:41:59.286925+0000","last_unstale":"2024-02-26T10:42:21.599200+0000","last_undegraded":"2024-02-26T10:42:21.599200+0000","last_fullsized":"2024-02-26T10:42:21.599200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:08.646623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597644+0000","last_change":"2024-02-26T10:42:20.611088+0000","last_active":"2024-02-26T10:42:21.597644+0000","last_peered":"2024-02-26T10:42:21.597644+0000","last_clean":"2024-02-26T10:42:21.597644+0000","last_became_active":"2024-02-26T10:42:20.610791+0000","last_became_peered":"2024-02-26T10:42:20.610791+0000","last_unstale":"2024-02-26T10:42:21.597644+0000","last_undegraded":"2024-02-26T10:42:21.597644+0000","last_fullsized":"2024-02-26T10:42:21.597644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:46.312779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922263+0000","last_change":"2024-02-26T10:41:59.297459+0000","last_active":"2024-02-26T10:42:21.922263+0000","last_peered":"2024-02-26T10:42:21.922263+0000","last_clean":"2024-02-26T10:42:21.922263+0000","last_became_active":"2024-02-26T10:41:59.296494+0000","last_became_peered":"2024-02-26T10:41:59.296494+0000","last_unstale":"2024-02-26T10:42:21.922263+0000","last_undegraded":"2024-02-26T10:42:21.922263+0000","last_fullsized":"2024-02-26T10:42:21.922263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919427+0000","last_change":"2024-02-26T10:42:20.587485+0000","last_active":"2024-02-26T10:42:21.919427+0000","last_peered":"2024-02-26T10:42:21.919427+0000","last_clean":"2024-02-26T10:42:21.919427+0000","last_became_active":"2024-02-26T10:42:20.587190+0000","last_became_peered":"2024-02-26T10:42:20.587190+0000","last_unstale":"2024-02-26T10:42:21.919427+0000","last_undegraded":"2024-02-26T10:42:21.919427+0000","last_fullsized":"2024-02-26T10:42:21.919427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:28:05.289716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593987+0000","last_change":"2024-02-26T10:41:59.288740+0000","last_active":"2024-02-26T10:42:21.593987+0000","last_peered":"2024-02-26T10:42:21.593987+0000","last_clean":"2024-02-26T10:42:21.593987+0000","last_became_active":"2024-02-26T10:41:59.288433+0000","last_became_peered":"2024-02-26T10:41:59.288433+0000","last_unstale":"2024-02-26T10:42:21.593987+0000","last_undegraded":"2024-02-26T10:42:21.593987+0000","last_fullsized":"2024-02-26T10:42:21.593987+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:21.766000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.592835+0000","last_change":"2024-02-26T10:42:20.614242+0000","last_active":"2024-02-26T10:42:21.592835+0000","last_peered":"2024-02-26T10:42:21.592835+0000","last_clean":"2024-02-26T10:42:21.592835+0000","last_became_active":"2024-02-26T10:42:20.613805+0000","last_became_peered":"2024-02-26T10:42:20.613805+0000","last_unstale":"2024-02-26T10:42:21.592835+0000","last_undegraded":"2024-02-26T10:42:21.592835+0000","last_fullsized":"2024-02-26T10:42:21.592835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:57.832401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593626+0000","last_change":"2024-02-26T10:41:59.293103+0000","last_active":"2024-02-26T10:42:21.593626+0000","last_peered":"2024-02-26T10:42:21.593626+0000","last_clean":"2024-02-26T10:42:21.593626+0000","last_became_active":"2024-02-26T10:41:59.292473+0000","last_became_peered":"2024-02-26T10:41:59.292473+0000","last_unstale":"2024-02-26T10:42:21.593626+0000","last_undegraded":"2024-02-26T10:42:21.593626+0000","last_fullsized":"2024-02-26T10:42:21.593626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:37.822421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596457+0000","last_change":"2024-02-26T10:42:20.611052+0000","last_active":"2024-02-26T10:42:21.596457+0000","last_peered":"2024-02-26T10:42:21.596457+0000","last_clean":"2024-02-26T10:42:21.596457+0000","last_became_active":"2024-02-26T10:42:20.610750+0000","last_became_peered":"2024-02-26T10:42:20.610750+0000","last_unstale":"2024-02-26T10:42:21.596457+0000","last_undegraded":"2024-02-26T10:42:21.596457+0000","last_fullsized":"2024-02-26T10:42:21.596457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:36:08.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597980+0000","last_change":"2024-02-26T10:41:59.294484+0000","last_active":"2024-02-26T10:42:21.597980+0000","last_peered":"2024-02-26T10:42:21.597980+0000","last_clean":"2024-02-26T10:42:21.597980+0000","last_became_active":"2024-02-26T10:41:59.293628+0000","last_became_peered":"2024-02-26T10:41:59.293628+0000","last_unstale":"2024-02-26T10:42:21.597980+0000","last_undegraded":"2024-02-26T10:42:21.597980+0000","last_fullsized":"2024-02-26T10:42:21.597980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:04.694202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.608162+0000","last_change":"2024-02-26T10:42:20.603849+0000","last_active":"2024-02-26T10:42:21.608162+0000","last_peered":"2024-02-26T10:42:21.608162+0000","last_clean":"2024-02-26T10:42:21.608162+0000","last_became_active":"2024-02-26T10:42:20.602569+0000","last_became_peered":"2024-02-26T10:42:20.602569+0000","last_unstale":"2024-02-26T10:42:21.608162+0000","last_undegraded":"2024-02-26T10:42:21.608162+0000","last_fullsized":"2024-02-26T10:42:21.608162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:47:28.179744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918698+0000","last_change":"2024-02-26T10:41:59.289588+0000","last_active":"2024-02-26T10:42:21.918698+0000","last_peered":"2024-02-26T10:42:21.918698+0000","last_clean":"2024-02-26T10:42:21.918698+0000","last_became_active":"2024-02-26T10:41:59.289223+0000","last_became_peered":"2024-02-26T10:41:59.289223+0000","last_unstale":"2024-02-26T10:42:21.918698+0000","last_undegraded":"2024-02-26T10:42:21.918698+0000","last_fullsized":"2024-02-26T10:42:21.918698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594035+0000","last_change":"2024-02-26T10:42:20.609260+0000","last_active":"2024-02-26T10:42:21.594035+0000","last_peered":"2024-02-26T10:42:21.594035+0000","last_clean":"2024-02-26T10:42:21.594035+0000","last_became_active":"2024-02-26T10:42:20.608926+0000","last_became_peered":"2024-02-26T10:42:20.608926+0000","last_unstale":"2024-02-26T10:42:21.594035+0000","last_undegraded":"2024-02-26T10:42:21.594035+0000","last_fullsized":"2024-02-26T10:42:21.594035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:49.006673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597323+0000","last_change":"2024-02-26T10:42:19.567980+0000","last_active":"2024-02-26T10:42:21.597323+0000","last_peered":"2024-02-26T10:42:21.597323+0000","last_clean":"2024-02-26T10:42:21.597323+0000","last_became_active":"2024-02-26T10:42:19.567535+0000","last_became_peered":"2024-02-26T10:42:19.567535+0000","last_unstale":"2024-02-26T10:42:21.597323+0000","last_undegraded":"2024-02-26T10:42:21.597323+0000","last_fullsized":"2024-02-26T10:42:21.597323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:20.708487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599467+0000","last_change":"2024-02-26T10:41:59.296979+0000","last_active":"2024-02-26T10:42:21.599467+0000","last_peered":"2024-02-26T10:42:21.599467+0000","last_clean":"2024-02-26T10:42:21.599467+0000","last_became_active":"2024-02-26T10:41:59.295612+0000","last_became_peered":"2024-02-26T10:41:59.295612+0000","last_unstale":"2024-02-26T10:42:21.599467+0000","last_undegraded":"2024-02-26T10:42:21.599467+0000","last_fullsized":"2024-02-26T10:42:21.599467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:53.564566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"75'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923351+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:42:21.923351+0000","last_peered":"2024-02-26T10:42:21.923351+0000","last_clean":"2024-02-26T10:42:21.923351+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:42:21.923351+0000","last_undegraded":"2024-02-26T10:42:21.923351+0000","last_fullsized":"2024-02-26T10:42:21.923351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:03.513627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599200+0000","last_change":"2024-02-26T10:42:20.604806+0000","last_active":"2024-02-26T10:42:21.599200+0000","last_peered":"2024-02-26T10:42:21.599200+0000","last_clean":"2024-02-26T10:42:21.599200+0000","last_became_active":"2024-02-26T10:42:20.603712+0000","last_became_peered":"2024-02-26T10:42:20.603712+0000","last_unstale":"2024-02-26T10:42:21.599200+0000","last_undegraded":"2024-02-26T10:42:21.599200+0000","last_fullsized":"2024-02-26T10:42:21.599200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:04.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596571+0000","last_change":"2024-02-26T10:42:19.571532+0000","last_active":"2024-02-26T10:42:21.596571+0000","last_peered":"2024-02-26T10:42:21.596571+0000","last_clean":"2024-02-26T10:42:21.596571+0000","last_became_active":"2024-02-26T10:42:19.570733+0000","last_became_peered":"2024-02-26T10:42:19.570733+0000","last_unstale":"2024-02-26T10:42:21.596571+0000","last_undegraded":"2024-02-26T10:42:21.596571+0000","last_fullsized":"2024-02-26T10:42:21.596571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:15:50.195046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923169+0000","last_change":"2024-02-26T10:41:59.304661+0000","last_active":"2024-02-26T10:42:21.923169+0000","last_peered":"2024-02-26T10:42:21.923169+0000","last_clean":"2024-02-26T10:42:21.923169+0000","last_became_active":"2024-02-26T10:41:59.304358+0000","last_became_peered":"2024-02-26T10:41:59.304358+0000","last_unstale":"2024-02-26T10:42:21.923169+0000","last_undegraded":"2024-02-26T10:42:21.923169+0000","last_fullsized":"2024-02-26T10:42:21.923169+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599815+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:42:21.599815+0000","last_peered":"2024-02-26T10:42:21.599815+0000","last_clean":"2024-02-26T10:42:21.599815+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:42:21.599815+0000","last_undegraded":"2024-02-26T10:42:21.599815+0000","last_fullsized":"2024-02-26T10:42:21.599815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:07.218989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607028+0000","last_change":"2024-02-26T10:42:20.602128+0000","last_active":"2024-02-26T10:42:21.607028+0000","last_peered":"2024-02-26T10:42:21.607028+0000","last_clean":"2024-02-26T10:42:21.607028+0000","last_became_active":"2024-02-26T10:42:20.601767+0000","last_became_peered":"2024-02-26T10:42:20.601767+0000","last_unstale":"2024-02-26T10:42:21.607028+0000","last_undegraded":"2024-02-26T10:42:21.607028+0000","last_fullsized":"2024-02-26T10:42:21.607028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:04.038444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"79'4","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597502+0000","last_change":"2024-02-26T10:42:19.566680+0000","last_active":"2024-02-26T10:42:21.597502+0000","last_peered":"2024-02-26T10:42:21.597502+0000","last_clean":"2024-02-26T10:42:21.597502+0000","last_became_active":"2024-02-26T10:42:19.566472+0000","last_became_peered":"2024-02-26T10:42:19.566472+0000","last_unstale":"2024-02-26T10:42:21.597502+0000","last_undegraded":"2024-02-26T10:42:21.597502+0000","last_fullsized":"2024-02-26T10:42:21.597502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:48:11.192552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922577+0000","last_change":"2024-02-26T10:41:59.310348+0000","last_active":"2024-02-26T10:42:21.922577+0000","last_peered":"2024-02-26T10:42:21.922577+0000","last_clean":"2024-02-26T10:42:21.922577+0000","last_became_active":"2024-02-26T10:41:59.309976+0000","last_became_peered":"2024-02-26T10:41:59.309976+0000","last_unstale":"2024-02-26T10:42:21.922577+0000","last_undegraded":"2024-02-26T10:42:21.922577+0000","last_fullsized":"2024-02-26T10:42:21.922577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597391+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:42:21.597391+0000","last_peered":"2024-02-26T10:42:21.597391+0000","last_clean":"2024-02-26T10:42:21.597391+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:42:21.597391+0000","last_undegraded":"2024-02-26T10:42:21.597391+0000","last_fullsized":"2024-02-26T10:42:21.597391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:15.932239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920454+0000","last_change":"2024-02-26T10:42:20.589965+0000","last_active":"2024-02-26T10:42:21.920454+0000","last_peered":"2024-02-26T10:42:21.920454+0000","last_clean":"2024-02-26T10:42:21.920454+0000","last_became_active":"2024-02-26T10:42:20.589576+0000","last_became_peered":"2024-02-26T10:42:20.589576+0000","last_unstale":"2024-02-26T10:42:21.920454+0000","last_undegraded":"2024-02-26T10:42:21.920454+0000","last_fullsized":"2024-02-26T10:42:21.920454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:00.537776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921464+0000","last_change":"2024-02-26T10:42:19.572136+0000","last_active":"2024-02-26T10:42:21.921464+0000","last_peered":"2024-02-26T10:42:21.921464+0000","last_clean":"2024-02-26T10:42:21.921464+0000","last_became_active":"2024-02-26T10:42:19.571490+0000","last_became_peered":"2024-02-26T10:42:19.571490+0000","last_unstale":"2024-02-26T10:42:21.921464+0000","last_undegraded":"2024-02-26T10:42:21.921464+0000","last_fullsized":"2024-02-26T10:42:21.921464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:20.708487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920789+0000","last_change":"2024-02-26T10:41:59.287952+0000","last_active":"2024-02-26T10:42:21.920789+0000","last_peered":"2024-02-26T10:42:21.920789+0000","last_clean":"2024-02-26T10:42:21.920789+0000","last_became_active":"2024-02-26T10:41:59.287395+0000","last_became_peered":"2024-02-26T10:41:59.287395+0000","last_unstale":"2024-02-26T10:42:21.920789+0000","last_undegraded":"2024-02-26T10:42:21.920789+0000","last_fullsized":"2024-02-26T10:42:21.920789+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:54.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596900+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:42:21.596900+0000","last_peered":"2024-02-26T10:42:21.596900+0000","last_clean":"2024-02-26T10:42:21.596900+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:42:21.596900+0000","last_undegraded":"2024-02-26T10:42:21.596900+0000","last_fullsized":"2024-02-26T10:42:21.596900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919224+0000","last_change":"2024-02-26T10:42:20.600458+0000","last_active":"2024-02-26T10:42:21.919224+0000","last_peered":"2024-02-26T10:42:21.919224+0000","last_clean":"2024-02-26T10:42:21.919224+0000","last_became_active":"2024-02-26T10:42:20.600119+0000","last_became_peered":"2024-02-26T10:42:20.600119+0000","last_unstale":"2024-02-26T10:42:21.919224+0000","last_undegraded":"2024-02-26T10:42:21.919224+0000","last_fullsized":"2024-02-26T10:42:21.919224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:43:19.105024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920102+0000","last_change":"2024-02-26T10:42:19.568599+0000","last_active":"2024-02-26T10:42:21.920102+0000","last_peered":"2024-02-26T10:42:21.920102+0000","last_clean":"2024-02-26T10:42:21.920102+0000","last_became_active":"2024-02-26T10:42:19.568102+0000","last_became_peered":"2024-02-26T10:42:19.568102+0000","last_unstale":"2024-02-26T10:42:21.920102+0000","last_undegraded":"2024-02-26T10:42:21.920102+0000","last_fullsized":"2024-02-26T10:42:21.920102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:31.472935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593727+0000","last_change":"2024-02-26T10:41:59.292771+0000","last_active":"2024-02-26T10:42:21.593727+0000","last_peered":"2024-02-26T10:42:21.593727+0000","last_clean":"2024-02-26T10:42:21.593727+0000","last_became_active":"2024-02-26T10:41:59.288778+0000","last_became_peered":"2024-02-26T10:41:59.288778+0000","last_unstale":"2024-02-26T10:42:21.593727+0000","last_undegraded":"2024-02-26T10:42:21.593727+0000","last_fullsized":"2024-02-26T10:42:21.593727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:18.908425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598899+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:42:21.598899+0000","last_peered":"2024-02-26T10:42:21.598899+0000","last_clean":"2024-02-26T10:42:21.598899+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:42:21.598899+0000","last_undegraded":"2024-02-26T10:42:21.598899+0000","last_fullsized":"2024-02-26T10:42:21.598899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:20.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597947+0000","last_change":"2024-02-26T10:42:20.601343+0000","last_active":"2024-02-26T10:42:21.597947+0000","last_peered":"2024-02-26T10:42:21.597947+0000","last_clean":"2024-02-26T10:42:21.597947+0000","last_became_active":"2024-02-26T10:42:20.601093+0000","last_became_peered":"2024-02-26T10:42:20.601093+0000","last_unstale":"2024-02-26T10:42:21.597947+0000","last_undegraded":"2024-02-26T10:42:21.597947+0000","last_fullsized":"2024-02-26T10:42:21.597947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:49.653558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.592859+0000","last_change":"2024-02-26T10:42:19.552098+0000","last_active":"2024-02-26T10:42:21.592859+0000","last_peered":"2024-02-26T10:42:21.592859+0000","last_clean":"2024-02-26T10:42:21.592859+0000","last_became_active":"2024-02-26T10:42:19.551763+0000","last_became_peered":"2024-02-26T10:42:19.551763+0000","last_unstale":"2024-02-26T10:42:21.592859+0000","last_undegraded":"2024-02-26T10:42:21.592859+0000","last_fullsized":"2024-02-26T10:42:21.592859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:37.228234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598678+0000","last_change":"2024-02-26T10:41:59.287430+0000","last_active":"2024-02-26T10:42:21.598678+0000","last_peered":"2024-02-26T10:42:21.598678+0000","last_clean":"2024-02-26T10:42:21.598678+0000","last_became_active":"2024-02-26T10:41:59.287010+0000","last_became_peered":"2024-02-26T10:41:59.287010+0000","last_unstale":"2024-02-26T10:42:21.598678+0000","last_undegraded":"2024-02-26T10:42:21.598678+0000","last_fullsized":"2024-02-26T10:42:21.598678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:24.065359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597856+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:42:21.597856+0000","last_peered":"2024-02-26T10:42:21.597856+0000","last_clean":"2024-02-26T10:42:21.597856+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:42:21.597856+0000","last_undegraded":"2024-02-26T10:42:21.597856+0000","last_fullsized":"2024-02-26T10:42:21.597856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:19.393230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732551+0000","last_change":"2024-02-26T10:42:20.604589+0000","last_active":"2024-02-26T10:42:21.732551+0000","last_peered":"2024-02-26T10:42:21.732551+0000","last_clean":"2024-02-26T10:42:21.732551+0000","last_became_active":"2024-02-26T10:42:20.604291+0000","last_became_peered":"2024-02-26T10:42:20.604291+0000","last_unstale":"2024-02-26T10:42:21.732551+0000","last_undegraded":"2024-02-26T10:42:21.732551+0000","last_fullsized":"2024-02-26T10:42:21.732551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:21.722907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593560+0000","last_change":"2024-02-26T10:42:19.566766+0000","last_active":"2024-02-26T10:42:21.593560+0000","last_peered":"2024-02-26T10:42:21.593560+0000","last_clean":"2024-02-26T10:42:21.593560+0000","last_became_active":"2024-02-26T10:42:19.566484+0000","last_became_peered":"2024-02-26T10:42:19.566484+0000","last_unstale":"2024-02-26T10:42:21.593560+0000","last_undegraded":"2024-02-26T10:42:21.593560+0000","last_fullsized":"2024-02-26T10:42:21.593560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:44.830713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918662+0000","last_change":"2024-02-26T10:41:59.301966+0000","last_active":"2024-02-26T10:42:21.918662+0000","last_peered":"2024-02-26T10:42:21.918662+0000","last_clean":"2024-02-26T10:42:21.918662+0000","last_became_active":"2024-02-26T10:41:59.301282+0000","last_became_peered":"2024-02-26T10:41:59.301282+0000","last_unstale":"2024-02-26T10:42:21.918662+0000","last_undegraded":"2024-02-26T10:42:21.918662+0000","last_fullsized":"2024-02-26T10:42:21.918662+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595885+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:42:21.595885+0000","last_peered":"2024-02-26T10:42:21.595885+0000","last_clean":"2024-02-26T10:42:21.595885+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:42:21.595885+0000","last_undegraded":"2024-02-26T10:42:21.595885+0000","last_fullsized":"2024-02-26T10:42:21.595885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:07.631703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920116+0000","last_change":"2024-02-26T10:42:20.604569+0000","last_active":"2024-02-26T10:42:21.920116+0000","last_peered":"2024-02-26T10:42:21.920116+0000","last_clean":"2024-02-26T10:42:21.920116+0000","last_became_active":"2024-02-26T10:42:20.603190+0000","last_became_peered":"2024-02-26T10:42:20.603190+0000","last_unstale":"2024-02-26T10:42:21.920116+0000","last_undegraded":"2024-02-26T10:42:21.920116+0000","last_fullsized":"2024-02-26T10:42:21.920116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:39:31.051666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595740+0000","last_change":"2024-02-26T10:42:19.572424+0000","last_active":"2024-02-26T10:42:21.595740+0000","last_peered":"2024-02-26T10:42:21.595740+0000","last_clean":"2024-02-26T10:42:21.595740+0000","last_became_active":"2024-02-26T10:42:19.572019+0000","last_became_peered":"2024-02-26T10:42:19.572019+0000","last_unstale":"2024-02-26T10:42:21.595740+0000","last_undegraded":"2024-02-26T10:42:21.595740+0000","last_fullsized":"2024-02-26T10:42:21.595740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:49.405962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919505+0000","last_change":"2024-02-26T10:41:59.289078+0000","last_active":"2024-02-26T10:42:21.919505+0000","last_peered":"2024-02-26T10:42:21.919505+0000","last_clean":"2024-02-26T10:42:21.919505+0000","last_became_active":"2024-02-26T10:41:59.288529+0000","last_became_peered":"2024-02-26T10:41:59.288529+0000","last_unstale":"2024-02-26T10:42:21.919505+0000","last_undegraded":"2024-02-26T10:42:21.919505+0000","last_fullsized":"2024-02-26T10:42:21.919505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:28.358063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593343+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:42:21.593343+0000","last_peered":"2024-02-26T10:42:21.593343+0000","last_clean":"2024-02-26T10:42:21.593343+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:42:21.593343+0000","last_undegraded":"2024-02-26T10:42:21.593343+0000","last_fullsized":"2024-02-26T10:42:21.593343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:56.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921892+0000","last_change":"2024-02-26T10:42:20.613586+0000","last_active":"2024-02-26T10:42:21.921892+0000","last_peered":"2024-02-26T10:42:21.921892+0000","last_clean":"2024-02-26T10:42:21.921892+0000","last_became_active":"2024-02-26T10:42:20.612911+0000","last_became_peered":"2024-02-26T10:42:20.612911+0000","last_unstale":"2024-02-26T10:42:21.921892+0000","last_undegraded":"2024-02-26T10:42:21.921892+0000","last_fullsized":"2024-02-26T10:42:21.921892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:28:05.289716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"79'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921212+0000","last_change":"2024-02-26T10:42:19.571927+0000","last_active":"2024-02-26T10:42:21.921212+0000","last_peered":"2024-02-26T10:42:21.921212+0000","last_clean":"2024-02-26T10:42:21.921212+0000","last_became_active":"2024-02-26T10:42:19.570459+0000","last_became_peered":"2024-02-26T10:42:19.570459+0000","last_unstale":"2024-02-26T10:42:21.921212+0000","last_undegraded":"2024-02-26T10:42:21.921212+0000","last_fullsized":"2024-02-26T10:42:21.921212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:28:55.454669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920536+0000","last_change":"2024-02-26T10:41:59.294420+0000","last_active":"2024-02-26T10:42:21.920536+0000","last_peered":"2024-02-26T10:42:21.920536+0000","last_clean":"2024-02-26T10:42:21.920536+0000","last_became_active":"2024-02-26T10:41:59.294147+0000","last_became_peered":"2024-02-26T10:41:59.294147+0000","last_unstale":"2024-02-26T10:42:21.920536+0000","last_undegraded":"2024-02-26T10:42:21.920536+0000","last_fullsized":"2024-02-26T10:42:21.920536+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:32.774644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923542+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:42:21.923542+0000","last_peered":"2024-02-26T10:42:21.923542+0000","last_clean":"2024-02-26T10:42:21.923542+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:42:21.923542+0000","last_undegraded":"2024-02-26T10:42:21.923542+0000","last_fullsized":"2024-02-26T10:42:21.923542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920057+0000","last_change":"2024-02-26T10:42:20.614388+0000","last_active":"2024-02-26T10:42:21.920057+0000","last_peered":"2024-02-26T10:42:21.920057+0000","last_clean":"2024-02-26T10:42:21.920057+0000","last_became_active":"2024-02-26T10:42:20.614082+0000","last_became_peered":"2024-02-26T10:42:20.614082+0000","last_unstale":"2024-02-26T10:42:21.920057+0000","last_undegraded":"2024-02-26T10:42:21.920057+0000","last_fullsized":"2024-02-26T10:42:21.920057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:22.708651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"79'6","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599338+0000","last_change":"2024-02-26T10:42:19.552022+0000","last_active":"2024-02-26T10:42:21.599338+0000","last_peered":"2024-02-26T10:42:21.599338+0000","last_clean":"2024-02-26T10:42:21.599338+0000","last_became_active":"2024-02-26T10:42:19.551436+0000","last_became_peered":"2024-02-26T10:42:19.551436+0000","last_unstale":"2024-02-26T10:42:21.599338+0000","last_undegraded":"2024-02-26T10:42:21.599338+0000","last_fullsized":"2024-02-26T10:42:21.599338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:49.253673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732649+0000","last_change":"2024-02-26T10:41:59.296775+0000","last_active":"2024-02-26T10:42:21.732649+0000","last_peered":"2024-02-26T10:42:21.732649+0000","last_clean":"2024-02-26T10:42:21.732649+0000","last_became_active":"2024-02-26T10:41:59.296431+0000","last_became_peered":"2024-02-26T10:41:59.296431+0000","last_unstale":"2024-02-26T10:42:21.732649+0000","last_undegraded":"2024-02-26T10:42:21.732649+0000","last_fullsized":"2024-02-26T10:42:21.732649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922609+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:42:21.922609+0000","last_peered":"2024-02-26T10:42:21.922609+0000","last_clean":"2024-02-26T10:42:21.922609+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:42:21.922609+0000","last_undegraded":"2024-02-26T10:42:21.922609+0000","last_fullsized":"2024-02-26T10:42:21.922609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:04.487859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596945+0000","last_change":"2024-02-26T10:42:20.613657+0000","last_active":"2024-02-26T10:42:21.596945+0000","last_peered":"2024-02-26T10:42:21.596945+0000","last_clean":"2024-02-26T10:42:21.596945+0000","last_became_active":"2024-02-26T10:42:20.613399+0000","last_became_peered":"2024-02-26T10:42:20.613399+0000","last_unstale":"2024-02-26T10:42:21.596945+0000","last_undegraded":"2024-02-26T10:42:21.596945+0000","last_fullsized":"2024-02-26T10:42:21.596945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:04.830480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594108+0000","last_change":"2024-02-26T10:42:19.562582+0000","last_active":"2024-02-26T10:42:21.594108+0000","last_peered":"2024-02-26T10:42:21.594108+0000","last_clean":"2024-02-26T10:42:21.594108+0000","last_became_active":"2024-02-26T10:42:19.562254+0000","last_became_peered":"2024-02-26T10:42:19.562254+0000","last_unstale":"2024-02-26T10:42:21.594108+0000","last_undegraded":"2024-02-26T10:42:21.594108+0000","last_fullsized":"2024-02-26T10:42:21.594108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:47.203598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596753+0000","last_change":"2024-02-26T10:41:59.300848+0000","last_active":"2024-02-26T10:42:21.596753+0000","last_peered":"2024-02-26T10:42:21.596753+0000","last_clean":"2024-02-26T10:42:21.596753+0000","last_became_active":"2024-02-26T10:41:59.300577+0000","last_became_peered":"2024-02-26T10:41:59.300577+0000","last_unstale":"2024-02-26T10:42:21.596753+0000","last_undegraded":"2024-02-26T10:42:21.596753+0000","last_fullsized":"2024-02-26T10:42:21.596753+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"75'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921964+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:42:21.921964+0000","last_peered":"2024-02-26T10:42:21.921964+0000","last_clean":"2024-02-26T10:42:21.921964+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:42:21.921964+0000","last_undegraded":"2024-02-26T10:42:21.921964+0000","last_fullsized":"2024-02-26T10:42:21.921964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599987+0000","last_change":"2024-02-26T10:42:20.602702+0000","last_active":"2024-02-26T10:42:21.599987+0000","last_peered":"2024-02-26T10:42:21.599987+0000","last_clean":"2024-02-26T10:42:21.599987+0000","last_became_active":"2024-02-26T10:42:20.602103+0000","last_became_peered":"2024-02-26T10:42:20.602103+0000","last_unstale":"2024-02-26T10:42:21.599987+0000","last_undegraded":"2024-02-26T10:42:21.599987+0000","last_fullsized":"2024-02-26T10:42:21.599987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:32.487354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919802+0000","last_change":"2024-02-26T10:42:19.564508+0000","last_active":"2024-02-26T10:42:21.919802+0000","last_peered":"2024-02-26T10:42:21.919802+0000","last_clean":"2024-02-26T10:42:21.919802+0000","last_became_active":"2024-02-26T10:42:19.564189+0000","last_became_peered":"2024-02-26T10:42:19.564189+0000","last_unstale":"2024-02-26T10:42:21.919802+0000","last_undegraded":"2024-02-26T10:42:21.919802+0000","last_fullsized":"2024-02-26T10:42:21.919802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:15:50.195046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599487+0000","last_change":"2024-02-26T10:41:59.303988+0000","last_active":"2024-02-26T10:42:21.599487+0000","last_peered":"2024-02-26T10:42:21.599487+0000","last_clean":"2024-02-26T10:42:21.599487+0000","last_became_active":"2024-02-26T10:41:59.303677+0000","last_became_peered":"2024-02-26T10:41:59.303677+0000","last_unstale":"2024-02-26T10:42:21.599487+0000","last_undegraded":"2024-02-26T10:42:21.599487+0000","last_fullsized":"2024-02-26T10:42:21.599487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:08.404204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"76'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732288+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:42:21.732288+0000","last_peered":"2024-02-26T10:42:21.732288+0000","last_clean":"2024-02-26T10:42:21.732288+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:42:21.732288+0000","last_undegraded":"2024-02-26T10:42:21.732288+0000","last_fullsized":"2024-02-26T10:42:21.732288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3115,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597726+0000","last_change":"2024-02-26T10:42:20.601327+0000","last_active":"2024-02-26T10:42:21.597726+0000","last_peered":"2024-02-26T10:42:21.597726+0000","last_clean":"2024-02-26T10:42:21.597726+0000","last_became_active":"2024-02-26T10:42:20.601024+0000","last_became_peered":"2024-02-26T10:42:20.601024+0000","last_unstale":"2024-02-26T10:42:21.597726+0000","last_undegraded":"2024-02-26T10:42:21.597726+0000","last_fullsized":"2024-02-26T10:42:21.597726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:21.908982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597832+0000","last_change":"2024-02-26T10:42:19.568035+0000","last_active":"2024-02-26T10:42:21.597832+0000","last_peered":"2024-02-26T10:42:21.597832+0000","last_clean":"2024-02-26T10:42:21.597832+0000","last_became_active":"2024-02-26T10:42:19.566434+0000","last_became_peered":"2024-02-26T10:42:19.566434+0000","last_unstale":"2024-02-26T10:42:21.597832+0000","last_undegraded":"2024-02-26T10:42:21.597832+0000","last_fullsized":"2024-02-26T10:42:21.597832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:56.817981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595923+0000","last_change":"2024-02-26T10:41:59.298661+0000","last_active":"2024-02-26T10:42:21.595923+0000","last_peered":"2024-02-26T10:42:21.595923+0000","last_clean":"2024-02-26T10:42:21.595923+0000","last_became_active":"2024-02-26T10:41:59.298384+0000","last_became_peered":"2024-02-26T10:41:59.298384+0000","last_unstale":"2024-02-26T10:42:21.595923+0000","last_undegraded":"2024-02-26T10:42:21.595923+0000","last_fullsized":"2024-02-26T10:42:21.595923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594203+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:42:21.594203+0000","last_peered":"2024-02-26T10:42:21.594203+0000","last_clean":"2024-02-26T10:42:21.594203+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:42:21.594203+0000","last_undegraded":"2024-02-26T10:42:21.594203+0000","last_fullsized":"2024-02-26T10:42:21.594203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:06.345056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.605948+0000","last_change":"2024-02-26T10:42:20.606489+0000","last_active":"2024-02-26T10:42:21.605948+0000","last_peered":"2024-02-26T10:42:21.605948+0000","last_clean":"2024-02-26T10:42:21.605948+0000","last_became_active":"2024-02-26T10:42:20.606100+0000","last_became_peered":"2024-02-26T10:42:20.606100+0000","last_unstale":"2024-02-26T10:42:21.605948+0000","last_undegraded":"2024-02-26T10:42:21.605948+0000","last_fullsized":"2024-02-26T10:42:21.605948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:38:30.603159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"79'8","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593114+0000","last_change":"2024-02-26T10:42:19.568564+0000","last_active":"2024-02-26T10:42:21.593114+0000","last_peered":"2024-02-26T10:42:21.593114+0000","last_clean":"2024-02-26T10:42:21.593114+0000","last_became_active":"2024-02-26T10:42:19.568278+0000","last_became_peered":"2024-02-26T10:42:19.568278+0000","last_unstale":"2024-02-26T10:42:21.593114+0000","last_undegraded":"2024-02-26T10:42:21.593114+0000","last_fullsized":"2024-02-26T10:42:21.593114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:51:20.739806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920052+0000","last_change":"2024-02-26T10:41:59.303009+0000","last_active":"2024-02-26T10:42:21.920052+0000","last_peered":"2024-02-26T10:42:21.920052+0000","last_clean":"2024-02-26T10:42:21.920052+0000","last_became_active":"2024-02-26T10:41:59.301703+0000","last_became_peered":"2024-02-26T10:41:59.301703+0000","last_unstale":"2024-02-26T10:42:21.920052+0000","last_undegraded":"2024-02-26T10:42:21.920052+0000","last_fullsized":"2024-02-26T10:42:21.920052+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919878+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:42:21.919878+0000","last_peered":"2024-02-26T10:42:21.919878+0000","last_clean":"2024-02-26T10:42:21.919878+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:42:21.919878+0000","last_undegraded":"2024-02-26T10:42:21.919878+0000","last_fullsized":"2024-02-26T10:42:21.919878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:15.119340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919738+0000","last_change":"2024-02-26T10:42:20.603915+0000","last_active":"2024-02-26T10:42:21.919738+0000","last_peered":"2024-02-26T10:42:21.919738+0000","last_clean":"2024-02-26T10:42:21.919738+0000","last_became_active":"2024-02-26T10:42:20.602756+0000","last_became_peered":"2024-02-26T10:42:20.602756+0000","last_unstale":"2024-02-26T10:42:21.919738+0000","last_undegraded":"2024-02-26T10:42:21.919738+0000","last_fullsized":"2024-02-26T10:42:21.919738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:22.708651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606964+0000","last_change":"2024-02-26T10:42:19.553678+0000","last_active":"2024-02-26T10:42:21.606964+0000","last_peered":"2024-02-26T10:42:21.606964+0000","last_clean":"2024-02-26T10:42:21.606964+0000","last_became_active":"2024-02-26T10:42:19.552678+0000","last_became_peered":"2024-02-26T10:42:19.552678+0000","last_unstale":"2024-02-26T10:42:21.606964+0000","last_undegraded":"2024-02-26T10:42:21.606964+0000","last_fullsized":"2024-02-26T10:42:21.606964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:23.347207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593333+0000","last_change":"2024-02-26T10:41:59.285657+0000","last_active":"2024-02-26T10:42:21.593333+0000","last_peered":"2024-02-26T10:42:21.593333+0000","last_clean":"2024-02-26T10:42:21.593333+0000","last_became_active":"2024-02-26T10:41:59.285365+0000","last_became_peered":"2024-02-26T10:41:59.285365+0000","last_unstale":"2024-02-26T10:42:21.593333+0000","last_undegraded":"2024-02-26T10:42:21.593333+0000","last_fullsized":"2024-02-26T10:42:21.593333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:10.689718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732027+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:42:21.732027+0000","last_peered":"2024-02-26T10:42:21.732027+0000","last_clean":"2024-02-26T10:42:21.732027+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:42:21.732027+0000","last_undegraded":"2024-02-26T10:42:21.732027+0000","last_fullsized":"2024-02-26T10:42:21.732027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593758+0000","last_change":"2024-02-26T10:42:20.600269+0000","last_active":"2024-02-26T10:42:21.593758+0000","last_peered":"2024-02-26T10:42:21.593758+0000","last_clean":"2024-02-26T10:42:21.593758+0000","last_became_active":"2024-02-26T10:42:20.599903+0000","last_became_peered":"2024-02-26T10:42:20.599903+0000","last_unstale":"2024-02-26T10:42:21.593758+0000","last_undegraded":"2024-02-26T10:42:21.593758+0000","last_fullsized":"2024-02-26T10:42:21.593758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:37.429754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607741+0000","last_change":"2024-02-26T10:42:19.552038+0000","last_active":"2024-02-26T10:42:21.607741+0000","last_peered":"2024-02-26T10:42:21.607741+0000","last_clean":"2024-02-26T10:42:21.607741+0000","last_became_active":"2024-02-26T10:42:19.551727+0000","last_became_peered":"2024-02-26T10:42:19.551727+0000","last_unstale":"2024-02-26T10:42:21.607741+0000","last_undegraded":"2024-02-26T10:42:21.607741+0000","last_fullsized":"2024-02-26T10:42:21.607741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:52:36.656696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.592939+0000","last_change":"2024-02-26T10:41:59.299415+0000","last_active":"2024-02-26T10:42:21.592939+0000","last_peered":"2024-02-26T10:42:21.592939+0000","last_clean":"2024-02-26T10:42:21.592939+0000","last_became_active":"2024-02-26T10:41:59.298995+0000","last_became_peered":"2024-02-26T10:41:59.298995+0000","last_unstale":"2024-02-26T10:42:21.592939+0000","last_undegraded":"2024-02-26T10:42:21.592939+0000","last_fullsized":"2024-02-26T10:42:21.592939+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:38.058564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"75'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595731+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:42:21.595731+0000","last_peered":"2024-02-26T10:42:21.595731+0000","last_clean":"2024-02-26T10:42:21.595731+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:42:21.595731+0000","last_undegraded":"2024-02-26T10:42:21.595731+0000","last_fullsized":"2024-02-26T10:42:21.595731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731651+0000","last_change":"2024-02-26T10:42:20.596696+0000","last_active":"2024-02-26T10:42:21.731651+0000","last_peered":"2024-02-26T10:42:21.731651+0000","last_clean":"2024-02-26T10:42:21.731651+0000","last_became_active":"2024-02-26T10:42:20.596339+0000","last_became_peered":"2024-02-26T10:42:20.596339+0000","last_unstale":"2024-02-26T10:42:21.731651+0000","last_undegraded":"2024-02-26T10:42:21.731651+0000","last_fullsized":"2024-02-26T10:42:21.731651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:04.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921901+0000","last_change":"2024-02-26T10:42:19.563227+0000","last_active":"2024-02-26T10:42:21.921901+0000","last_peered":"2024-02-26T10:42:21.921901+0000","last_clean":"2024-02-26T10:42:21.921901+0000","last_became_active":"2024-02-26T10:42:19.562881+0000","last_became_peered":"2024-02-26T10:42:19.562881+0000","last_unstale":"2024-02-26T10:42:21.921901+0000","last_undegraded":"2024-02-26T10:42:21.921901+0000","last_fullsized":"2024-02-26T10:42:21.921901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:20.708487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921256+0000","last_change":"2024-02-26T10:41:59.290032+0000","last_active":"2024-02-26T10:42:21.921256+0000","last_peered":"2024-02-26T10:42:21.921256+0000","last_clean":"2024-02-26T10:42:21.921256+0000","last_became_active":"2024-02-26T10:41:59.289604+0000","last_became_peered":"2024-02-26T10:41:59.289604+0000","last_unstale":"2024-02-26T10:42:21.921256+0000","last_undegraded":"2024-02-26T10:42:21.921256+0000","last_fullsized":"2024-02-26T10:42:21.921256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:23.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923889+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:42:21.923889+0000","last_peered":"2024-02-26T10:42:21.923889+0000","last_clean":"2024-02-26T10:42:21.923889+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:42:21.923889+0000","last_undegraded":"2024-02-26T10:42:21.923889+0000","last_fullsized":"2024-02-26T10:42:21.923889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920669+0000","last_change":"2024-02-26T10:42:20.608827+0000","last_active":"2024-02-26T10:42:21.920669+0000","last_peered":"2024-02-26T10:42:21.920669+0000","last_clean":"2024-02-26T10:42:21.920669+0000","last_became_active":"2024-02-26T10:42:20.607639+0000","last_became_peered":"2024-02-26T10:42:20.607639+0000","last_unstale":"2024-02-26T10:42:21.920669+0000","last_undegraded":"2024-02-26T10:42:21.920669+0000","last_fullsized":"2024-02-26T10:42:21.920669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596686+0000","last_change":"2024-02-26T10:42:19.571833+0000","last_active":"2024-02-26T10:42:21.596686+0000","last_peered":"2024-02-26T10:42:21.596686+0000","last_clean":"2024-02-26T10:42:21.596686+0000","last_became_active":"2024-02-26T10:42:19.571240+0000","last_became_peered":"2024-02-26T10:42:19.571240+0000","last_unstale":"2024-02-26T10:42:21.596686+0000","last_undegraded":"2024-02-26T10:42:21.596686+0000","last_fullsized":"2024-02-26T10:42:21.596686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:31.100239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923218+0000","last_change":"2024-02-26T10:41:59.299545+0000","last_active":"2024-02-26T10:42:21.923218+0000","last_peered":"2024-02-26T10:42:21.923218+0000","last_clean":"2024-02-26T10:42:21.923218+0000","last_became_active":"2024-02-26T10:41:59.298970+0000","last_became_peered":"2024-02-26T10:41:59.298970+0000","last_unstale":"2024-02-26T10:42:21.923218+0000","last_undegraded":"2024-02-26T10:42:21.923218+0000","last_fullsized":"2024-02-26T10:42:21.923218+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597588+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:42:21.597588+0000","last_peered":"2024-02-26T10:42:21.597588+0000","last_clean":"2024-02-26T10:42:21.597588+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:42:21.597588+0000","last_undegraded":"2024-02-26T10:42:21.597588+0000","last_fullsized":"2024-02-26T10:42:21.597588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923009+0000","last_change":"2024-02-26T10:42:20.601903+0000","last_active":"2024-02-26T10:42:21.923009+0000","last_peered":"2024-02-26T10:42:21.923009+0000","last_clean":"2024-02-26T10:42:21.923009+0000","last_became_active":"2024-02-26T10:42:20.600587+0000","last_became_peered":"2024-02-26T10:42:20.600587+0000","last_unstale":"2024-02-26T10:42:21.923009+0000","last_undegraded":"2024-02-26T10:42:21.923009+0000","last_fullsized":"2024-02-26T10:42:21.923009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:08:23.198928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599271+0000","last_change":"2024-02-26T10:42:19.557245+0000","last_active":"2024-02-26T10:42:21.599271+0000","last_peered":"2024-02-26T10:42:21.599271+0000","last_clean":"2024-02-26T10:42:21.599271+0000","last_became_active":"2024-02-26T10:42:19.556592+0000","last_became_peered":"2024-02-26T10:42:19.556592+0000","last_unstale":"2024-02-26T10:42:21.599271+0000","last_undegraded":"2024-02-26T10:42:21.599271+0000","last_fullsized":"2024-02-26T10:42:21.599271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:21.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599528+0000","last_change":"2024-02-26T10:41:59.283472+0000","last_active":"2024-02-26T10:42:21.599528+0000","last_peered":"2024-02-26T10:42:21.599528+0000","last_clean":"2024-02-26T10:42:21.599528+0000","last_became_active":"2024-02-26T10:41:59.282978+0000","last_became_peered":"2024-02-26T10:41:59.282978+0000","last_unstale":"2024-02-26T10:42:21.599528+0000","last_undegraded":"2024-02-26T10:42:21.599528+0000","last_fullsized":"2024-02-26T10:42:21.599528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:50.202422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596863+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:42:21.596863+0000","last_peered":"2024-02-26T10:42:21.596863+0000","last_clean":"2024-02-26T10:42:21.596863+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:42:21.596863+0000","last_undegraded":"2024-02-26T10:42:21.596863+0000","last_fullsized":"2024-02-26T10:42:21.596863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607379+0000","last_change":"2024-02-26T10:42:20.590282+0000","last_active":"2024-02-26T10:42:21.607379+0000","last_peered":"2024-02-26T10:42:21.607379+0000","last_clean":"2024-02-26T10:42:21.607379+0000","last_became_active":"2024-02-26T10:42:20.589970+0000","last_became_peered":"2024-02-26T10:42:20.589970+0000","last_unstale":"2024-02-26T10:42:21.607379+0000","last_undegraded":"2024-02-26T10:42:21.607379+0000","last_fullsized":"2024-02-26T10:42:21.607379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:52.821056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731271+0000","last_change":"2024-02-26T10:42:19.555195+0000","last_active":"2024-02-26T10:42:21.731271+0000","last_peered":"2024-02-26T10:42:21.731271+0000","last_clean":"2024-02-26T10:42:21.731271+0000","last_became_active":"2024-02-26T10:42:19.554748+0000","last_became_peered":"2024-02-26T10:42:19.554748+0000","last_unstale":"2024-02-26T10:42:21.731271+0000","last_undegraded":"2024-02-26T10:42:21.731271+0000","last_fullsized":"2024-02-26T10:42:21.731271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:53:41.883453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607841+0000","last_change":"2024-02-26T10:41:59.297414+0000","last_active":"2024-02-26T10:42:21.607841+0000","last_peered":"2024-02-26T10:42:21.607841+0000","last_clean":"2024-02-26T10:42:21.607841+0000","last_became_active":"2024-02-26T10:41:59.296486+0000","last_became_peered":"2024-02-26T10:41:59.296486+0000","last_unstale":"2024-02-26T10:42:21.607841+0000","last_undegraded":"2024-02-26T10:42:21.607841+0000","last_fullsized":"2024-02-26T10:42:21.607841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:48.071684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920657+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:42:21.920657+0000","last_peered":"2024-02-26T10:42:21.920657+0000","last_clean":"2024-02-26T10:42:21.920657+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:42:21.920657+0000","last_undegraded":"2024-02-26T10:42:21.920657+0000","last_fullsized":"2024-02-26T10:42:21.920657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:46.825934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919541+0000","last_change":"2024-02-26T10:42:20.601029+0000","last_active":"2024-02-26T10:42:21.919541+0000","last_peered":"2024-02-26T10:42:21.919541+0000","last_clean":"2024-02-26T10:42:21.919541+0000","last_became_active":"2024-02-26T10:42:20.600736+0000","last_became_peered":"2024-02-26T10:42:20.600736+0000","last_unstale":"2024-02-26T10:42:21.919541+0000","last_undegraded":"2024-02-26T10:42:21.919541+0000","last_fullsized":"2024-02-26T10:42:21.919541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:00:26.977878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595424+0000","last_change":"2024-02-26T10:42:19.569242+0000","last_active":"2024-02-26T10:42:21.595424+0000","last_peered":"2024-02-26T10:42:21.595424+0000","last_clean":"2024-02-26T10:42:21.595424+0000","last_became_active":"2024-02-26T10:42:19.568936+0000","last_became_peered":"2024-02-26T10:42:19.568936+0000","last_unstale":"2024-02-26T10:42:21.595424+0000","last_undegraded":"2024-02-26T10:42:21.595424+0000","last_fullsized":"2024-02-26T10:42:21.595424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:31.472935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731385+0000","last_change":"2024-02-26T10:41:59.292484+0000","last_active":"2024-02-26T10:42:21.731385+0000","last_peered":"2024-02-26T10:42:21.731385+0000","last_clean":"2024-02-26T10:42:21.731385+0000","last_became_active":"2024-02-26T10:41:59.292140+0000","last_became_peered":"2024-02-26T10:41:59.292140+0000","last_unstale":"2024-02-26T10:42:21.731385+0000","last_undegraded":"2024-02-26T10:42:21.731385+0000","last_fullsized":"2024-02-26T10:42:21.731385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607911+0000","last_change":"2024-02-26T10:41:58.269128+0000","last_active":"2024-02-26T10:42:21.607911+0000","last_peered":"2024-02-26T10:42:21.607911+0000","last_clean":"2024-02-26T10:42:21.607911+0000","last_became_active":"2024-02-26T10:41:58.268746+0000","last_became_peered":"2024-02-26T10:41:58.268746+0000","last_unstale":"2024-02-26T10:42:21.607911+0000","last_undegraded":"2024-02-26T10:42:21.607911+0000","last_fullsized":"2024-02-26T10:42:21.607911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:46:53.257949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606879+0000","last_change":"2024-02-26T10:42:20.591666+0000","last_active":"2024-02-26T10:42:21.606879+0000","last_peered":"2024-02-26T10:42:21.606879+0000","last_clean":"2024-02-26T10:42:21.606879+0000","last_became_active":"2024-02-26T10:42:20.590442+0000","last_became_peered":"2024-02-26T10:42:20.590442+0000","last_unstale":"2024-02-26T10:42:21.606879+0000","last_undegraded":"2024-02-26T10:42:21.606879+0000","last_fullsized":"2024-02-26T10:42:21.606879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:17:26.997989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596426+0000","last_change":"2024-02-26T10:42:19.567393+0000","last_active":"2024-02-26T10:42:21.596426+0000","last_peered":"2024-02-26T10:42:21.596426+0000","last_clean":"2024-02-26T10:42:21.596426+0000","last_became_active":"2024-02-26T10:42:19.567099+0000","last_became_peered":"2024-02-26T10:42:19.567099+0000","last_unstale":"2024-02-26T10:42:21.596426+0000","last_undegraded":"2024-02-26T10:42:21.596426+0000","last_fullsized":"2024-02-26T10:42:21.596426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:37:57.688332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606757+0000","last_change":"2024-02-26T10:41:59.292780+0000","last_active":"2024-02-26T10:42:21.606757+0000","last_peered":"2024-02-26T10:42:21.606757+0000","last_clean":"2024-02-26T10:42:21.606757+0000","last_became_active":"2024-02-26T10:41:59.292526+0000","last_became_peered":"2024-02-26T10:41:59.292526+0000","last_unstale":"2024-02-26T10:42:21.606757+0000","last_undegraded":"2024-02-26T10:42:21.606757+0000","last_fullsized":"2024-02-26T10:42:21.606757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:12.549758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606015+0000","last_change":"2024-02-26T10:41:58.269538+0000","last_active":"2024-02-26T10:42:21.606015+0000","last_peered":"2024-02-26T10:42:21.606015+0000","last_clean":"2024-02-26T10:42:21.606015+0000","last_became_active":"2024-02-26T10:41:58.269200+0000","last_became_peered":"2024-02-26T10:41:58.269200+0000","last_unstale":"2024-02-26T10:42:21.606015+0000","last_undegraded":"2024-02-26T10:42:21.606015+0000","last_fullsized":"2024-02-26T10:42:21.606015+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:27.687984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606853+0000","last_change":"2024-02-26T10:42:20.615371+0000","last_active":"2024-02-26T10:42:21.606853+0000","last_peered":"2024-02-26T10:42:21.606853+0000","last_clean":"2024-02-26T10:42:21.606853+0000","last_became_active":"2024-02-26T10:42:20.615041+0000","last_became_peered":"2024-02-26T10:42:20.615041+0000","last_unstale":"2024-02-26T10:42:21.606853+0000","last_undegraded":"2024-02-26T10:42:21.606853+0000","last_fullsized":"2024-02-26T10:42:21.606853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:18:45.636096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731579+0000","last_change":"2024-02-26T10:42:19.553615+0000","last_active":"2024-02-26T10:42:21.731579+0000","last_peered":"2024-02-26T10:42:21.731579+0000","last_clean":"2024-02-26T10:42:21.731579+0000","last_became_active":"2024-02-26T10:42:19.553134+0000","last_became_peered":"2024-02-26T10:42:19.553134+0000","last_unstale":"2024-02-26T10:42:21.731579+0000","last_undegraded":"2024-02-26T10:42:21.731579+0000","last_fullsized":"2024-02-26T10:42:21.731579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732198+0000","last_change":"2024-02-26T10:41:59.305037+0000","last_active":"2024-02-26T10:42:21.732198+0000","last_peered":"2024-02-26T10:42:21.732198+0000","last_clean":"2024-02-26T10:42:21.732198+0000","last_became_active":"2024-02-26T10:41:59.304739+0000","last_became_peered":"2024-02-26T10:41:59.304739+0000","last_unstale":"2024-02-26T10:42:21.732198+0000","last_undegraded":"2024-02-26T10:42:21.732198+0000","last_fullsized":"2024-02-26T10:42:21.732198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919349+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:42:21.919349+0000","last_peered":"2024-02-26T10:42:21.919349+0000","last_clean":"2024-02-26T10:42:21.919349+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:42:21.919349+0000","last_undegraded":"2024-02-26T10:42:21.919349+0000","last_fullsized":"2024-02-26T10:42:21.919349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:26.696259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594206+0000","last_change":"2024-02-26T10:42:20.601613+0000","last_active":"2024-02-26T10:42:21.594206+0000","last_peered":"2024-02-26T10:42:21.594206+0000","last_clean":"2024-02-26T10:42:21.594206+0000","last_became_active":"2024-02-26T10:42:20.600643+0000","last_became_peered":"2024-02-26T10:42:20.600643+0000","last_unstale":"2024-02-26T10:42:21.594206+0000","last_undegraded":"2024-02-26T10:42:21.594206+0000","last_fullsized":"2024-02-26T10:42:21.594206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:44.707485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596894+0000","last_change":"2024-02-26T10:42:19.570519+0000","last_active":"2024-02-26T10:42:21.596894+0000","last_peered":"2024-02-26T10:42:21.596894+0000","last_clean":"2024-02-26T10:42:21.596894+0000","last_became_active":"2024-02-26T10:42:19.569990+0000","last_became_peered":"2024-02-26T10:42:19.569990+0000","last_unstale":"2024-02-26T10:42:21.596894+0000","last_undegraded":"2024-02-26T10:42:21.596894+0000","last_fullsized":"2024-02-26T10:42:21.596894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:28:55.454669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599628+0000","last_change":"2024-02-26T10:41:59.289483+0000","last_active":"2024-02-26T10:42:21.599628+0000","last_peered":"2024-02-26T10:42:21.599628+0000","last_clean":"2024-02-26T10:42:21.599628+0000","last_became_active":"2024-02-26T10:41:59.288870+0000","last_became_peered":"2024-02-26T10:41:59.288870+0000","last_unstale":"2024-02-26T10:42:21.599628+0000","last_undegraded":"2024-02-26T10:42:21.599628+0000","last_fullsized":"2024-02-26T10:42:21.599628+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:02.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594381+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:42:21.594381+0000","last_peered":"2024-02-26T10:42:21.594381+0000","last_clean":"2024-02-26T10:42:21.594381+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:42:21.594381+0000","last_undegraded":"2024-02-26T10:42:21.594381+0000","last_fullsized":"2024-02-26T10:42:21.594381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921818+0000","last_change":"2024-02-26T10:42:20.591827+0000","last_active":"2024-02-26T10:42:21.921818+0000","last_peered":"2024-02-26T10:42:21.921818+0000","last_clean":"2024-02-26T10:42:21.921818+0000","last_became_active":"2024-02-26T10:42:20.591457+0000","last_became_peered":"2024-02-26T10:42:20.591457+0000","last_unstale":"2024-02-26T10:42:21.921818+0000","last_undegraded":"2024-02-26T10:42:21.921818+0000","last_fullsized":"2024-02-26T10:42:21.921818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:43:19.105024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921931+0000","last_change":"2024-02-26T10:42:19.567437+0000","last_active":"2024-02-26T10:42:21.921931+0000","last_peered":"2024-02-26T10:42:21.921931+0000","last_clean":"2024-02-26T10:42:21.921931+0000","last_became_active":"2024-02-26T10:42:19.567127+0000","last_became_peered":"2024-02-26T10:42:19.567127+0000","last_unstale":"2024-02-26T10:42:21.921931+0000","last_undegraded":"2024-02-26T10:42:21.921931+0000","last_fullsized":"2024-02-26T10:42:21.921931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:03.477315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921441+0000","last_change":"2024-02-26T10:41:59.294199+0000","last_active":"2024-02-26T10:42:21.921441+0000","last_peered":"2024-02-26T10:42:21.921441+0000","last_clean":"2024-02-26T10:42:21.921441+0000","last_became_active":"2024-02-26T10:41:59.293906+0000","last_became_peered":"2024-02-26T10:41:59.293906+0000","last_unstale":"2024-02-26T10:42:21.921441+0000","last_undegraded":"2024-02-26T10:42:21.921441+0000","last_fullsized":"2024-02-26T10:42:21.921441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:27.711178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"76'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594512+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:42:21.594512+0000","last_peered":"2024-02-26T10:42:21.594512+0000","last_clean":"2024-02-26T10:42:21.594512+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:42:21.594512+0000","last_undegraded":"2024-02-26T10:42:21.594512+0000","last_fullsized":"2024-02-26T10:42:21.594512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:02:34.734605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597341+0000","last_change":"2024-02-26T10:42:20.609982+0000","last_active":"2024-02-26T10:42:21.597341+0000","last_peered":"2024-02-26T10:42:21.597341+0000","last_clean":"2024-02-26T10:42:21.597341+0000","last_became_active":"2024-02-26T10:42:20.609652+0000","last_became_peered":"2024-02-26T10:42:20.609652+0000","last_unstale":"2024-02-26T10:42:21.597341+0000","last_undegraded":"2024-02-26T10:42:21.597341+0000","last_fullsized":"2024-02-26T10:42:21.597341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:14:11.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732471+0000","last_change":"2024-02-26T10:42:19.558924+0000","last_active":"2024-02-26T10:42:21.732471+0000","last_peered":"2024-02-26T10:42:21.732471+0000","last_clean":"2024-02-26T10:42:21.732471+0000","last_became_active":"2024-02-26T10:42:19.558600+0000","last_became_peered":"2024-02-26T10:42:19.558600+0000","last_unstale":"2024-02-26T10:42:21.732471+0000","last_undegraded":"2024-02-26T10:42:21.732471+0000","last_fullsized":"2024-02-26T10:42:21.732471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:53.055720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922828+0000","last_change":"2024-02-26T10:41:59.298323+0000","last_active":"2024-02-26T10:42:21.922828+0000","last_peered":"2024-02-26T10:42:21.922828+0000","last_clean":"2024-02-26T10:42:21.922828+0000","last_became_active":"2024-02-26T10:41:59.295932+0000","last_became_peered":"2024-02-26T10:41:59.295932+0000","last_unstale":"2024-02-26T10:42:21.922828+0000","last_undegraded":"2024-02-26T10:42:21.922828+0000","last_fullsized":"2024-02-26T10:42:21.922828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922270+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:42:21.922270+0000","last_peered":"2024-02-26T10:42:21.922270+0000","last_clean":"2024-02-26T10:42:21.922270+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:42:21.922270+0000","last_undegraded":"2024-02-26T10:42:21.922270+0000","last_fullsized":"2024-02-26T10:42:21.922270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593931+0000","last_change":"2024-02-26T10:42:20.597829+0000","last_active":"2024-02-26T10:42:21.593931+0000","last_peered":"2024-02-26T10:42:21.593931+0000","last_clean":"2024-02-26T10:42:21.593931+0000","last_became_active":"2024-02-26T10:42:20.597548+0000","last_became_peered":"2024-02-26T10:42:20.597548+0000","last_unstale":"2024-02-26T10:42:21.593931+0000","last_undegraded":"2024-02-26T10:42:21.593931+0000","last_fullsized":"2024-02-26T10:42:21.593931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:09.136348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.604651+0000","last_change":"2024-02-26T10:42:19.568185+0000","last_active":"2024-02-26T10:42:21.604651+0000","last_peered":"2024-02-26T10:42:21.604651+0000","last_clean":"2024-02-26T10:42:21.604651+0000","last_became_active":"2024-02-26T10:42:19.567744+0000","last_became_peered":"2024-02-26T10:42:19.567744+0000","last_unstale":"2024-02-26T10:42:21.604651+0000","last_undegraded":"2024-02-26T10:42:21.604651+0000","last_fullsized":"2024-02-26T10:42:21.604651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:03:24.757719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599796+0000","last_change":"2024-02-26T10:41:59.293572+0000","last_active":"2024-02-26T10:42:21.599796+0000","last_peered":"2024-02-26T10:42:21.599796+0000","last_clean":"2024-02-26T10:42:21.599796+0000","last_became_active":"2024-02-26T10:41:59.292292+0000","last_became_peered":"2024-02-26T10:41:59.292292+0000","last_unstale":"2024-02-26T10:42:21.599796+0000","last_undegraded":"2024-02-26T10:42:21.599796+0000","last_fullsized":"2024-02-26T10:42:21.599796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:26.922522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922976+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:42:21.922976+0000","last_peered":"2024-02-26T10:42:21.922976+0000","last_clean":"2024-02-26T10:42:21.922976+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:42:21.922976+0000","last_undegraded":"2024-02-26T10:42:21.922976+0000","last_fullsized":"2024-02-26T10:42:21.922976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:14.749617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606155+0000","last_change":"2024-02-26T10:42:20.606350+0000","last_active":"2024-02-26T10:42:21.606155+0000","last_peered":"2024-02-26T10:42:21.606155+0000","last_clean":"2024-02-26T10:42:21.606155+0000","last_became_active":"2024-02-26T10:42:20.605955+0000","last_became_peered":"2024-02-26T10:42:20.605955+0000","last_unstale":"2024-02-26T10:42:21.606155+0000","last_undegraded":"2024-02-26T10:42:21.606155+0000","last_fullsized":"2024-02-26T10:42:21.606155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:34:18.543420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919265+0000","last_change":"2024-02-26T10:42:19.551290+0000","last_active":"2024-02-26T10:42:21.919265+0000","last_peered":"2024-02-26T10:42:21.919265+0000","last_clean":"2024-02-26T10:42:21.919265+0000","last_became_active":"2024-02-26T10:42:19.550642+0000","last_became_peered":"2024-02-26T10:42:19.550642+0000","last_unstale":"2024-02-26T10:42:21.919265+0000","last_undegraded":"2024-02-26T10:42:21.919265+0000","last_fullsized":"2024-02-26T10:42:21.919265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:49.405962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731999+0000","last_change":"2024-02-26T10:41:59.296354+0000","last_active":"2024-02-26T10:42:21.731999+0000","last_peered":"2024-02-26T10:42:21.731999+0000","last_clean":"2024-02-26T10:42:21.731999+0000","last_became_active":"2024-02-26T10:41:59.296012+0000","last_became_peered":"2024-02-26T10:41:59.296012+0000","last_unstale":"2024-02-26T10:42:21.731999+0000","last_undegraded":"2024-02-26T10:42:21.731999+0000","last_fullsized":"2024-02-26T10:42:21.731999+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596586+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:42:21.596586+0000","last_peered":"2024-02-26T10:42:21.596586+0000","last_clean":"2024-02-26T10:42:21.596586+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:42:21.596586+0000","last_undegraded":"2024-02-26T10:42:21.596586+0000","last_fullsized":"2024-02-26T10:42:21.596586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:50:59.443811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919511+0000","last_change":"2024-02-26T10:42:20.595281+0000","last_active":"2024-02-26T10:42:21.919511+0000","last_peered":"2024-02-26T10:42:21.919511+0000","last_clean":"2024-02-26T10:42:21.919511+0000","last_became_active":"2024-02-26T10:42:20.593810+0000","last_became_peered":"2024-02-26T10:42:20.593810+0000","last_unstale":"2024-02-26T10:42:21.919511+0000","last_undegraded":"2024-02-26T10:42:21.919511+0000","last_fullsized":"2024-02-26T10:42:21.919511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:05:16.206266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606336+0000","last_change":"2024-02-26T10:42:19.564800+0000","last_active":"2024-02-26T10:42:21.606336+0000","last_peered":"2024-02-26T10:42:21.606336+0000","last_clean":"2024-02-26T10:42:21.606336+0000","last_became_active":"2024-02-26T10:42:19.564472+0000","last_became_peered":"2024-02-26T10:42:19.564472+0000","last_unstale":"2024-02-26T10:42:21.606336+0000","last_undegraded":"2024-02-26T10:42:21.606336+0000","last_fullsized":"2024-02-26T10:42:21.606336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:58.917162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596513+0000","last_change":"2024-02-26T10:41:59.297360+0000","last_active":"2024-02-26T10:42:21.596513+0000","last_peered":"2024-02-26T10:42:21.596513+0000","last_clean":"2024-02-26T10:42:21.596513+0000","last_became_active":"2024-02-26T10:41:59.297031+0000","last_became_peered":"2024-02-26T10:41:59.297031+0000","last_unstale":"2024-02-26T10:42:21.596513+0000","last_undegraded":"2024-02-26T10:42:21.596513+0000","last_fullsized":"2024-02-26T10:42:21.596513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598011+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:42:21.598011+0000","last_peered":"2024-02-26T10:42:21.598011+0000","last_clean":"2024-02-26T10:42:21.598011+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:42:21.598011+0000","last_undegraded":"2024-02-26T10:42:21.598011+0000","last_fullsized":"2024-02-26T10:42:21.598011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:58:22.045852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607530+0000","last_change":"2024-02-26T10:42:20.606543+0000","last_active":"2024-02-26T10:42:21.607530+0000","last_peered":"2024-02-26T10:42:21.607530+0000","last_clean":"2024-02-26T10:42:21.607530+0000","last_became_active":"2024-02-26T10:42:20.606202+0000","last_became_peered":"2024-02-26T10:42:20.606202+0000","last_unstale":"2024-02-26T10:42:21.607530+0000","last_undegraded":"2024-02-26T10:42:21.607530+0000","last_fullsized":"2024-02-26T10:42:21.607530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:14.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595742+0000","last_change":"2024-02-26T10:42:19.564857+0000","last_active":"2024-02-26T10:42:21.595742+0000","last_peered":"2024-02-26T10:42:21.595742+0000","last_clean":"2024-02-26T10:42:21.595742+0000","last_became_active":"2024-02-26T10:42:19.564550+0000","last_became_peered":"2024-02-26T10:42:19.564550+0000","last_unstale":"2024-02-26T10:42:21.595742+0000","last_undegraded":"2024-02-26T10:42:21.595742+0000","last_fullsized":"2024-02-26T10:42:21.595742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:06:15.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918939+0000","last_change":"2024-02-26T10:41:59.295832+0000","last_active":"2024-02-26T10:42:21.918939+0000","last_peered":"2024-02-26T10:42:21.918939+0000","last_clean":"2024-02-26T10:42:21.918939+0000","last_became_active":"2024-02-26T10:41:59.295548+0000","last_became_peered":"2024-02-26T10:41:59.295548+0000","last_unstale":"2024-02-26T10:42:21.918939+0000","last_undegraded":"2024-02-26T10:42:21.918939+0000","last_fullsized":"2024-02-26T10:42:21.918939+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920777+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:42:21.920777+0000","last_peered":"2024-02-26T10:42:21.920777+0000","last_clean":"2024-02-26T10:42:21.920777+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:42:21.920777+0000","last_undegraded":"2024-02-26T10:42:21.920777+0000","last_fullsized":"2024-02-26T10:42:21.920777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598541+0000","last_change":"2024-02-26T10:42:20.598744+0000","last_active":"2024-02-26T10:42:21.598541+0000","last_peered":"2024-02-26T10:42:21.598541+0000","last_clean":"2024-02-26T10:42:21.598541+0000","last_became_active":"2024-02-26T10:42:20.598453+0000","last_became_peered":"2024-02-26T10:42:20.598453+0000","last_unstale":"2024-02-26T10:42:21.598541+0000","last_undegraded":"2024-02-26T10:42:21.598541+0000","last_fullsized":"2024-02-26T10:42:21.598541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:54.070139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919646+0000","last_change":"2024-02-26T10:42:19.567123+0000","last_active":"2024-02-26T10:42:21.919646+0000","last_peered":"2024-02-26T10:42:21.919646+0000","last_clean":"2024-02-26T10:42:21.919646+0000","last_became_active":"2024-02-26T10:42:19.566824+0000","last_became_peered":"2024-02-26T10:42:19.566824+0000","last_unstale":"2024-02-26T10:42:21.919646+0000","last_undegraded":"2024-02-26T10:42:21.919646+0000","last_fullsized":"2024-02-26T10:42:21.919646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:25:21.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920523+0000","last_change":"2024-02-26T10:41:59.300681+0000","last_active":"2024-02-26T10:42:21.920523+0000","last_peered":"2024-02-26T10:42:21.920523+0000","last_clean":"2024-02-26T10:42:21.920523+0000","last_became_active":"2024-02-26T10:41:59.299849+0000","last_became_peered":"2024-02-26T10:41:59.299849+0000","last_unstale":"2024-02-26T10:42:21.920523+0000","last_undegraded":"2024-02-26T10:42:21.920523+0000","last_fullsized":"2024-02-26T10:42:21.920523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597947+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:42:21.597947+0000","last_peered":"2024-02-26T10:42:21.597947+0000","last_clean":"2024-02-26T10:42:21.597947+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:42:21.597947+0000","last_undegraded":"2024-02-26T10:42:21.597947+0000","last_fullsized":"2024-02-26T10:42:21.597947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:40:26.347537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607430+0000","last_change":"2024-02-26T10:42:20.594531+0000","last_active":"2024-02-26T10:42:21.607430+0000","last_peered":"2024-02-26T10:42:21.607430+0000","last_clean":"2024-02-26T10:42:21.607430+0000","last_became_active":"2024-02-26T10:42:20.594237+0000","last_became_peered":"2024-02-26T10:42:20.594237+0000","last_unstale":"2024-02-26T10:42:21.607430+0000","last_undegraded":"2024-02-26T10:42:21.607430+0000","last_fullsized":"2024-02-26T10:42:21.607430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:00:51.720533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921150+0000","last_change":"2024-02-26T10:42:19.574963+0000","last_active":"2024-02-26T10:42:21.921150+0000","last_peered":"2024-02-26T10:42:21.921150+0000","last_clean":"2024-02-26T10:42:21.921150+0000","last_became_active":"2024-02-26T10:42:19.574602+0000","last_became_peered":"2024-02-26T10:42:19.574602+0000","last_unstale":"2024-02-26T10:42:21.921150+0000","last_undegraded":"2024-02-26T10:42:21.921150+0000","last_fullsized":"2024-02-26T10:42:21.921150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:31.472935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594291+0000","last_change":"2024-02-26T10:41:59.295485+0000","last_active":"2024-02-26T10:42:21.594291+0000","last_peered":"2024-02-26T10:42:21.594291+0000","last_clean":"2024-02-26T10:42:21.594291+0000","last_became_active":"2024-02-26T10:41:59.295154+0000","last_became_peered":"2024-02-26T10:41:59.295154+0000","last_unstale":"2024-02-26T10:42:21.594291+0000","last_undegraded":"2024-02-26T10:42:21.594291+0000","last_fullsized":"2024-02-26T10:42:21.594291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:15.764536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921353+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:42:21.921353+0000","last_peered":"2024-02-26T10:42:21.921353+0000","last_clean":"2024-02-26T10:42:21.921353+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:42:21.921353+0000","last_undegraded":"2024-02-26T10:42:21.921353+0000","last_fullsized":"2024-02-26T10:42:21.921353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919456+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:42:21.919456+0000","last_peered":"2024-02-26T10:42:21.919456+0000","last_clean":"2024-02-26T10:42:21.919456+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:42:21.919456+0000","last_undegraded":"2024-02-26T10:42:21.919456+0000","last_fullsized":"2024-02-26T10:42:21.919456+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":3913,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542194,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1585152,"data_stored":1033134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":443930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":445791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542195,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":444320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":447333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":443200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":991232,"data_stored":442778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1581056,"data_stored":1033568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:25.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:42:25.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.340+0000 7f803e600640 1 Processor -- start 2024-02-26T10:42:25.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.340+0000 7f803e600640 1 -- start start 2024-02-26T10:42:25.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.342+0000 7f803e600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80380a94a0 0x7f803806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.342+0000 7f803e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806db00 0x7f80380abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.342+0000 7f803e600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803806df90 con 0x7f803806ac20 2024-02-26T10:42:25.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.342+0000 7f803e600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380a9a80 con 0x7f80380a94a0 2024-02-26T10:42:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.342+0000 7f803e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380a9d60 con 0x7f803806db00 2024-02-26T10:42:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.342+0000 7f802fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80380a94a0 0x7f803806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f802fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80380a94a0 0x7f803806b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34440/0 (socket says 172.21.15.67:34440) 2024-02-26T10:42:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f802fe00640 1 -- 172.21.15.67:0/1961479981 learned_addr learned my addr 172.21.15.67:0/1961479981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f803dc00640 1 --2- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806db00 0x7f80380abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f802fe00640 1 -- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806db00 msgr2=0x7f80380abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:25.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f802fe00640 1 --2- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806db00 0x7f80380abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f802fe00640 1 -- 172.21.15.67:0/1961479981 >> v1:172.21.15.82:6789/0 conn(0x7f803806ac20 legacy=0x7f8038153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.343+0000 7f802fe00640 1 -- 172.21.15.67:0/1961479981 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380ab750 con 0x7f80380a94a0 2024-02-26T10:42:25.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.345+0000 7f802fe00640 1 --2- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f803806b000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8020009870 tx=0x7f802002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0662605c32921e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f8037e00640 1 -- 172.21.15.67:0/1961479981 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020034070 con 0x7f80380a94a0 2024-02-26T10:42:25.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f8037e00640 1 -- 172.21.15.67:0/1961479981 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020030840 con 0x7f80380a94a0 2024-02-26T10:42:25.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f8037e00640 1 -- 172.21.15.67:0/1961479981 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020030c10 con 0x7f80380a94a0 2024-02-26T10:42:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 -- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 msgr2=0x7f803806b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 --2- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f803806b000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8020009870 tx=0x7f802002b980 comp rx=0 tx=0).stop 2024-02-26T10:42:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 -- 172.21.15.67:0/1961479981 shutdown_connections 2024-02-26T10:42:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 --2- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806db00 0x7f80380abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 --2- 172.21.15.67:0/1961479981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f803806b000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 -- 172.21.15.67:0/1961479981 >> 172.21.15.67:0/1961479981 conn(0x7f80380c63f0 msgr2=0x7f80380c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 -- 172.21.15.67:0/1961479981 shutdown_connections 2024-02-26T10:42:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.346+0000 7f803e600640 1 -- 172.21.15.67:0/1961479981 wait complete. 2024-02-26T10:42:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.348+0000 7f803e600640 1 Processor -- start 2024-02-26T10:42:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.348+0000 7f803e600640 1 -- start start 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.349+0000 7f803e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806ac20 0x7f80381dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.349+0000 7f803e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 0x7f8038072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.349+0000 7f803e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f80381dd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.349+0000 7f803e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038150410 con 0x7f803806db00 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.349+0000 7f803e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380ac490 con 0x7f80380a94a0 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.349+0000 7f803e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8038150590 con 0x7f803806ac20 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 0x7f8038072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 0x7f8038072040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50610/0 (socket says 172.21.15.67:50610) 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 -- 172.21.15.67:0/3674245976 learned_addr learned my addr 172.21.15.67:0/3674245976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f803d200640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806ac20 0x7f80381dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f803dc00640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f80381dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 -- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806ac20 msgr2=0x7f80381dc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806ac20 0x7f80381dc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 -- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 msgr2=0x7f80381dd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f80381dd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 -- 172.21.15.67:0/3674245976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038156970 con 0x7f803806db00 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.350+0000 7f802fe00640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 0x7f8038072040 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8020000c00 tx=0x7f80200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.351+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020034040 con 0x7f803806db00 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.351+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020039d90 con 0x7f803806db00 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.351+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020043630 con 0x7f803806db00 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.351+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8038156c50 con 0x7f803806db00 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.351+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80381de690 con 0x7f803806db00 2024-02-26T10:42:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.352+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8038074a30 con 0x7f803806db00 2024-02-26T10:42:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.353+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f802004a060 con 0x7f803806db00 2024-02-26T10:42:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.354+0000 7f8035600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f801406b4d0 0x7f801406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.355+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f80200ac4f0 con 0x7f803806db00 2024-02-26T10:42:25.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.355+0000 7f803d200640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f801406b4d0 0x7f801406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.356+0000 7f803d200640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f801406b4d0 0x7f801406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8028001d40 tx=0x7f8028009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.359+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f802003e070 con 0x7f803806db00 2024-02-26T10:42:25.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.556+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f803806b9b0 con 0x7f801406b4d0 2024-02-26T10:42:25.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.590+0000 7f8035600640 1 -- 172.21.15.67:0/3674245976 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490654 (secure 0 0 0) 0x7f803806b9b0 con 0x7f801406b4d0 2024-02-26T10:42:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:25.605 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f801406b4d0 msgr2=0x7f801406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f801406b4d0 0x7f801406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8028001d40 tx=0x7f8028009040 comp rx=0 tx=0).stop 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 msgr2=0x7f8038072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 0x7f8038072040 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8020000c00 tx=0x7f80200046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 shutdown_connections 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f801406b4d0 0x7f801406d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80380a94a0 0x7f80381dd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f803806db00 0x7f8038072040 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 --2- 172.21.15.67:0/3674245976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f803806ac20 0x7f80381dc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 >> 172.21.15.67:0/3674245976 conn(0x7f80380c63f0 msgr2=0x7f80380c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.600+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 shutdown_connections 2024-02-26T10:42:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.601+0000 7f803e600640 1 -- 172.21.15.67:0/3674245976 wait complete. 2024-02-26T10:42:25.627 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-02-26T10:42:24.412072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":367,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258404,"kb_used_data":8936,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716316,"statfs":{"total":773094113280,"available":772829507584,"internally_reserved":0,"allocated":9150464,"data_stored":4734054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.889059"},"pg_stats":[{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593602+0000","last_change":"2024-02-26T10:42:20.608978+0000","last_active":"2024-02-26T10:42:21.593602+0000","last_peered":"2024-02-26T10:42:21.593602+0000","last_clean":"2024-02-26T10:42:21.593602+0000","last_became_active":"2024-02-26T10:42:20.608613+0000","last_became_peered":"2024-02-26T10:42:20.608613+0000","last_unstale":"2024-02-26T10:42:21.593602+0000","last_undegraded":"2024-02-26T10:42:21.593602+0000","last_fullsized":"2024-02-26T10:42:21.593602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:48.452971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923082+0000","last_change":"2024-02-26T10:41:59.308476+0000","last_active":"2024-02-26T10:42:21.923082+0000","last_peered":"2024-02-26T10:42:21.923082+0000","last_clean":"2024-02-26T10:42:21.923082+0000","last_became_active":"2024-02-26T10:41:59.307995+0000","last_became_peered":"2024-02-26T10:41:59.307995+0000","last_unstale":"2024-02-26T10:42:21.923082+0000","last_undegraded":"2024-02-26T10:42:21.923082+0000","last_fullsized":"2024-02-26T10:42:21.923082+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919927+0000","last_change":"2024-02-26T10:42:20.593534+0000","last_active":"2024-02-26T10:42:21.919927+0000","last_peered":"2024-02-26T10:42:21.919927+0000","last_clean":"2024-02-26T10:42:21.919927+0000","last_became_active":"2024-02-26T10:42:20.593129+0000","last_became_peered":"2024-02-26T10:42:20.593129+0000","last_unstale":"2024-02-26T10:42:21.919927+0000","last_undegraded":"2024-02-26T10:42:21.919927+0000","last_fullsized":"2024-02-26T10:42:21.919927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:39:31.051666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594484+0000","last_change":"2024-02-26T10:41:59.300851+0000","last_active":"2024-02-26T10:42:21.594484+0000","last_peered":"2024-02-26T10:42:21.594484+0000","last_clean":"2024-02-26T10:42:21.594484+0000","last_became_active":"2024-02-26T10:41:59.300574+0000","last_became_peered":"2024-02-26T10:41:59.300574+0000","last_unstale":"2024-02-26T10:42:21.594484+0000","last_undegraded":"2024-02-26T10:42:21.594484+0000","last_fullsized":"2024-02-26T10:42:21.594484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:04.528547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593696+0000","last_change":"2024-02-26T10:42:20.605522+0000","last_active":"2024-02-26T10:42:21.593696+0000","last_peered":"2024-02-26T10:42:21.593696+0000","last_clean":"2024-02-26T10:42:21.593696+0000","last_became_active":"2024-02-26T10:42:20.605209+0000","last_became_peered":"2024-02-26T10:42:20.605209+0000","last_unstale":"2024-02-26T10:42:21.593696+0000","last_undegraded":"2024-02-26T10:42:21.593696+0000","last_fullsized":"2024-02-26T10:42:21.593696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:31.467164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920373+0000","last_change":"2024-02-26T10:41:59.293058+0000","last_active":"2024-02-26T10:42:21.920373+0000","last_peered":"2024-02-26T10:42:21.920373+0000","last_clean":"2024-02-26T10:42:21.920373+0000","last_became_active":"2024-02-26T10:41:59.292789+0000","last_became_peered":"2024-02-26T10:41:59.292789+0000","last_unstale":"2024-02-26T10:42:21.920373+0000","last_undegraded":"2024-02-26T10:42:21.920373+0000","last_fullsized":"2024-02-26T10:42:21.920373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597521+0000","last_change":"2024-02-26T10:42:20.613590+0000","last_active":"2024-02-26T10:42:21.597521+0000","last_peered":"2024-02-26T10:42:21.597521+0000","last_clean":"2024-02-26T10:42:21.597521+0000","last_became_active":"2024-02-26T10:42:20.612995+0000","last_became_peered":"2024-02-26T10:42:20.612995+0000","last_unstale":"2024-02-26T10:42:21.597521+0000","last_undegraded":"2024-02-26T10:42:21.597521+0000","last_fullsized":"2024-02-26T10:42:21.597521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:37:10.632162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606436+0000","last_change":"2024-02-26T10:41:59.296577+0000","last_active":"2024-02-26T10:42:21.606436+0000","last_peered":"2024-02-26T10:42:21.606436+0000","last_clean":"2024-02-26T10:42:21.606436+0000","last_became_active":"2024-02-26T10:41:59.296264+0000","last_became_peered":"2024-02-26T10:41:59.296264+0000","last_unstale":"2024-02-26T10:42:21.606436+0000","last_undegraded":"2024-02-26T10:42:21.606436+0000","last_fullsized":"2024-02-26T10:42:21.606436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:13.257965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918739+0000","last_change":"2024-02-26T10:42:20.600877+0000","last_active":"2024-02-26T10:42:21.918739+0000","last_peered":"2024-02-26T10:42:21.918739+0000","last_clean":"2024-02-26T10:42:21.918739+0000","last_became_active":"2024-02-26T10:42:20.599771+0000","last_became_peered":"2024-02-26T10:42:20.599771+0000","last_unstale":"2024-02-26T10:42:21.918739+0000","last_undegraded":"2024-02-26T10:42:21.918739+0000","last_fullsized":"2024-02-26T10:42:21.918739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:53.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598460+0000","last_change":"2024-02-26T10:41:59.298038+0000","last_active":"2024-02-26T10:42:21.598460+0000","last_peered":"2024-02-26T10:42:21.598460+0000","last_clean":"2024-02-26T10:42:21.598460+0000","last_became_active":"2024-02-26T10:41:59.297699+0000","last_became_peered":"2024-02-26T10:41:59.297699+0000","last_unstale":"2024-02-26T10:42:21.598460+0000","last_undegraded":"2024-02-26T10:42:21.598460+0000","last_fullsized":"2024-02-26T10:42:21.598460+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:30.879707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595462+0000","last_change":"2024-02-26T10:42:20.615279+0000","last_active":"2024-02-26T10:42:21.595462+0000","last_peered":"2024-02-26T10:42:21.595462+0000","last_clean":"2024-02-26T10:42:21.595462+0000","last_became_active":"2024-02-26T10:42:20.610374+0000","last_became_peered":"2024-02-26T10:42:20.610374+0000","last_unstale":"2024-02-26T10:42:21.595462+0000","last_undegraded":"2024-02-26T10:42:21.595462+0000","last_fullsized":"2024-02-26T10:42:21.595462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731746+0000","last_change":"2024-02-26T10:41:59.295626+0000","last_active":"2024-02-26T10:42:21.731746+0000","last_peered":"2024-02-26T10:42:21.731746+0000","last_clean":"2024-02-26T10:42:21.731746+0000","last_became_active":"2024-02-26T10:41:59.295266+0000","last_became_peered":"2024-02-26T10:41:59.295266+0000","last_unstale":"2024-02-26T10:42:21.731746+0000","last_undegraded":"2024-02-26T10:42:21.731746+0000","last_fullsized":"2024-02-26T10:42:21.731746+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921041+0000","last_change":"2024-02-26T10:42:20.594928+0000","last_active":"2024-02-26T10:42:21.921041+0000","last_peered":"2024-02-26T10:42:21.921041+0000","last_clean":"2024-02-26T10:42:21.921041+0000","last_became_active":"2024-02-26T10:42:20.594651+0000","last_became_peered":"2024-02-26T10:42:20.594651+0000","last_unstale":"2024-02-26T10:42:21.921041+0000","last_undegraded":"2024-02-26T10:42:21.921041+0000","last_fullsized":"2024-02-26T10:42:21.921041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:53.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.605914+0000","last_change":"2024-02-26T10:41:59.287398+0000","last_active":"2024-02-26T10:42:21.605914+0000","last_peered":"2024-02-26T10:42:21.605914+0000","last_clean":"2024-02-26T10:42:21.605914+0000","last_became_active":"2024-02-26T10:41:59.286305+0000","last_became_peered":"2024-02-26T10:41:59.286305+0000","last_unstale":"2024-02-26T10:42:21.605914+0000","last_undegraded":"2024-02-26T10:42:21.605914+0000","last_fullsized":"2024-02-26T10:42:21.605914+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:54.865254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920442+0000","last_change":"2024-02-26T10:42:20.600066+0000","last_active":"2024-02-26T10:42:21.920442+0000","last_peered":"2024-02-26T10:42:21.920442+0000","last_clean":"2024-02-26T10:42:21.920442+0000","last_became_active":"2024-02-26T10:42:20.599736+0000","last_became_peered":"2024-02-26T10:42:20.599736+0000","last_unstale":"2024-02-26T10:42:21.920442+0000","last_undegraded":"2024-02-26T10:42:21.920442+0000","last_fullsized":"2024-02-26T10:42:21.920442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:08:23.198928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.730946+0000","last_change":"2024-02-26T10:41:59.307667+0000","last_active":"2024-02-26T10:42:21.730946+0000","last_peered":"2024-02-26T10:42:21.730946+0000","last_clean":"2024-02-26T10:42:21.730946+0000","last_became_active":"2024-02-26T10:41:59.307319+0000","last_became_peered":"2024-02-26T10:41:59.307319+0000","last_unstale":"2024-02-26T10:42:21.730946+0000","last_undegraded":"2024-02-26T10:42:21.730946+0000","last_fullsized":"2024-02-26T10:42:21.730946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919840+0000","last_change":"2024-02-26T10:42:20.605813+0000","last_active":"2024-02-26T10:42:21.919840+0000","last_peered":"2024-02-26T10:42:21.919840+0000","last_clean":"2024-02-26T10:42:21.919840+0000","last_became_active":"2024-02-26T10:42:20.605522+0000","last_became_peered":"2024-02-26T10:42:20.605522+0000","last_unstale":"2024-02-26T10:42:21.919840+0000","last_undegraded":"2024-02-26T10:42:21.919840+0000","last_fullsized":"2024-02-26T10:42:21.919840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:28:05.289716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732630+0000","last_change":"2024-02-26T10:41:59.304927+0000","last_active":"2024-02-26T10:42:21.732630+0000","last_peered":"2024-02-26T10:42:21.732630+0000","last_clean":"2024-02-26T10:42:21.732630+0000","last_became_active":"2024-02-26T10:41:59.304641+0000","last_became_peered":"2024-02-26T10:41:59.304641+0000","last_unstale":"2024-02-26T10:42:21.732630+0000","last_undegraded":"2024-02-26T10:42:21.732630+0000","last_fullsized":"2024-02-26T10:42:21.732630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599031+0000","last_change":"2024-02-26T10:42:20.612287+0000","last_active":"2024-02-26T10:42:21.599031+0000","last_peered":"2024-02-26T10:42:21.599031+0000","last_clean":"2024-02-26T10:42:21.599031+0000","last_became_active":"2024-02-26T10:42:20.611835+0000","last_became_peered":"2024-02-26T10:42:20.611835+0000","last_unstale":"2024-02-26T10:42:21.599031+0000","last_undegraded":"2024-02-26T10:42:21.599031+0000","last_fullsized":"2024-02-26T10:42:21.599031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:25:22.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922537+0000","last_change":"2024-02-26T10:41:59.293335+0000","last_active":"2024-02-26T10:42:21.922537+0000","last_peered":"2024-02-26T10:42:21.922537+0000","last_clean":"2024-02-26T10:42:21.922537+0000","last_became_active":"2024-02-26T10:41:59.292937+0000","last_became_peered":"2024-02-26T10:41:59.292937+0000","last_unstale":"2024-02-26T10:42:21.922537+0000","last_undegraded":"2024-02-26T10:42:21.922537+0000","last_fullsized":"2024-02-26T10:42:21.922537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599864+0000","last_change":"2024-02-26T10:42:20.606711+0000","last_active":"2024-02-26T10:42:21.599864+0000","last_peered":"2024-02-26T10:42:21.599864+0000","last_clean":"2024-02-26T10:42:21.599864+0000","last_became_active":"2024-02-26T10:42:20.606355+0000","last_became_peered":"2024-02-26T10:42:20.606355+0000","last_unstale":"2024-02-26T10:42:21.599864+0000","last_undegraded":"2024-02-26T10:42:21.599864+0000","last_fullsized":"2024-02-26T10:42:21.599864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:21.722907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593945+0000","last_change":"2024-02-26T10:41:59.282216+0000","last_active":"2024-02-26T10:42:21.593945+0000","last_peered":"2024-02-26T10:42:21.593945+0000","last_clean":"2024-02-26T10:42:21.593945+0000","last_became_active":"2024-02-26T10:41:59.281891+0000","last_became_peered":"2024-02-26T10:41:59.281891+0000","last_unstale":"2024-02-26T10:42:21.593945+0000","last_undegraded":"2024-02-26T10:42:21.593945+0000","last_fullsized":"2024-02-26T10:42:21.593945+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:07.272911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921936+0000","last_change":"2024-02-26T10:42:20.600140+0000","last_active":"2024-02-26T10:42:21.921936+0000","last_peered":"2024-02-26T10:42:21.921936+0000","last_clean":"2024-02-26T10:42:21.921936+0000","last_became_active":"2024-02-26T10:42:20.599862+0000","last_became_peered":"2024-02-26T10:42:20.599862+0000","last_unstale":"2024-02-26T10:42:21.921936+0000","last_undegraded":"2024-02-26T10:42:21.921936+0000","last_fullsized":"2024-02-26T10:42:21.921936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:22.708651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922096+0000","last_change":"2024-02-26T10:41:59.290942+0000","last_active":"2024-02-26T10:42:21.922096+0000","last_peered":"2024-02-26T10:42:21.922096+0000","last_clean":"2024-02-26T10:42:21.922096+0000","last_became_active":"2024-02-26T10:41:59.288616+0000","last_became_peered":"2024-02-26T10:41:59.288616+0000","last_unstale":"2024-02-26T10:42:21.922096+0000","last_undegraded":"2024-02-26T10:42:21.922096+0000","last_fullsized":"2024-02-26T10:42:21.922096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919033+0000","last_change":"2024-02-26T10:42:20.598690+0000","last_active":"2024-02-26T10:42:21.919033+0000","last_peered":"2024-02-26T10:42:21.919033+0000","last_clean":"2024-02-26T10:42:21.919033+0000","last_became_active":"2024-02-26T10:42:20.598374+0000","last_became_peered":"2024-02-26T10:42:20.598374+0000","last_unstale":"2024-02-26T10:42:21.919033+0000","last_undegraded":"2024-02-26T10:42:21.919033+0000","last_fullsized":"2024-02-26T10:42:21.919033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:49:53.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919776+0000","last_change":"2024-02-26T10:41:59.281911+0000","last_active":"2024-02-26T10:42:21.919776+0000","last_peered":"2024-02-26T10:42:21.919776+0000","last_clean":"2024-02-26T10:42:21.919776+0000","last_became_active":"2024-02-26T10:41:59.281270+0000","last_became_peered":"2024-02-26T10:41:59.281270+0000","last_unstale":"2024-02-26T10:42:21.919776+0000","last_undegraded":"2024-02-26T10:42:21.919776+0000","last_fullsized":"2024-02-26T10:42:21.919776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:38:10.171669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918527+0000","last_change":"2024-02-26T10:42:20.613254+0000","last_active":"2024-02-26T10:42:21.918527+0000","last_peered":"2024-02-26T10:42:21.918527+0000","last_clean":"2024-02-26T10:42:21.918527+0000","last_became_active":"2024-02-26T10:42:20.612937+0000","last_became_peered":"2024-02-26T10:42:20.612937+0000","last_unstale":"2024-02-26T10:42:21.918527+0000","last_undegraded":"2024-02-26T10:42:21.918527+0000","last_fullsized":"2024-02-26T10:42:21.918527+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:00:26.977878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607737+0000","last_change":"2024-02-26T10:41:59.310387+0000","last_active":"2024-02-26T10:42:21.607737+0000","last_peered":"2024-02-26T10:42:21.607737+0000","last_clean":"2024-02-26T10:42:21.607737+0000","last_became_active":"2024-02-26T10:41:59.310111+0000","last_became_peered":"2024-02-26T10:41:59.310111+0000","last_unstale":"2024-02-26T10:42:21.607737+0000","last_undegraded":"2024-02-26T10:42:21.607737+0000","last_fullsized":"2024-02-26T10:42:21.607737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:53:52.167978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920180+0000","last_change":"2024-02-26T10:42:20.605674+0000","last_active":"2024-02-26T10:42:21.920180+0000","last_peered":"2024-02-26T10:42:21.920180+0000","last_clean":"2024-02-26T10:42:21.920180+0000","last_became_active":"2024-02-26T10:42:20.605420+0000","last_became_peered":"2024-02-26T10:42:20.605420+0000","last_unstale":"2024-02-26T10:42:21.920180+0000","last_undegraded":"2024-02-26T10:42:21.920180+0000","last_fullsized":"2024-02-26T10:42:21.920180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:08:23.198928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919181+0000","last_change":"2024-02-26T10:41:59.289328+0000","last_active":"2024-02-26T10:42:21.919181+0000","last_peered":"2024-02-26T10:42:21.919181+0000","last_clean":"2024-02-26T10:42:21.919181+0000","last_became_active":"2024-02-26T10:41:59.288951+0000","last_became_peered":"2024-02-26T10:41:59.288951+0000","last_unstale":"2024-02-26T10:42:21.919181+0000","last_undegraded":"2024-02-26T10:42:21.919181+0000","last_fullsized":"2024-02-26T10:42:21.919181+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:00.613487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607467+0000","last_change":"2024-02-26T10:42:20.592320+0000","last_active":"2024-02-26T10:42:21.607467+0000","last_peered":"2024-02-26T10:42:21.607467+0000","last_clean":"2024-02-26T10:42:21.607467+0000","last_became_active":"2024-02-26T10:42:20.592031+0000","last_became_peered":"2024-02-26T10:42:20.592031+0000","last_unstale":"2024-02-26T10:42:21.607467+0000","last_undegraded":"2024-02-26T10:42:21.607467+0000","last_fullsized":"2024-02-26T10:42:21.607467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:46:48.209177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597600+0000","last_change":"2024-02-26T10:41:59.290536+0000","last_active":"2024-02-26T10:42:21.597600+0000","last_peered":"2024-02-26T10:42:21.597600+0000","last_clean":"2024-02-26T10:42:21.597600+0000","last_became_active":"2024-02-26T10:41:59.289661+0000","last_became_peered":"2024-02-26T10:41:59.289661+0000","last_unstale":"2024-02-26T10:42:21.597600+0000","last_undegraded":"2024-02-26T10:42:21.597600+0000","last_fullsized":"2024-02-26T10:42:21.597600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:56.415415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606888+0000","last_change":"2024-02-26T10:42:20.600988+0000","last_active":"2024-02-26T10:42:21.606888+0000","last_peered":"2024-02-26T10:42:21.606888+0000","last_clean":"2024-02-26T10:42:21.606888+0000","last_became_active":"2024-02-26T10:42:20.600715+0000","last_became_peered":"2024-02-26T10:42:20.600715+0000","last_unstale":"2024-02-26T10:42:21.606888+0000","last_undegraded":"2024-02-26T10:42:21.606888+0000","last_fullsized":"2024-02-26T10:42:21.606888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:21:38.693700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.600013+0000","last_change":"2024-02-26T10:41:59.291203+0000","last_active":"2024-02-26T10:42:21.600013+0000","last_peered":"2024-02-26T10:42:21.600013+0000","last_clean":"2024-02-26T10:42:21.600013+0000","last_became_active":"2024-02-26T10:41:59.290916+0000","last_became_peered":"2024-02-26T10:41:59.290916+0000","last_unstale":"2024-02-26T10:42:21.600013+0000","last_undegraded":"2024-02-26T10:42:21.600013+0000","last_fullsized":"2024-02-26T10:42:21.600013+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:09.397334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598998+0000","last_change":"2024-02-26T10:42:20.598910+0000","last_active":"2024-02-26T10:42:21.598998+0000","last_peered":"2024-02-26T10:42:21.598998+0000","last_clean":"2024-02-26T10:42:21.598998+0000","last_became_active":"2024-02-26T10:42:20.598544+0000","last_became_peered":"2024-02-26T10:42:20.598544+0000","last_unstale":"2024-02-26T10:42:21.598998+0000","last_undegraded":"2024-02-26T10:42:21.598998+0000","last_fullsized":"2024-02-26T10:42:21.598998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:29:17.248481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607437+0000","last_change":"2024-02-26T10:41:59.291282+0000","last_active":"2024-02-26T10:42:21.607437+0000","last_peered":"2024-02-26T10:42:21.607437+0000","last_clean":"2024-02-26T10:42:21.607437+0000","last_became_active":"2024-02-26T10:41:59.291008+0000","last_became_peered":"2024-02-26T10:41:59.291008+0000","last_unstale":"2024-02-26T10:42:21.607437+0000","last_undegraded":"2024-02-26T10:42:21.607437+0000","last_fullsized":"2024-02-26T10:42:21.607437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:55:38.972893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597561+0000","last_change":"2024-02-26T10:42:20.609153+0000","last_active":"2024-02-26T10:42:21.597561+0000","last_peered":"2024-02-26T10:42:21.597561+0000","last_clean":"2024-02-26T10:42:21.597561+0000","last_became_active":"2024-02-26T10:42:20.608837+0000","last_became_peered":"2024-02-26T10:42:20.608837+0000","last_unstale":"2024-02-26T10:42:21.597561+0000","last_undegraded":"2024-02-26T10:42:21.597561+0000","last_fullsized":"2024-02-26T10:42:21.597561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:19:34.963516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597240+0000","last_change":"2024-02-26T10:41:59.289830+0000","last_active":"2024-02-26T10:42:21.597240+0000","last_peered":"2024-02-26T10:42:21.597240+0000","last_clean":"2024-02-26T10:42:21.597240+0000","last_became_active":"2024-02-26T10:41:59.289466+0000","last_became_peered":"2024-02-26T10:41:59.289466+0000","last_unstale":"2024-02-26T10:42:21.597240+0000","last_undegraded":"2024-02-26T10:42:21.597240+0000","last_fullsized":"2024-02-26T10:42:21.597240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920343+0000","last_change":"2024-02-26T10:42:20.596858+0000","last_active":"2024-02-26T10:42:21.920343+0000","last_peered":"2024-02-26T10:42:21.920343+0000","last_clean":"2024-02-26T10:42:21.920343+0000","last_became_active":"2024-02-26T10:42:20.596571+0000","last_became_peered":"2024-02-26T10:42:20.596571+0000","last_unstale":"2024-02-26T10:42:21.920343+0000","last_undegraded":"2024-02-26T10:42:21.920343+0000","last_fullsized":"2024-02-26T10:42:21.920343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606502+0000","last_change":"2024-02-26T10:41:59.294466+0000","last_active":"2024-02-26T10:42:21.606502+0000","last_peered":"2024-02-26T10:42:21.606502+0000","last_clean":"2024-02-26T10:42:21.606502+0000","last_became_active":"2024-02-26T10:41:59.294199+0000","last_became_peered":"2024-02-26T10:41:59.294199+0000","last_unstale":"2024-02-26T10:42:21.606502+0000","last_undegraded":"2024-02-26T10:42:21.606502+0000","last_fullsized":"2024-02-26T10:42:21.606502+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:25:06.679617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595594+0000","last_change":"2024-02-26T10:42:20.609952+0000","last_active":"2024-02-26T10:42:21.595594+0000","last_peered":"2024-02-26T10:42:21.595594+0000","last_clean":"2024-02-26T10:42:21.595594+0000","last_became_active":"2024-02-26T10:42:20.609624+0000","last_became_peered":"2024-02-26T10:42:20.609624+0000","last_unstale":"2024-02-26T10:42:21.595594+0000","last_undegraded":"2024-02-26T10:42:21.595594+0000","last_fullsized":"2024-02-26T10:42:21.595594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:00.537776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606992+0000","last_change":"2024-02-26T10:41:59.302058+0000","last_active":"2024-02-26T10:42:21.606992+0000","last_peered":"2024-02-26T10:42:21.606992+0000","last_clean":"2024-02-26T10:42:21.606992+0000","last_became_active":"2024-02-26T10:41:59.301773+0000","last_became_peered":"2024-02-26T10:41:59.301773+0000","last_unstale":"2024-02-26T10:42:21.606992+0000","last_undegraded":"2024-02-26T10:42:21.606992+0000","last_fullsized":"2024-02-26T10:42:21.606992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:35:58.489330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919011+0000","last_change":"2024-02-26T10:42:20.605841+0000","last_active":"2024-02-26T10:42:21.919011+0000","last_peered":"2024-02-26T10:42:21.919011+0000","last_clean":"2024-02-26T10:42:21.919011+0000","last_became_active":"2024-02-26T10:42:20.605501+0000","last_became_peered":"2024-02-26T10:42:20.605501+0000","last_unstale":"2024-02-26T10:42:21.919011+0000","last_undegraded":"2024-02-26T10:42:21.919011+0000","last_fullsized":"2024-02-26T10:42:21.919011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:43:19.105024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593520+0000","last_change":"2024-02-26T10:41:59.306729+0000","last_active":"2024-02-26T10:42:21.593520+0000","last_peered":"2024-02-26T10:42:21.593520+0000","last_clean":"2024-02-26T10:42:21.593520+0000","last_became_active":"2024-02-26T10:41:59.306464+0000","last_became_peered":"2024-02-26T10:41:59.306464+0000","last_unstale":"2024-02-26T10:42:21.593520+0000","last_undegraded":"2024-02-26T10:42:21.593520+0000","last_fullsized":"2024-02-26T10:42:21.593520+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:10:56.409926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606312+0000","last_change":"2024-02-26T10:42:20.593902+0000","last_active":"2024-02-26T10:42:21.606312+0000","last_peered":"2024-02-26T10:42:21.606312+0000","last_clean":"2024-02-26T10:42:21.606312+0000","last_became_active":"2024-02-26T10:42:20.592385+0000","last_became_peered":"2024-02-26T10:42:20.592385+0000","last_unstale":"2024-02-26T10:42:21.606312+0000","last_undegraded":"2024-02-26T10:42:21.606312+0000","last_fullsized":"2024-02-26T10:42:21.606312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:07:57.945835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920232+0000","last_change":"2024-02-26T10:41:59.304328+0000","last_active":"2024-02-26T10:42:21.920232+0000","last_peered":"2024-02-26T10:42:21.920232+0000","last_clean":"2024-02-26T10:42:21.920232+0000","last_became_active":"2024-02-26T10:41:59.304025+0000","last_became_peered":"2024-02-26T10:41:59.304025+0000","last_unstale":"2024-02-26T10:42:21.920232+0000","last_undegraded":"2024-02-26T10:42:21.920232+0000","last_fullsized":"2024-02-26T10:42:21.920232+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595614+0000","last_change":"2024-02-26T10:42:20.612258+0000","last_active":"2024-02-26T10:42:21.595614+0000","last_peered":"2024-02-26T10:42:21.595614+0000","last_clean":"2024-02-26T10:42:21.595614+0000","last_became_active":"2024-02-26T10:42:20.610215+0000","last_became_peered":"2024-02-26T10:42:20.610215+0000","last_unstale":"2024-02-26T10:42:21.595614+0000","last_undegraded":"2024-02-26T10:42:21.595614+0000","last_fullsized":"2024-02-26T10:42:21.595614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:02:48.385871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606580+0000","last_change":"2024-02-26T10:41:59.303897+0000","last_active":"2024-02-26T10:42:21.606580+0000","last_peered":"2024-02-26T10:42:21.606580+0000","last_clean":"2024-02-26T10:42:21.606580+0000","last_became_active":"2024-02-26T10:41:59.301812+0000","last_became_peered":"2024-02-26T10:41:59.301812+0000","last_unstale":"2024-02-26T10:42:21.606580+0000","last_undegraded":"2024-02-26T10:42:21.606580+0000","last_fullsized":"2024-02-26T10:42:21.606580+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:03:57.549048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920949+0000","last_change":"2024-02-26T10:42:20.608517+0000","last_active":"2024-02-26T10:42:21.920949+0000","last_peered":"2024-02-26T10:42:21.920949+0000","last_clean":"2024-02-26T10:42:21.920949+0000","last_became_active":"2024-02-26T10:42:20.606683+0000","last_became_peered":"2024-02-26T10:42:20.606683+0000","last_unstale":"2024-02-26T10:42:21.920949+0000","last_undegraded":"2024-02-26T10:42:21.920949+0000","last_fullsized":"2024-02-26T10:42:21.920949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:00.537776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594217+0000","last_change":"2024-02-26T10:41:59.297092+0000","last_active":"2024-02-26T10:42:21.594217+0000","last_peered":"2024-02-26T10:42:21.594217+0000","last_clean":"2024-02-26T10:42:21.594217+0000","last_became_active":"2024-02-26T10:41:59.295816+0000","last_became_peered":"2024-02-26T10:41:59.295816+0000","last_unstale":"2024-02-26T10:42:21.594217+0000","last_undegraded":"2024-02-26T10:42:21.594217+0000","last_fullsized":"2024-02-26T10:42:21.594217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:05.502778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922832+0000","last_change":"2024-02-26T10:42:20.594955+0000","last_active":"2024-02-26T10:42:21.922832+0000","last_peered":"2024-02-26T10:42:21.922832+0000","last_clean":"2024-02-26T10:42:21.922832+0000","last_became_active":"2024-02-26T10:42:20.592013+0000","last_became_peered":"2024-02-26T10:42:20.592013+0000","last_unstale":"2024-02-26T10:42:21.922832+0000","last_undegraded":"2024-02-26T10:42:21.922832+0000","last_fullsized":"2024-02-26T10:42:21.922832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:39:31.051666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920890+0000","last_change":"2024-02-26T10:41:59.307675+0000","last_active":"2024-02-26T10:42:21.920890+0000","last_peered":"2024-02-26T10:42:21.920890+0000","last_clean":"2024-02-26T10:42:21.920890+0000","last_became_active":"2024-02-26T10:41:59.307360+0000","last_became_peered":"2024-02-26T10:41:59.307360+0000","last_unstale":"2024-02-26T10:42:21.920890+0000","last_undegraded":"2024-02-26T10:42:21.920890+0000","last_fullsized":"2024-02-26T10:42:21.920890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923737+0000","last_change":"2024-02-26T10:42:20.613331+0000","last_active":"2024-02-26T10:42:21.923737+0000","last_peered":"2024-02-26T10:42:21.923737+0000","last_clean":"2024-02-26T10:42:21.923737+0000","last_became_active":"2024-02-26T10:42:20.613039+0000","last_became_peered":"2024-02-26T10:42:20.613039+0000","last_unstale":"2024-02-26T10:42:21.923737+0000","last_undegraded":"2024-02-26T10:42:21.923737+0000","last_fullsized":"2024-02-26T10:42:21.923737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599200+0000","last_change":"2024-02-26T10:41:59.287990+0000","last_active":"2024-02-26T10:42:21.599200+0000","last_peered":"2024-02-26T10:42:21.599200+0000","last_clean":"2024-02-26T10:42:21.599200+0000","last_became_active":"2024-02-26T10:41:59.286925+0000","last_became_peered":"2024-02-26T10:41:59.286925+0000","last_unstale":"2024-02-26T10:42:21.599200+0000","last_undegraded":"2024-02-26T10:42:21.599200+0000","last_fullsized":"2024-02-26T10:42:21.599200+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:08.646623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597644+0000","last_change":"2024-02-26T10:42:20.611088+0000","last_active":"2024-02-26T10:42:21.597644+0000","last_peered":"2024-02-26T10:42:21.597644+0000","last_clean":"2024-02-26T10:42:21.597644+0000","last_became_active":"2024-02-26T10:42:20.610791+0000","last_became_peered":"2024-02-26T10:42:20.610791+0000","last_unstale":"2024-02-26T10:42:21.597644+0000","last_undegraded":"2024-02-26T10:42:21.597644+0000","last_fullsized":"2024-02-26T10:42:21.597644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:28:46.312779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922263+0000","last_change":"2024-02-26T10:41:59.297459+0000","last_active":"2024-02-26T10:42:21.922263+0000","last_peered":"2024-02-26T10:42:21.922263+0000","last_clean":"2024-02-26T10:42:21.922263+0000","last_became_active":"2024-02-26T10:41:59.296494+0000","last_became_peered":"2024-02-26T10:41:59.296494+0000","last_unstale":"2024-02-26T10:42:21.922263+0000","last_undegraded":"2024-02-26T10:42:21.922263+0000","last_fullsized":"2024-02-26T10:42:21.922263+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919427+0000","last_change":"2024-02-26T10:42:20.587485+0000","last_active":"2024-02-26T10:42:21.919427+0000","last_peered":"2024-02-26T10:42:21.919427+0000","last_clean":"2024-02-26T10:42:21.919427+0000","last_became_active":"2024-02-26T10:42:20.587190+0000","last_became_peered":"2024-02-26T10:42:20.587190+0000","last_unstale":"2024-02-26T10:42:21.919427+0000","last_undegraded":"2024-02-26T10:42:21.919427+0000","last_fullsized":"2024-02-26T10:42:21.919427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:28:05.289716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593987+0000","last_change":"2024-02-26T10:41:59.288740+0000","last_active":"2024-02-26T10:42:21.593987+0000","last_peered":"2024-02-26T10:42:21.593987+0000","last_clean":"2024-02-26T10:42:21.593987+0000","last_became_active":"2024-02-26T10:41:59.288433+0000","last_became_peered":"2024-02-26T10:41:59.288433+0000","last_unstale":"2024-02-26T10:42:21.593987+0000","last_undegraded":"2024-02-26T10:42:21.593987+0000","last_fullsized":"2024-02-26T10:42:21.593987+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:57:21.766000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.592835+0000","last_change":"2024-02-26T10:42:20.614242+0000","last_active":"2024-02-26T10:42:21.592835+0000","last_peered":"2024-02-26T10:42:21.592835+0000","last_clean":"2024-02-26T10:42:21.592835+0000","last_became_active":"2024-02-26T10:42:20.613805+0000","last_became_peered":"2024-02-26T10:42:20.613805+0000","last_unstale":"2024-02-26T10:42:21.592835+0000","last_undegraded":"2024-02-26T10:42:21.592835+0000","last_fullsized":"2024-02-26T10:42:21.592835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:57.832401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593626+0000","last_change":"2024-02-26T10:41:59.293103+0000","last_active":"2024-02-26T10:42:21.593626+0000","last_peered":"2024-02-26T10:42:21.593626+0000","last_clean":"2024-02-26T10:42:21.593626+0000","last_became_active":"2024-02-26T10:41:59.292473+0000","last_became_peered":"2024-02-26T10:41:59.292473+0000","last_unstale":"2024-02-26T10:42:21.593626+0000","last_undegraded":"2024-02-26T10:42:21.593626+0000","last_fullsized":"2024-02-26T10:42:21.593626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:54:37.822421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596457+0000","last_change":"2024-02-26T10:42:20.611052+0000","last_active":"2024-02-26T10:42:21.596457+0000","last_peered":"2024-02-26T10:42:21.596457+0000","last_clean":"2024-02-26T10:42:21.596457+0000","last_became_active":"2024-02-26T10:42:20.610750+0000","last_became_peered":"2024-02-26T10:42:20.610750+0000","last_unstale":"2024-02-26T10:42:21.596457+0000","last_undegraded":"2024-02-26T10:42:21.596457+0000","last_fullsized":"2024-02-26T10:42:21.596457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:36:08.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597980+0000","last_change":"2024-02-26T10:41:59.294484+0000","last_active":"2024-02-26T10:42:21.597980+0000","last_peered":"2024-02-26T10:42:21.597980+0000","last_clean":"2024-02-26T10:42:21.597980+0000","last_became_active":"2024-02-26T10:41:59.293628+0000","last_became_peered":"2024-02-26T10:41:59.293628+0000","last_unstale":"2024-02-26T10:42:21.597980+0000","last_undegraded":"2024-02-26T10:42:21.597980+0000","last_fullsized":"2024-02-26T10:42:21.597980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:04.694202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.608162+0000","last_change":"2024-02-26T10:42:20.603849+0000","last_active":"2024-02-26T10:42:21.608162+0000","last_peered":"2024-02-26T10:42:21.608162+0000","last_clean":"2024-02-26T10:42:21.608162+0000","last_became_active":"2024-02-26T10:42:20.602569+0000","last_became_peered":"2024-02-26T10:42:20.602569+0000","last_unstale":"2024-02-26T10:42:21.608162+0000","last_undegraded":"2024-02-26T10:42:21.608162+0000","last_fullsized":"2024-02-26T10:42:21.608162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:47:28.179744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918698+0000","last_change":"2024-02-26T10:41:59.289588+0000","last_active":"2024-02-26T10:42:21.918698+0000","last_peered":"2024-02-26T10:42:21.918698+0000","last_clean":"2024-02-26T10:42:21.918698+0000","last_became_active":"2024-02-26T10:41:59.289223+0000","last_became_peered":"2024-02-26T10:41:59.289223+0000","last_unstale":"2024-02-26T10:42:21.918698+0000","last_undegraded":"2024-02-26T10:42:21.918698+0000","last_fullsized":"2024-02-26T10:42:21.918698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594035+0000","last_change":"2024-02-26T10:42:20.609260+0000","last_active":"2024-02-26T10:42:21.594035+0000","last_peered":"2024-02-26T10:42:21.594035+0000","last_clean":"2024-02-26T10:42:21.594035+0000","last_became_active":"2024-02-26T10:42:20.608926+0000","last_became_peered":"2024-02-26T10:42:20.608926+0000","last_unstale":"2024-02-26T10:42:21.594035+0000","last_undegraded":"2024-02-26T10:42:21.594035+0000","last_fullsized":"2024-02-26T10:42:21.594035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:49.006673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597323+0000","last_change":"2024-02-26T10:42:19.567980+0000","last_active":"2024-02-26T10:42:21.597323+0000","last_peered":"2024-02-26T10:42:21.597323+0000","last_clean":"2024-02-26T10:42:21.597323+0000","last_became_active":"2024-02-26T10:42:19.567535+0000","last_became_peered":"2024-02-26T10:42:19.567535+0000","last_unstale":"2024-02-26T10:42:21.597323+0000","last_undegraded":"2024-02-26T10:42:21.597323+0000","last_fullsized":"2024-02-26T10:42:21.597323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:20.708487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599467+0000","last_change":"2024-02-26T10:41:59.296979+0000","last_active":"2024-02-26T10:42:21.599467+0000","last_peered":"2024-02-26T10:42:21.599467+0000","last_clean":"2024-02-26T10:42:21.599467+0000","last_became_active":"2024-02-26T10:41:59.295612+0000","last_became_peered":"2024-02-26T10:41:59.295612+0000","last_unstale":"2024-02-26T10:42:21.599467+0000","last_undegraded":"2024-02-26T10:42:21.599467+0000","last_fullsized":"2024-02-26T10:42:21.599467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:27:53.564566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"75'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923351+0000","last_change":"2024-02-26T10:41:58.274013+0000","last_active":"2024-02-26T10:42:21.923351+0000","last_peered":"2024-02-26T10:42:21.923351+0000","last_clean":"2024-02-26T10:42:21.923351+0000","last_became_active":"2024-02-26T10:41:58.273685+0000","last_became_peered":"2024-02-26T10:41:58.273685+0000","last_unstale":"2024-02-26T10:42:21.923351+0000","last_undegraded":"2024-02-26T10:42:21.923351+0000","last_fullsized":"2024-02-26T10:42:21.923351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:10:03.513627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599200+0000","last_change":"2024-02-26T10:42:20.604806+0000","last_active":"2024-02-26T10:42:21.599200+0000","last_peered":"2024-02-26T10:42:21.599200+0000","last_clean":"2024-02-26T10:42:21.599200+0000","last_became_active":"2024-02-26T10:42:20.603712+0000","last_became_peered":"2024-02-26T10:42:20.603712+0000","last_unstale":"2024-02-26T10:42:21.599200+0000","last_undegraded":"2024-02-26T10:42:21.599200+0000","last_fullsized":"2024-02-26T10:42:21.599200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:04.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596571+0000","last_change":"2024-02-26T10:42:19.571532+0000","last_active":"2024-02-26T10:42:21.596571+0000","last_peered":"2024-02-26T10:42:21.596571+0000","last_clean":"2024-02-26T10:42:21.596571+0000","last_became_active":"2024-02-26T10:42:19.570733+0000","last_became_peered":"2024-02-26T10:42:19.570733+0000","last_unstale":"2024-02-26T10:42:21.596571+0000","last_undegraded":"2024-02-26T10:42:21.596571+0000","last_fullsized":"2024-02-26T10:42:21.596571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:15:50.195046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923169+0000","last_change":"2024-02-26T10:41:59.304661+0000","last_active":"2024-02-26T10:42:21.923169+0000","last_peered":"2024-02-26T10:42:21.923169+0000","last_clean":"2024-02-26T10:42:21.923169+0000","last_became_active":"2024-02-26T10:41:59.304358+0000","last_became_peered":"2024-02-26T10:41:59.304358+0000","last_unstale":"2024-02-26T10:42:21.923169+0000","last_undegraded":"2024-02-26T10:42:21.923169+0000","last_fullsized":"2024-02-26T10:42:21.923169+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599815+0000","last_change":"2024-02-26T10:41:58.269527+0000","last_active":"2024-02-26T10:42:21.599815+0000","last_peered":"2024-02-26T10:42:21.599815+0000","last_clean":"2024-02-26T10:42:21.599815+0000","last_became_active":"2024-02-26T10:41:58.269066+0000","last_became_peered":"2024-02-26T10:41:58.269066+0000","last_unstale":"2024-02-26T10:42:21.599815+0000","last_undegraded":"2024-02-26T10:42:21.599815+0000","last_fullsized":"2024-02-26T10:42:21.599815+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:09:07.218989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607028+0000","last_change":"2024-02-26T10:42:20.602128+0000","last_active":"2024-02-26T10:42:21.607028+0000","last_peered":"2024-02-26T10:42:21.607028+0000","last_clean":"2024-02-26T10:42:21.607028+0000","last_became_active":"2024-02-26T10:42:20.601767+0000","last_became_peered":"2024-02-26T10:42:20.601767+0000","last_unstale":"2024-02-26T10:42:21.607028+0000","last_undegraded":"2024-02-26T10:42:21.607028+0000","last_fullsized":"2024-02-26T10:42:21.607028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:10:04.038444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"79'4","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597502+0000","last_change":"2024-02-26T10:42:19.566680+0000","last_active":"2024-02-26T10:42:21.597502+0000","last_peered":"2024-02-26T10:42:21.597502+0000","last_clean":"2024-02-26T10:42:21.597502+0000","last_became_active":"2024-02-26T10:42:19.566472+0000","last_became_peered":"2024-02-26T10:42:19.566472+0000","last_unstale":"2024-02-26T10:42:21.597502+0000","last_undegraded":"2024-02-26T10:42:21.597502+0000","last_fullsized":"2024-02-26T10:42:21.597502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:48:11.192552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922577+0000","last_change":"2024-02-26T10:41:59.310348+0000","last_active":"2024-02-26T10:42:21.922577+0000","last_peered":"2024-02-26T10:42:21.922577+0000","last_clean":"2024-02-26T10:42:21.922577+0000","last_became_active":"2024-02-26T10:41:59.309976+0000","last_became_peered":"2024-02-26T10:41:59.309976+0000","last_unstale":"2024-02-26T10:42:21.922577+0000","last_undegraded":"2024-02-26T10:42:21.922577+0000","last_fullsized":"2024-02-26T10:42:21.922577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:16.134260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597391+0000","last_change":"2024-02-26T10:41:58.272842+0000","last_active":"2024-02-26T10:42:21.597391+0000","last_peered":"2024-02-26T10:42:21.597391+0000","last_clean":"2024-02-26T10:42:21.597391+0000","last_became_active":"2024-02-26T10:41:58.272464+0000","last_became_peered":"2024-02-26T10:41:58.272464+0000","last_unstale":"2024-02-26T10:42:21.597391+0000","last_undegraded":"2024-02-26T10:42:21.597391+0000","last_fullsized":"2024-02-26T10:42:21.597391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:15.932239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920454+0000","last_change":"2024-02-26T10:42:20.589965+0000","last_active":"2024-02-26T10:42:21.920454+0000","last_peered":"2024-02-26T10:42:21.920454+0000","last_clean":"2024-02-26T10:42:21.920454+0000","last_became_active":"2024-02-26T10:42:20.589576+0000","last_became_peered":"2024-02-26T10:42:20.589576+0000","last_unstale":"2024-02-26T10:42:21.920454+0000","last_undegraded":"2024-02-26T10:42:21.920454+0000","last_fullsized":"2024-02-26T10:42:21.920454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:14:00.537776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921464+0000","last_change":"2024-02-26T10:42:19.572136+0000","last_active":"2024-02-26T10:42:21.921464+0000","last_peered":"2024-02-26T10:42:21.921464+0000","last_clean":"2024-02-26T10:42:21.921464+0000","last_became_active":"2024-02-26T10:42:19.571490+0000","last_became_peered":"2024-02-26T10:42:19.571490+0000","last_unstale":"2024-02-26T10:42:21.921464+0000","last_undegraded":"2024-02-26T10:42:21.921464+0000","last_fullsized":"2024-02-26T10:42:21.921464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:20.708487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920789+0000","last_change":"2024-02-26T10:41:59.287952+0000","last_active":"2024-02-26T10:42:21.920789+0000","last_peered":"2024-02-26T10:42:21.920789+0000","last_clean":"2024-02-26T10:42:21.920789+0000","last_became_active":"2024-02-26T10:41:59.287395+0000","last_became_peered":"2024-02-26T10:41:59.287395+0000","last_unstale":"2024-02-26T10:42:21.920789+0000","last_undegraded":"2024-02-26T10:42:21.920789+0000","last_fullsized":"2024-02-26T10:42:21.920789+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:43:54.144333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596900+0000","last_change":"2024-02-26T10:41:58.264925+0000","last_active":"2024-02-26T10:42:21.596900+0000","last_peered":"2024-02-26T10:42:21.596900+0000","last_clean":"2024-02-26T10:42:21.596900+0000","last_became_active":"2024-02-26T10:41:58.264514+0000","last_became_peered":"2024-02-26T10:41:58.264514+0000","last_unstale":"2024-02-26T10:42:21.596900+0000","last_undegraded":"2024-02-26T10:42:21.596900+0000","last_fullsized":"2024-02-26T10:42:21.596900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919224+0000","last_change":"2024-02-26T10:42:20.600458+0000","last_active":"2024-02-26T10:42:21.919224+0000","last_peered":"2024-02-26T10:42:21.919224+0000","last_clean":"2024-02-26T10:42:21.919224+0000","last_became_active":"2024-02-26T10:42:20.600119+0000","last_became_peered":"2024-02-26T10:42:20.600119+0000","last_unstale":"2024-02-26T10:42:21.919224+0000","last_undegraded":"2024-02-26T10:42:21.919224+0000","last_fullsized":"2024-02-26T10:42:21.919224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:43:19.105024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920102+0000","last_change":"2024-02-26T10:42:19.568599+0000","last_active":"2024-02-26T10:42:21.920102+0000","last_peered":"2024-02-26T10:42:21.920102+0000","last_clean":"2024-02-26T10:42:21.920102+0000","last_became_active":"2024-02-26T10:42:19.568102+0000","last_became_peered":"2024-02-26T10:42:19.568102+0000","last_unstale":"2024-02-26T10:42:21.920102+0000","last_undegraded":"2024-02-26T10:42:21.920102+0000","last_fullsized":"2024-02-26T10:42:21.920102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:31.472935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593727+0000","last_change":"2024-02-26T10:41:59.292771+0000","last_active":"2024-02-26T10:42:21.593727+0000","last_peered":"2024-02-26T10:42:21.593727+0000","last_clean":"2024-02-26T10:42:21.593727+0000","last_became_active":"2024-02-26T10:41:59.288778+0000","last_became_peered":"2024-02-26T10:41:59.288778+0000","last_unstale":"2024-02-26T10:42:21.593727+0000","last_undegraded":"2024-02-26T10:42:21.593727+0000","last_fullsized":"2024-02-26T10:42:21.593727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:05:18.908425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598899+0000","last_change":"2024-02-26T10:41:58.272643+0000","last_active":"2024-02-26T10:42:21.598899+0000","last_peered":"2024-02-26T10:42:21.598899+0000","last_clean":"2024-02-26T10:42:21.598899+0000","last_became_active":"2024-02-26T10:41:58.271814+0000","last_became_peered":"2024-02-26T10:41:58.271814+0000","last_unstale":"2024-02-26T10:42:21.598899+0000","last_undegraded":"2024-02-26T10:42:21.598899+0000","last_fullsized":"2024-02-26T10:42:21.598899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:18:20.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597947+0000","last_change":"2024-02-26T10:42:20.601343+0000","last_active":"2024-02-26T10:42:21.597947+0000","last_peered":"2024-02-26T10:42:21.597947+0000","last_clean":"2024-02-26T10:42:21.597947+0000","last_became_active":"2024-02-26T10:42:20.601093+0000","last_became_peered":"2024-02-26T10:42:20.601093+0000","last_unstale":"2024-02-26T10:42:21.597947+0000","last_undegraded":"2024-02-26T10:42:21.597947+0000","last_fullsized":"2024-02-26T10:42:21.597947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:49.653558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.592859+0000","last_change":"2024-02-26T10:42:19.552098+0000","last_active":"2024-02-26T10:42:21.592859+0000","last_peered":"2024-02-26T10:42:21.592859+0000","last_clean":"2024-02-26T10:42:21.592859+0000","last_became_active":"2024-02-26T10:42:19.551763+0000","last_became_peered":"2024-02-26T10:42:19.551763+0000","last_unstale":"2024-02-26T10:42:21.592859+0000","last_undegraded":"2024-02-26T10:42:21.592859+0000","last_fullsized":"2024-02-26T10:42:21.592859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:55:37.228234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598678+0000","last_change":"2024-02-26T10:41:59.287430+0000","last_active":"2024-02-26T10:42:21.598678+0000","last_peered":"2024-02-26T10:42:21.598678+0000","last_clean":"2024-02-26T10:42:21.598678+0000","last_became_active":"2024-02-26T10:41:59.287010+0000","last_became_peered":"2024-02-26T10:41:59.287010+0000","last_unstale":"2024-02-26T10:42:21.598678+0000","last_undegraded":"2024-02-26T10:42:21.598678+0000","last_fullsized":"2024-02-26T10:42:21.598678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:24.065359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597856+0000","last_change":"2024-02-26T10:41:58.270613+0000","last_active":"2024-02-26T10:42:21.597856+0000","last_peered":"2024-02-26T10:42:21.597856+0000","last_clean":"2024-02-26T10:42:21.597856+0000","last_became_active":"2024-02-26T10:41:58.270296+0000","last_became_peered":"2024-02-26T10:41:58.270296+0000","last_unstale":"2024-02-26T10:42:21.597856+0000","last_undegraded":"2024-02-26T10:42:21.597856+0000","last_fullsized":"2024-02-26T10:42:21.597856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:19.393230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732551+0000","last_change":"2024-02-26T10:42:20.604589+0000","last_active":"2024-02-26T10:42:21.732551+0000","last_peered":"2024-02-26T10:42:21.732551+0000","last_clean":"2024-02-26T10:42:21.732551+0000","last_became_active":"2024-02-26T10:42:20.604291+0000","last_became_peered":"2024-02-26T10:42:20.604291+0000","last_unstale":"2024-02-26T10:42:21.732551+0000","last_undegraded":"2024-02-26T10:42:21.732551+0000","last_fullsized":"2024-02-26T10:42:21.732551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:21.722907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593560+0000","last_change":"2024-02-26T10:42:19.566766+0000","last_active":"2024-02-26T10:42:21.593560+0000","last_peered":"2024-02-26T10:42:21.593560+0000","last_clean":"2024-02-26T10:42:21.593560+0000","last_became_active":"2024-02-26T10:42:19.566484+0000","last_became_peered":"2024-02-26T10:42:19.566484+0000","last_unstale":"2024-02-26T10:42:21.593560+0000","last_undegraded":"2024-02-26T10:42:21.593560+0000","last_fullsized":"2024-02-26T10:42:21.593560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:25:44.830713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918662+0000","last_change":"2024-02-26T10:41:59.301966+0000","last_active":"2024-02-26T10:42:21.918662+0000","last_peered":"2024-02-26T10:42:21.918662+0000","last_clean":"2024-02-26T10:42:21.918662+0000","last_became_active":"2024-02-26T10:41:59.301282+0000","last_became_peered":"2024-02-26T10:41:59.301282+0000","last_unstale":"2024-02-26T10:42:21.918662+0000","last_undegraded":"2024-02-26T10:42:21.918662+0000","last_fullsized":"2024-02-26T10:42:21.918662+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:36.536906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595885+0000","last_change":"2024-02-26T10:41:58.253825+0000","last_active":"2024-02-26T10:42:21.595885+0000","last_peered":"2024-02-26T10:42:21.595885+0000","last_clean":"2024-02-26T10:42:21.595885+0000","last_became_active":"2024-02-26T10:41:58.253549+0000","last_became_peered":"2024-02-26T10:41:58.253549+0000","last_unstale":"2024-02-26T10:42:21.595885+0000","last_undegraded":"2024-02-26T10:42:21.595885+0000","last_fullsized":"2024-02-26T10:42:21.595885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:43:07.631703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920116+0000","last_change":"2024-02-26T10:42:20.604569+0000","last_active":"2024-02-26T10:42:21.920116+0000","last_peered":"2024-02-26T10:42:21.920116+0000","last_clean":"2024-02-26T10:42:21.920116+0000","last_became_active":"2024-02-26T10:42:20.603190+0000","last_became_peered":"2024-02-26T10:42:20.603190+0000","last_unstale":"2024-02-26T10:42:21.920116+0000","last_undegraded":"2024-02-26T10:42:21.920116+0000","last_fullsized":"2024-02-26T10:42:21.920116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:39:31.051666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595740+0000","last_change":"2024-02-26T10:42:19.572424+0000","last_active":"2024-02-26T10:42:21.595740+0000","last_peered":"2024-02-26T10:42:21.595740+0000","last_clean":"2024-02-26T10:42:21.595740+0000","last_became_active":"2024-02-26T10:42:19.572019+0000","last_became_peered":"2024-02-26T10:42:19.572019+0000","last_unstale":"2024-02-26T10:42:21.595740+0000","last_undegraded":"2024-02-26T10:42:21.595740+0000","last_fullsized":"2024-02-26T10:42:21.595740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:49.405962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919505+0000","last_change":"2024-02-26T10:41:59.289078+0000","last_active":"2024-02-26T10:42:21.919505+0000","last_peered":"2024-02-26T10:42:21.919505+0000","last_clean":"2024-02-26T10:42:21.919505+0000","last_became_active":"2024-02-26T10:41:59.288529+0000","last_became_peered":"2024-02-26T10:41:59.288529+0000","last_unstale":"2024-02-26T10:42:21.919505+0000","last_undegraded":"2024-02-26T10:42:21.919505+0000","last_fullsized":"2024-02-26T10:42:21.919505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:08:28.358063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593343+0000","last_change":"2024-02-26T10:41:58.265531+0000","last_active":"2024-02-26T10:42:21.593343+0000","last_peered":"2024-02-26T10:42:21.593343+0000","last_clean":"2024-02-26T10:42:21.593343+0000","last_became_active":"2024-02-26T10:41:58.265148+0000","last_became_peered":"2024-02-26T10:41:58.265148+0000","last_unstale":"2024-02-26T10:42:21.593343+0000","last_undegraded":"2024-02-26T10:42:21.593343+0000","last_fullsized":"2024-02-26T10:42:21.593343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:39:56.459437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921892+0000","last_change":"2024-02-26T10:42:20.613586+0000","last_active":"2024-02-26T10:42:21.921892+0000","last_peered":"2024-02-26T10:42:21.921892+0000","last_clean":"2024-02-26T10:42:21.921892+0000","last_became_active":"2024-02-26T10:42:20.612911+0000","last_became_peered":"2024-02-26T10:42:20.612911+0000","last_unstale":"2024-02-26T10:42:21.921892+0000","last_undegraded":"2024-02-26T10:42:21.921892+0000","last_fullsized":"2024-02-26T10:42:21.921892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:28:05.289716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"79'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921212+0000","last_change":"2024-02-26T10:42:19.571927+0000","last_active":"2024-02-26T10:42:21.921212+0000","last_peered":"2024-02-26T10:42:21.921212+0000","last_clean":"2024-02-26T10:42:21.921212+0000","last_became_active":"2024-02-26T10:42:19.570459+0000","last_became_peered":"2024-02-26T10:42:19.570459+0000","last_unstale":"2024-02-26T10:42:21.921212+0000","last_undegraded":"2024-02-26T10:42:21.921212+0000","last_fullsized":"2024-02-26T10:42:21.921212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:28:55.454669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920536+0000","last_change":"2024-02-26T10:41:59.294420+0000","last_active":"2024-02-26T10:42:21.920536+0000","last_peered":"2024-02-26T10:42:21.920536+0000","last_clean":"2024-02-26T10:42:21.920536+0000","last_became_active":"2024-02-26T10:41:59.294147+0000","last_became_peered":"2024-02-26T10:41:59.294147+0000","last_unstale":"2024-02-26T10:42:21.920536+0000","last_undegraded":"2024-02-26T10:42:21.920536+0000","last_fullsized":"2024-02-26T10:42:21.920536+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:32.774644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923542+0000","last_change":"2024-02-26T10:41:58.270752+0000","last_active":"2024-02-26T10:42:21.923542+0000","last_peered":"2024-02-26T10:42:21.923542+0000","last_clean":"2024-02-26T10:42:21.923542+0000","last_became_active":"2024-02-26T10:41:58.270094+0000","last_became_peered":"2024-02-26T10:41:58.270094+0000","last_unstale":"2024-02-26T10:42:21.923542+0000","last_undegraded":"2024-02-26T10:42:21.923542+0000","last_fullsized":"2024-02-26T10:42:21.923542+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:46:32.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920057+0000","last_change":"2024-02-26T10:42:20.614388+0000","last_active":"2024-02-26T10:42:21.920057+0000","last_peered":"2024-02-26T10:42:21.920057+0000","last_clean":"2024-02-26T10:42:21.920057+0000","last_became_active":"2024-02-26T10:42:20.614082+0000","last_became_peered":"2024-02-26T10:42:20.614082+0000","last_unstale":"2024-02-26T10:42:21.920057+0000","last_undegraded":"2024-02-26T10:42:21.920057+0000","last_fullsized":"2024-02-26T10:42:21.920057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:22.708651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"79'6","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599338+0000","last_change":"2024-02-26T10:42:19.552022+0000","last_active":"2024-02-26T10:42:21.599338+0000","last_peered":"2024-02-26T10:42:21.599338+0000","last_clean":"2024-02-26T10:42:21.599338+0000","last_became_active":"2024-02-26T10:42:19.551436+0000","last_became_peered":"2024-02-26T10:42:19.551436+0000","last_unstale":"2024-02-26T10:42:21.599338+0000","last_undegraded":"2024-02-26T10:42:21.599338+0000","last_fullsized":"2024-02-26T10:42:21.599338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:49.253673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732649+0000","last_change":"2024-02-26T10:41:59.296775+0000","last_active":"2024-02-26T10:42:21.732649+0000","last_peered":"2024-02-26T10:42:21.732649+0000","last_clean":"2024-02-26T10:42:21.732649+0000","last_became_active":"2024-02-26T10:41:59.296431+0000","last_became_peered":"2024-02-26T10:41:59.296431+0000","last_unstale":"2024-02-26T10:42:21.732649+0000","last_undegraded":"2024-02-26T10:42:21.732649+0000","last_fullsized":"2024-02-26T10:42:21.732649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922609+0000","last_change":"2024-02-26T10:41:58.274673+0000","last_active":"2024-02-26T10:42:21.922609+0000","last_peered":"2024-02-26T10:42:21.922609+0000","last_clean":"2024-02-26T10:42:21.922609+0000","last_became_active":"2024-02-26T10:41:58.273808+0000","last_became_peered":"2024-02-26T10:41:58.273808+0000","last_unstale":"2024-02-26T10:42:21.922609+0000","last_undegraded":"2024-02-26T10:42:21.922609+0000","last_fullsized":"2024-02-26T10:42:21.922609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T11:56:04.487859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596945+0000","last_change":"2024-02-26T10:42:20.613657+0000","last_active":"2024-02-26T10:42:21.596945+0000","last_peered":"2024-02-26T10:42:21.596945+0000","last_clean":"2024-02-26T10:42:21.596945+0000","last_became_active":"2024-02-26T10:42:20.613399+0000","last_became_peered":"2024-02-26T10:42:20.613399+0000","last_unstale":"2024-02-26T10:42:21.596945+0000","last_undegraded":"2024-02-26T10:42:21.596945+0000","last_fullsized":"2024-02-26T10:42:21.596945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:53:04.830480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594108+0000","last_change":"2024-02-26T10:42:19.562582+0000","last_active":"2024-02-26T10:42:21.594108+0000","last_peered":"2024-02-26T10:42:21.594108+0000","last_clean":"2024-02-26T10:42:21.594108+0000","last_became_active":"2024-02-26T10:42:19.562254+0000","last_became_peered":"2024-02-26T10:42:19.562254+0000","last_unstale":"2024-02-26T10:42:21.594108+0000","last_undegraded":"2024-02-26T10:42:21.594108+0000","last_fullsized":"2024-02-26T10:42:21.594108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:47.203598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596753+0000","last_change":"2024-02-26T10:41:59.300848+0000","last_active":"2024-02-26T10:42:21.596753+0000","last_peered":"2024-02-26T10:42:21.596753+0000","last_clean":"2024-02-26T10:42:21.596753+0000","last_became_active":"2024-02-26T10:41:59.300577+0000","last_became_peered":"2024-02-26T10:41:59.300577+0000","last_unstale":"2024-02-26T10:42:21.596753+0000","last_undegraded":"2024-02-26T10:42:21.596753+0000","last_fullsized":"2024-02-26T10:42:21.596753+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"75'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921964+0000","last_change":"2024-02-26T10:41:58.274487+0000","last_active":"2024-02-26T10:42:21.921964+0000","last_peered":"2024-02-26T10:42:21.921964+0000","last_clean":"2024-02-26T10:42:21.921964+0000","last_became_active":"2024-02-26T10:41:58.273986+0000","last_became_peered":"2024-02-26T10:41:58.273986+0000","last_unstale":"2024-02-26T10:42:21.921964+0000","last_undegraded":"2024-02-26T10:42:21.921964+0000","last_fullsized":"2024-02-26T10:42:21.921964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:41:03.679283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599987+0000","last_change":"2024-02-26T10:42:20.602702+0000","last_active":"2024-02-26T10:42:21.599987+0000","last_peered":"2024-02-26T10:42:21.599987+0000","last_clean":"2024-02-26T10:42:21.599987+0000","last_became_active":"2024-02-26T10:42:20.602103+0000","last_became_peered":"2024-02-26T10:42:20.602103+0000","last_unstale":"2024-02-26T10:42:21.599987+0000","last_undegraded":"2024-02-26T10:42:21.599987+0000","last_fullsized":"2024-02-26T10:42:21.599987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:32.487354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919802+0000","last_change":"2024-02-26T10:42:19.564508+0000","last_active":"2024-02-26T10:42:21.919802+0000","last_peered":"2024-02-26T10:42:21.919802+0000","last_clean":"2024-02-26T10:42:21.919802+0000","last_became_active":"2024-02-26T10:42:19.564189+0000","last_became_peered":"2024-02-26T10:42:19.564189+0000","last_unstale":"2024-02-26T10:42:21.919802+0000","last_undegraded":"2024-02-26T10:42:21.919802+0000","last_fullsized":"2024-02-26T10:42:21.919802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:15:50.195046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599487+0000","last_change":"2024-02-26T10:41:59.303988+0000","last_active":"2024-02-26T10:42:21.599487+0000","last_peered":"2024-02-26T10:42:21.599487+0000","last_clean":"2024-02-26T10:42:21.599487+0000","last_became_active":"2024-02-26T10:41:59.303677+0000","last_became_peered":"2024-02-26T10:41:59.303677+0000","last_unstale":"2024-02-26T10:42:21.599487+0000","last_undegraded":"2024-02-26T10:42:21.599487+0000","last_fullsized":"2024-02-26T10:42:21.599487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:08.404204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"76'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732288+0000","last_change":"2024-02-26T10:41:58.259980+0000","last_active":"2024-02-26T10:42:21.732288+0000","last_peered":"2024-02-26T10:42:21.732288+0000","last_clean":"2024-02-26T10:42:21.732288+0000","last_became_active":"2024-02-26T10:41:58.258722+0000","last_became_peered":"2024-02-26T10:41:58.258722+0000","last_unstale":"2024-02-26T10:42:21.732288+0000","last_undegraded":"2024-02-26T10:42:21.732288+0000","last_fullsized":"2024-02-26T10:42:21.732288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3115,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597726+0000","last_change":"2024-02-26T10:42:20.601327+0000","last_active":"2024-02-26T10:42:21.597726+0000","last_peered":"2024-02-26T10:42:21.597726+0000","last_clean":"2024-02-26T10:42:21.597726+0000","last_became_active":"2024-02-26T10:42:20.601024+0000","last_became_peered":"2024-02-26T10:42:20.601024+0000","last_unstale":"2024-02-26T10:42:21.597726+0000","last_undegraded":"2024-02-26T10:42:21.597726+0000","last_fullsized":"2024-02-26T10:42:21.597726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:21.908982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597832+0000","last_change":"2024-02-26T10:42:19.568035+0000","last_active":"2024-02-26T10:42:21.597832+0000","last_peered":"2024-02-26T10:42:21.597832+0000","last_clean":"2024-02-26T10:42:21.597832+0000","last_became_active":"2024-02-26T10:42:19.566434+0000","last_became_peered":"2024-02-26T10:42:19.566434+0000","last_unstale":"2024-02-26T10:42:21.597832+0000","last_undegraded":"2024-02-26T10:42:21.597832+0000","last_fullsized":"2024-02-26T10:42:21.597832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:09:56.817981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595923+0000","last_change":"2024-02-26T10:41:59.298661+0000","last_active":"2024-02-26T10:42:21.595923+0000","last_peered":"2024-02-26T10:42:21.595923+0000","last_clean":"2024-02-26T10:42:21.595923+0000","last_became_active":"2024-02-26T10:41:59.298384+0000","last_became_peered":"2024-02-26T10:41:59.298384+0000","last_unstale":"2024-02-26T10:42:21.595923+0000","last_undegraded":"2024-02-26T10:42:21.595923+0000","last_fullsized":"2024-02-26T10:42:21.595923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:54:12.995953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594203+0000","last_change":"2024-02-26T10:41:58.258608+0000","last_active":"2024-02-26T10:42:21.594203+0000","last_peered":"2024-02-26T10:42:21.594203+0000","last_clean":"2024-02-26T10:42:21.594203+0000","last_became_active":"2024-02-26T10:41:58.258324+0000","last_became_peered":"2024-02-26T10:41:58.258324+0000","last_unstale":"2024-02-26T10:42:21.594203+0000","last_undegraded":"2024-02-26T10:42:21.594203+0000","last_fullsized":"2024-02-26T10:42:21.594203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:37:06.345056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.605948+0000","last_change":"2024-02-26T10:42:20.606489+0000","last_active":"2024-02-26T10:42:21.605948+0000","last_peered":"2024-02-26T10:42:21.605948+0000","last_clean":"2024-02-26T10:42:21.605948+0000","last_became_active":"2024-02-26T10:42:20.606100+0000","last_became_peered":"2024-02-26T10:42:20.606100+0000","last_unstale":"2024-02-26T10:42:21.605948+0000","last_undegraded":"2024-02-26T10:42:21.605948+0000","last_fullsized":"2024-02-26T10:42:21.605948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:38:30.603159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"79'8","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593114+0000","last_change":"2024-02-26T10:42:19.568564+0000","last_active":"2024-02-26T10:42:21.593114+0000","last_peered":"2024-02-26T10:42:21.593114+0000","last_clean":"2024-02-26T10:42:21.593114+0000","last_became_active":"2024-02-26T10:42:19.568278+0000","last_became_peered":"2024-02-26T10:42:19.568278+0000","last_unstale":"2024-02-26T10:42:21.593114+0000","last_undegraded":"2024-02-26T10:42:21.593114+0000","last_fullsized":"2024-02-26T10:42:21.593114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:51:20.739806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920052+0000","last_change":"2024-02-26T10:41:59.303009+0000","last_active":"2024-02-26T10:42:21.920052+0000","last_peered":"2024-02-26T10:42:21.920052+0000","last_clean":"2024-02-26T10:42:21.920052+0000","last_became_active":"2024-02-26T10:41:59.301703+0000","last_became_peered":"2024-02-26T10:41:59.301703+0000","last_unstale":"2024-02-26T10:42:21.920052+0000","last_undegraded":"2024-02-26T10:42:21.920052+0000","last_fullsized":"2024-02-26T10:42:21.920052+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:52:27.157476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919878+0000","last_change":"2024-02-26T10:41:58.263654+0000","last_active":"2024-02-26T10:42:21.919878+0000","last_peered":"2024-02-26T10:42:21.919878+0000","last_clean":"2024-02-26T10:42:21.919878+0000","last_became_active":"2024-02-26T10:41:58.263367+0000","last_became_peered":"2024-02-26T10:41:58.263367+0000","last_unstale":"2024-02-26T10:42:21.919878+0000","last_undegraded":"2024-02-26T10:42:21.919878+0000","last_fullsized":"2024-02-26T10:42:21.919878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:15.119340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919738+0000","last_change":"2024-02-26T10:42:20.603915+0000","last_active":"2024-02-26T10:42:21.919738+0000","last_peered":"2024-02-26T10:42:21.919738+0000","last_clean":"2024-02-26T10:42:21.919738+0000","last_became_active":"2024-02-26T10:42:20.602756+0000","last_became_peered":"2024-02-26T10:42:20.602756+0000","last_unstale":"2024-02-26T10:42:21.919738+0000","last_undegraded":"2024-02-26T10:42:21.919738+0000","last_fullsized":"2024-02-26T10:42:21.919738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:22.708651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606964+0000","last_change":"2024-02-26T10:42:19.553678+0000","last_active":"2024-02-26T10:42:21.606964+0000","last_peered":"2024-02-26T10:42:21.606964+0000","last_clean":"2024-02-26T10:42:21.606964+0000","last_became_active":"2024-02-26T10:42:19.552678+0000","last_became_peered":"2024-02-26T10:42:19.552678+0000","last_unstale":"2024-02-26T10:42:21.606964+0000","last_undegraded":"2024-02-26T10:42:21.606964+0000","last_fullsized":"2024-02-26T10:42:21.606964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:48:23.347207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593333+0000","last_change":"2024-02-26T10:41:59.285657+0000","last_active":"2024-02-26T10:42:21.593333+0000","last_peered":"2024-02-26T10:42:21.593333+0000","last_clean":"2024-02-26T10:42:21.593333+0000","last_became_active":"2024-02-26T10:41:59.285365+0000","last_became_peered":"2024-02-26T10:41:59.285365+0000","last_unstale":"2024-02-26T10:42:21.593333+0000","last_undegraded":"2024-02-26T10:42:21.593333+0000","last_fullsized":"2024-02-26T10:42:21.593333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:18:10.689718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732027+0000","last_change":"2024-02-26T10:41:58.268013+0000","last_active":"2024-02-26T10:42:21.732027+0000","last_peered":"2024-02-26T10:42:21.732027+0000","last_clean":"2024-02-26T10:42:21.732027+0000","last_became_active":"2024-02-26T10:41:58.267703+0000","last_became_peered":"2024-02-26T10:41:58.267703+0000","last_unstale":"2024-02-26T10:42:21.732027+0000","last_undegraded":"2024-02-26T10:42:21.732027+0000","last_fullsized":"2024-02-26T10:42:21.732027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:01.684574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593758+0000","last_change":"2024-02-26T10:42:20.600269+0000","last_active":"2024-02-26T10:42:21.593758+0000","last_peered":"2024-02-26T10:42:21.593758+0000","last_clean":"2024-02-26T10:42:21.593758+0000","last_became_active":"2024-02-26T10:42:20.599903+0000","last_became_peered":"2024-02-26T10:42:20.599903+0000","last_unstale":"2024-02-26T10:42:21.593758+0000","last_undegraded":"2024-02-26T10:42:21.593758+0000","last_fullsized":"2024-02-26T10:42:21.593758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:22:37.429754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607741+0000","last_change":"2024-02-26T10:42:19.552038+0000","last_active":"2024-02-26T10:42:21.607741+0000","last_peered":"2024-02-26T10:42:21.607741+0000","last_clean":"2024-02-26T10:42:21.607741+0000","last_became_active":"2024-02-26T10:42:19.551727+0000","last_became_peered":"2024-02-26T10:42:19.551727+0000","last_unstale":"2024-02-26T10:42:21.607741+0000","last_undegraded":"2024-02-26T10:42:21.607741+0000","last_fullsized":"2024-02-26T10:42:21.607741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:52:36.656696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.592939+0000","last_change":"2024-02-26T10:41:59.299415+0000","last_active":"2024-02-26T10:42:21.592939+0000","last_peered":"2024-02-26T10:42:21.592939+0000","last_clean":"2024-02-26T10:42:21.592939+0000","last_became_active":"2024-02-26T10:41:59.298995+0000","last_became_peered":"2024-02-26T10:41:59.298995+0000","last_unstale":"2024-02-26T10:42:21.592939+0000","last_undegraded":"2024-02-26T10:42:21.592939+0000","last_fullsized":"2024-02-26T10:42:21.592939+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:13:38.058564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"75'8","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595731+0000","last_change":"2024-02-26T10:41:58.271729+0000","last_active":"2024-02-26T10:42:21.595731+0000","last_peered":"2024-02-26T10:42:21.595731+0000","last_clean":"2024-02-26T10:42:21.595731+0000","last_became_active":"2024-02-26T10:41:58.271453+0000","last_became_peered":"2024-02-26T10:41:58.271453+0000","last_unstale":"2024-02-26T10:42:21.595731+0000","last_undegraded":"2024-02-26T10:42:21.595731+0000","last_fullsized":"2024-02-26T10:42:21.595731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731651+0000","last_change":"2024-02-26T10:42:20.596696+0000","last_active":"2024-02-26T10:42:21.731651+0000","last_peered":"2024-02-26T10:42:21.731651+0000","last_clean":"2024-02-26T10:42:21.731651+0000","last_became_active":"2024-02-26T10:42:20.596339+0000","last_became_peered":"2024-02-26T10:42:20.596339+0000","last_unstale":"2024-02-26T10:42:21.731651+0000","last_undegraded":"2024-02-26T10:42:21.731651+0000","last_fullsized":"2024-02-26T10:42:21.731651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:04.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921901+0000","last_change":"2024-02-26T10:42:19.563227+0000","last_active":"2024-02-26T10:42:21.921901+0000","last_peered":"2024-02-26T10:42:21.921901+0000","last_clean":"2024-02-26T10:42:21.921901+0000","last_became_active":"2024-02-26T10:42:19.562881+0000","last_became_peered":"2024-02-26T10:42:19.562881+0000","last_unstale":"2024-02-26T10:42:21.921901+0000","last_undegraded":"2024-02-26T10:42:21.921901+0000","last_fullsized":"2024-02-26T10:42:21.921901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:10:20.708487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921256+0000","last_change":"2024-02-26T10:41:59.290032+0000","last_active":"2024-02-26T10:42:21.921256+0000","last_peered":"2024-02-26T10:42:21.921256+0000","last_clean":"2024-02-26T10:42:21.921256+0000","last_became_active":"2024-02-26T10:41:59.289604+0000","last_became_peered":"2024-02-26T10:41:59.289604+0000","last_unstale":"2024-02-26T10:42:21.921256+0000","last_undegraded":"2024-02-26T10:42:21.921256+0000","last_fullsized":"2024-02-26T10:42:21.921256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:23.411990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923889+0000","last_change":"2024-02-26T10:41:58.265015+0000","last_active":"2024-02-26T10:42:21.923889+0000","last_peered":"2024-02-26T10:42:21.923889+0000","last_clean":"2024-02-26T10:42:21.923889+0000","last_became_active":"2024-02-26T10:41:58.264203+0000","last_became_peered":"2024-02-26T10:41:58.264203+0000","last_unstale":"2024-02-26T10:42:21.923889+0000","last_undegraded":"2024-02-26T10:42:21.923889+0000","last_fullsized":"2024-02-26T10:42:21.923889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:58:55.400495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920669+0000","last_change":"2024-02-26T10:42:20.608827+0000","last_active":"2024-02-26T10:42:21.920669+0000","last_peered":"2024-02-26T10:42:21.920669+0000","last_clean":"2024-02-26T10:42:21.920669+0000","last_became_active":"2024-02-26T10:42:20.607639+0000","last_became_peered":"2024-02-26T10:42:20.607639+0000","last_unstale":"2024-02-26T10:42:21.920669+0000","last_undegraded":"2024-02-26T10:42:21.920669+0000","last_fullsized":"2024-02-26T10:42:21.920669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:34:12.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596686+0000","last_change":"2024-02-26T10:42:19.571833+0000","last_active":"2024-02-26T10:42:21.596686+0000","last_peered":"2024-02-26T10:42:21.596686+0000","last_clean":"2024-02-26T10:42:21.596686+0000","last_became_active":"2024-02-26T10:42:19.571240+0000","last_became_peered":"2024-02-26T10:42:19.571240+0000","last_unstale":"2024-02-26T10:42:21.596686+0000","last_undegraded":"2024-02-26T10:42:21.596686+0000","last_fullsized":"2024-02-26T10:42:21.596686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:31.100239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923218+0000","last_change":"2024-02-26T10:41:59.299545+0000","last_active":"2024-02-26T10:42:21.923218+0000","last_peered":"2024-02-26T10:42:21.923218+0000","last_clean":"2024-02-26T10:42:21.923218+0000","last_became_active":"2024-02-26T10:41:59.298970+0000","last_became_peered":"2024-02-26T10:41:59.298970+0000","last_unstale":"2024-02-26T10:42:21.923218+0000","last_undegraded":"2024-02-26T10:42:21.923218+0000","last_fullsized":"2024-02-26T10:42:21.923218+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:47.840854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597588+0000","last_change":"2024-02-26T10:41:58.268137+0000","last_active":"2024-02-26T10:42:21.597588+0000","last_peered":"2024-02-26T10:42:21.597588+0000","last_clean":"2024-02-26T10:42:21.597588+0000","last_became_active":"2024-02-26T10:41:58.267593+0000","last_became_peered":"2024-02-26T10:41:58.267593+0000","last_unstale":"2024-02-26T10:42:21.597588+0000","last_undegraded":"2024-02-26T10:42:21.597588+0000","last_fullsized":"2024-02-26T10:42:21.597588+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:07.389285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.923009+0000","last_change":"2024-02-26T10:42:20.601903+0000","last_active":"2024-02-26T10:42:21.923009+0000","last_peered":"2024-02-26T10:42:21.923009+0000","last_clean":"2024-02-26T10:42:21.923009+0000","last_became_active":"2024-02-26T10:42:20.600587+0000","last_became_peered":"2024-02-26T10:42:20.600587+0000","last_unstale":"2024-02-26T10:42:21.923009+0000","last_undegraded":"2024-02-26T10:42:21.923009+0000","last_fullsized":"2024-02-26T10:42:21.923009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:08:23.198928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599271+0000","last_change":"2024-02-26T10:42:19.557245+0000","last_active":"2024-02-26T10:42:21.599271+0000","last_peered":"2024-02-26T10:42:21.599271+0000","last_clean":"2024-02-26T10:42:21.599271+0000","last_became_active":"2024-02-26T10:42:19.556592+0000","last_became_peered":"2024-02-26T10:42:19.556592+0000","last_unstale":"2024-02-26T10:42:21.599271+0000","last_undegraded":"2024-02-26T10:42:21.599271+0000","last_fullsized":"2024-02-26T10:42:21.599271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:21.404901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599528+0000","last_change":"2024-02-26T10:41:59.283472+0000","last_active":"2024-02-26T10:42:21.599528+0000","last_peered":"2024-02-26T10:42:21.599528+0000","last_clean":"2024-02-26T10:42:21.599528+0000","last_became_active":"2024-02-26T10:41:59.282978+0000","last_became_peered":"2024-02-26T10:41:59.282978+0000","last_unstale":"2024-02-26T10:42:21.599528+0000","last_undegraded":"2024-02-26T10:42:21.599528+0000","last_fullsized":"2024-02-26T10:42:21.599528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:50.202422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596863+0000","last_change":"2024-02-26T10:41:58.262608+0000","last_active":"2024-02-26T10:42:21.596863+0000","last_peered":"2024-02-26T10:42:21.596863+0000","last_clean":"2024-02-26T10:42:21.596863+0000","last_became_active":"2024-02-26T10:41:58.262273+0000","last_became_peered":"2024-02-26T10:41:58.262273+0000","last_unstale":"2024-02-26T10:42:21.596863+0000","last_undegraded":"2024-02-26T10:42:21.596863+0000","last_fullsized":"2024-02-26T10:42:21.596863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:12:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607379+0000","last_change":"2024-02-26T10:42:20.590282+0000","last_active":"2024-02-26T10:42:21.607379+0000","last_peered":"2024-02-26T10:42:21.607379+0000","last_clean":"2024-02-26T10:42:21.607379+0000","last_became_active":"2024-02-26T10:42:20.589970+0000","last_became_peered":"2024-02-26T10:42:20.589970+0000","last_unstale":"2024-02-26T10:42:21.607379+0000","last_undegraded":"2024-02-26T10:42:21.607379+0000","last_fullsized":"2024-02-26T10:42:21.607379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:37:52.821056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731271+0000","last_change":"2024-02-26T10:42:19.555195+0000","last_active":"2024-02-26T10:42:21.731271+0000","last_peered":"2024-02-26T10:42:21.731271+0000","last_clean":"2024-02-26T10:42:21.731271+0000","last_became_active":"2024-02-26T10:42:19.554748+0000","last_became_peered":"2024-02-26T10:42:19.554748+0000","last_unstale":"2024-02-26T10:42:21.731271+0000","last_undegraded":"2024-02-26T10:42:21.731271+0000","last_fullsized":"2024-02-26T10:42:21.731271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:53:41.883453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607841+0000","last_change":"2024-02-26T10:41:59.297414+0000","last_active":"2024-02-26T10:42:21.607841+0000","last_peered":"2024-02-26T10:42:21.607841+0000","last_clean":"2024-02-26T10:42:21.607841+0000","last_became_active":"2024-02-26T10:41:59.296486+0000","last_became_peered":"2024-02-26T10:41:59.296486+0000","last_unstale":"2024-02-26T10:42:21.607841+0000","last_undegraded":"2024-02-26T10:42:21.607841+0000","last_fullsized":"2024-02-26T10:42:21.607841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:00:48.071684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920657+0000","last_change":"2024-02-26T10:41:58.274213+0000","last_active":"2024-02-26T10:42:21.920657+0000","last_peered":"2024-02-26T10:42:21.920657+0000","last_clean":"2024-02-26T10:42:21.920657+0000","last_became_active":"2024-02-26T10:41:58.273864+0000","last_became_peered":"2024-02-26T10:41:58.273864+0000","last_unstale":"2024-02-26T10:42:21.920657+0000","last_undegraded":"2024-02-26T10:42:21.920657+0000","last_fullsized":"2024-02-26T10:42:21.920657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:15:46.825934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919541+0000","last_change":"2024-02-26T10:42:20.601029+0000","last_active":"2024-02-26T10:42:21.919541+0000","last_peered":"2024-02-26T10:42:21.919541+0000","last_clean":"2024-02-26T10:42:21.919541+0000","last_became_active":"2024-02-26T10:42:20.600736+0000","last_became_peered":"2024-02-26T10:42:20.600736+0000","last_unstale":"2024-02-26T10:42:21.919541+0000","last_undegraded":"2024-02-26T10:42:21.919541+0000","last_fullsized":"2024-02-26T10:42:21.919541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:00:26.977878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595424+0000","last_change":"2024-02-26T10:42:19.569242+0000","last_active":"2024-02-26T10:42:21.595424+0000","last_peered":"2024-02-26T10:42:21.595424+0000","last_clean":"2024-02-26T10:42:21.595424+0000","last_became_active":"2024-02-26T10:42:19.568936+0000","last_became_peered":"2024-02-26T10:42:19.568936+0000","last_unstale":"2024-02-26T10:42:21.595424+0000","last_undegraded":"2024-02-26T10:42:21.595424+0000","last_fullsized":"2024-02-26T10:42:21.595424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:31.472935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731385+0000","last_change":"2024-02-26T10:41:59.292484+0000","last_active":"2024-02-26T10:42:21.731385+0000","last_peered":"2024-02-26T10:42:21.731385+0000","last_clean":"2024-02-26T10:42:21.731385+0000","last_became_active":"2024-02-26T10:41:59.292140+0000","last_became_peered":"2024-02-26T10:41:59.292140+0000","last_unstale":"2024-02-26T10:42:21.731385+0000","last_undegraded":"2024-02-26T10:42:21.731385+0000","last_fullsized":"2024-02-26T10:42:21.731385+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:43:48.659155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607911+0000","last_change":"2024-02-26T10:41:58.269128+0000","last_active":"2024-02-26T10:42:21.607911+0000","last_peered":"2024-02-26T10:42:21.607911+0000","last_clean":"2024-02-26T10:42:21.607911+0000","last_became_active":"2024-02-26T10:41:58.268746+0000","last_became_peered":"2024-02-26T10:41:58.268746+0000","last_unstale":"2024-02-26T10:42:21.607911+0000","last_undegraded":"2024-02-26T10:42:21.607911+0000","last_fullsized":"2024-02-26T10:42:21.607911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T10:46:53.257949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606879+0000","last_change":"2024-02-26T10:42:20.591666+0000","last_active":"2024-02-26T10:42:21.606879+0000","last_peered":"2024-02-26T10:42:21.606879+0000","last_clean":"2024-02-26T10:42:21.606879+0000","last_became_active":"2024-02-26T10:42:20.590442+0000","last_became_peered":"2024-02-26T10:42:20.590442+0000","last_unstale":"2024-02-26T10:42:21.606879+0000","last_undegraded":"2024-02-26T10:42:21.606879+0000","last_fullsized":"2024-02-26T10:42:21.606879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:17:26.997989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596426+0000","last_change":"2024-02-26T10:42:19.567393+0000","last_active":"2024-02-26T10:42:21.596426+0000","last_peered":"2024-02-26T10:42:21.596426+0000","last_clean":"2024-02-26T10:42:21.596426+0000","last_became_active":"2024-02-26T10:42:19.567099+0000","last_became_peered":"2024-02-26T10:42:19.567099+0000","last_unstale":"2024-02-26T10:42:21.596426+0000","last_undegraded":"2024-02-26T10:42:21.596426+0000","last_fullsized":"2024-02-26T10:42:21.596426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:37:57.688332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606757+0000","last_change":"2024-02-26T10:41:59.292780+0000","last_active":"2024-02-26T10:42:21.606757+0000","last_peered":"2024-02-26T10:42:21.606757+0000","last_clean":"2024-02-26T10:42:21.606757+0000","last_became_active":"2024-02-26T10:41:59.292526+0000","last_became_peered":"2024-02-26T10:41:59.292526+0000","last_unstale":"2024-02-26T10:42:21.606757+0000","last_undegraded":"2024-02-26T10:42:21.606757+0000","last_fullsized":"2024-02-26T10:42:21.606757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:45:12.549758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606015+0000","last_change":"2024-02-26T10:41:58.269538+0000","last_active":"2024-02-26T10:42:21.606015+0000","last_peered":"2024-02-26T10:42:21.606015+0000","last_clean":"2024-02-26T10:42:21.606015+0000","last_became_active":"2024-02-26T10:41:58.269200+0000","last_became_peered":"2024-02-26T10:41:58.269200+0000","last_unstale":"2024-02-26T10:42:21.606015+0000","last_undegraded":"2024-02-26T10:42:21.606015+0000","last_fullsized":"2024-02-26T10:42:21.606015+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:57:27.687984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606853+0000","last_change":"2024-02-26T10:42:20.615371+0000","last_active":"2024-02-26T10:42:21.606853+0000","last_peered":"2024-02-26T10:42:21.606853+0000","last_clean":"2024-02-26T10:42:21.606853+0000","last_became_active":"2024-02-26T10:42:20.615041+0000","last_became_peered":"2024-02-26T10:42:20.615041+0000","last_unstale":"2024-02-26T10:42:21.606853+0000","last_undegraded":"2024-02-26T10:42:21.606853+0000","last_fullsized":"2024-02-26T10:42:21.606853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T22:18:45.636096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731579+0000","last_change":"2024-02-26T10:42:19.553615+0000","last_active":"2024-02-26T10:42:21.731579+0000","last_peered":"2024-02-26T10:42:21.731579+0000","last_clean":"2024-02-26T10:42:21.731579+0000","last_became_active":"2024-02-26T10:42:19.553134+0000","last_became_peered":"2024-02-26T10:42:19.553134+0000","last_unstale":"2024-02-26T10:42:21.731579+0000","last_undegraded":"2024-02-26T10:42:21.731579+0000","last_fullsized":"2024-02-26T10:42:21.731579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:43:20.894563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732198+0000","last_change":"2024-02-26T10:41:59.305037+0000","last_active":"2024-02-26T10:42:21.732198+0000","last_peered":"2024-02-26T10:42:21.732198+0000","last_clean":"2024-02-26T10:42:21.732198+0000","last_became_active":"2024-02-26T10:41:59.304739+0000","last_became_peered":"2024-02-26T10:41:59.304739+0000","last_unstale":"2024-02-26T10:42:21.732198+0000","last_undegraded":"2024-02-26T10:42:21.732198+0000","last_fullsized":"2024-02-26T10:42:21.732198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:53.687067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919349+0000","last_change":"2024-02-26T10:41:58.270104+0000","last_active":"2024-02-26T10:42:21.919349+0000","last_peered":"2024-02-26T10:42:21.919349+0000","last_clean":"2024-02-26T10:42:21.919349+0000","last_became_active":"2024-02-26T10:41:58.269672+0000","last_became_peered":"2024-02-26T10:41:58.269672+0000","last_unstale":"2024-02-26T10:42:21.919349+0000","last_undegraded":"2024-02-26T10:42:21.919349+0000","last_fullsized":"2024-02-26T10:42:21.919349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:26.696259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594206+0000","last_change":"2024-02-26T10:42:20.601613+0000","last_active":"2024-02-26T10:42:21.594206+0000","last_peered":"2024-02-26T10:42:21.594206+0000","last_clean":"2024-02-26T10:42:21.594206+0000","last_became_active":"2024-02-26T10:42:20.600643+0000","last_became_peered":"2024-02-26T10:42:20.600643+0000","last_unstale":"2024-02-26T10:42:21.594206+0000","last_undegraded":"2024-02-26T10:42:21.594206+0000","last_fullsized":"2024-02-26T10:42:21.594206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:36:44.707485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596894+0000","last_change":"2024-02-26T10:42:19.570519+0000","last_active":"2024-02-26T10:42:21.596894+0000","last_peered":"2024-02-26T10:42:21.596894+0000","last_clean":"2024-02-26T10:42:21.596894+0000","last_became_active":"2024-02-26T10:42:19.569990+0000","last_became_peered":"2024-02-26T10:42:19.569990+0000","last_unstale":"2024-02-26T10:42:21.596894+0000","last_undegraded":"2024-02-26T10:42:21.596894+0000","last_fullsized":"2024-02-26T10:42:21.596894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:28:55.454669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599628+0000","last_change":"2024-02-26T10:41:59.289483+0000","last_active":"2024-02-26T10:42:21.599628+0000","last_peered":"2024-02-26T10:42:21.599628+0000","last_clean":"2024-02-26T10:42:21.599628+0000","last_became_active":"2024-02-26T10:41:59.288870+0000","last_became_peered":"2024-02-26T10:41:59.288870+0000","last_unstale":"2024-02-26T10:42:21.599628+0000","last_undegraded":"2024-02-26T10:42:21.599628+0000","last_fullsized":"2024-02-26T10:42:21.599628+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:23:02.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"75'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594381+0000","last_change":"2024-02-26T10:41:58.271966+0000","last_active":"2024-02-26T10:42:21.594381+0000","last_peered":"2024-02-26T10:42:21.594381+0000","last_clean":"2024-02-26T10:42:21.594381+0000","last_became_active":"2024-02-26T10:41:58.271666+0000","last_became_peered":"2024-02-26T10:41:58.271666+0000","last_unstale":"2024-02-26T10:42:21.594381+0000","last_undegraded":"2024-02-26T10:42:21.594381+0000","last_fullsized":"2024-02-26T10:42:21.594381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T16:11:38.765907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921818+0000","last_change":"2024-02-26T10:42:20.591827+0000","last_active":"2024-02-26T10:42:21.921818+0000","last_peered":"2024-02-26T10:42:21.921818+0000","last_clean":"2024-02-26T10:42:21.921818+0000","last_became_active":"2024-02-26T10:42:20.591457+0000","last_became_peered":"2024-02-26T10:42:20.591457+0000","last_unstale":"2024-02-26T10:42:21.921818+0000","last_undegraded":"2024-02-26T10:42:21.921818+0000","last_fullsized":"2024-02-26T10:42:21.921818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:43:19.105024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921931+0000","last_change":"2024-02-26T10:42:19.567437+0000","last_active":"2024-02-26T10:42:21.921931+0000","last_peered":"2024-02-26T10:42:21.921931+0000","last_clean":"2024-02-26T10:42:21.921931+0000","last_became_active":"2024-02-26T10:42:19.567127+0000","last_became_peered":"2024-02-26T10:42:19.567127+0000","last_unstale":"2024-02-26T10:42:21.921931+0000","last_undegraded":"2024-02-26T10:42:21.921931+0000","last_fullsized":"2024-02-26T10:42:21.921931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T19:58:03.477315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921441+0000","last_change":"2024-02-26T10:41:59.294199+0000","last_active":"2024-02-26T10:42:21.921441+0000","last_peered":"2024-02-26T10:42:21.921441+0000","last_clean":"2024-02-26T10:42:21.921441+0000","last_became_active":"2024-02-26T10:41:59.293906+0000","last_became_peered":"2024-02-26T10:41:59.293906+0000","last_unstale":"2024-02-26T10:42:21.921441+0000","last_undegraded":"2024-02-26T10:42:21.921441+0000","last_fullsized":"2024-02-26T10:42:21.921441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:16:27.711178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"76'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594512+0000","last_change":"2024-02-26T10:41:58.248049+0000","last_active":"2024-02-26T10:42:21.594512+0000","last_peered":"2024-02-26T10:42:21.594512+0000","last_clean":"2024-02-26T10:42:21.594512+0000","last_became_active":"2024-02-26T10:41:58.247173+0000","last_became_peered":"2024-02-26T10:41:58.247173+0000","last_unstale":"2024-02-26T10:42:21.594512+0000","last_undegraded":"2024-02-26T10:42:21.594512+0000","last_fullsized":"2024-02-26T10:42:21.594512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:02:34.734605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597341+0000","last_change":"2024-02-26T10:42:20.609982+0000","last_active":"2024-02-26T10:42:21.597341+0000","last_peered":"2024-02-26T10:42:21.597341+0000","last_clean":"2024-02-26T10:42:21.597341+0000","last_became_active":"2024-02-26T10:42:20.609652+0000","last_became_peered":"2024-02-26T10:42:20.609652+0000","last_unstale":"2024-02-26T10:42:21.597341+0000","last_undegraded":"2024-02-26T10:42:21.597341+0000","last_fullsized":"2024-02-26T10:42:21.597341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:14:11.763202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.732471+0000","last_change":"2024-02-26T10:42:19.558924+0000","last_active":"2024-02-26T10:42:21.732471+0000","last_peered":"2024-02-26T10:42:21.732471+0000","last_clean":"2024-02-26T10:42:21.732471+0000","last_became_active":"2024-02-26T10:42:19.558600+0000","last_became_peered":"2024-02-26T10:42:19.558600+0000","last_unstale":"2024-02-26T10:42:21.732471+0000","last_undegraded":"2024-02-26T10:42:21.732471+0000","last_fullsized":"2024-02-26T10:42:21.732471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:53.055720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922828+0000","last_change":"2024-02-26T10:41:59.298323+0000","last_active":"2024-02-26T10:42:21.922828+0000","last_peered":"2024-02-26T10:42:21.922828+0000","last_clean":"2024-02-26T10:42:21.922828+0000","last_became_active":"2024-02-26T10:41:59.295932+0000","last_became_peered":"2024-02-26T10:41:59.295932+0000","last_unstale":"2024-02-26T10:42:21.922828+0000","last_undegraded":"2024-02-26T10:42:21.922828+0000","last_fullsized":"2024-02-26T10:42:21.922828+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922270+0000","last_change":"2024-02-26T10:41:58.273887+0000","last_active":"2024-02-26T10:42:21.922270+0000","last_peered":"2024-02-26T10:42:21.922270+0000","last_clean":"2024-02-26T10:42:21.922270+0000","last_became_active":"2024-02-26T10:41:58.271683+0000","last_became_peered":"2024-02-26T10:41:58.271683+0000","last_unstale":"2024-02-26T10:42:21.922270+0000","last_undegraded":"2024-02-26T10:42:21.922270+0000","last_fullsized":"2024-02-26T10:42:21.922270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:45:08.382415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.593931+0000","last_change":"2024-02-26T10:42:20.597829+0000","last_active":"2024-02-26T10:42:21.593931+0000","last_peered":"2024-02-26T10:42:21.593931+0000","last_clean":"2024-02-26T10:42:21.593931+0000","last_became_active":"2024-02-26T10:42:20.597548+0000","last_became_peered":"2024-02-26T10:42:20.597548+0000","last_unstale":"2024-02-26T10:42:21.593931+0000","last_undegraded":"2024-02-26T10:42:21.593931+0000","last_fullsized":"2024-02-26T10:42:21.593931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:16:09.136348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.604651+0000","last_change":"2024-02-26T10:42:19.568185+0000","last_active":"2024-02-26T10:42:21.604651+0000","last_peered":"2024-02-26T10:42:21.604651+0000","last_clean":"2024-02-26T10:42:21.604651+0000","last_became_active":"2024-02-26T10:42:19.567744+0000","last_became_peered":"2024-02-26T10:42:19.567744+0000","last_unstale":"2024-02-26T10:42:21.604651+0000","last_undegraded":"2024-02-26T10:42:21.604651+0000","last_fullsized":"2024-02-26T10:42:21.604651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:03:24.757719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.599796+0000","last_change":"2024-02-26T10:41:59.293572+0000","last_active":"2024-02-26T10:42:21.599796+0000","last_peered":"2024-02-26T10:42:21.599796+0000","last_clean":"2024-02-26T10:42:21.599796+0000","last_became_active":"2024-02-26T10:41:59.292292+0000","last_became_peered":"2024-02-26T10:41:59.292292+0000","last_unstale":"2024-02-26T10:42:21.599796+0000","last_undegraded":"2024-02-26T10:42:21.599796+0000","last_fullsized":"2024-02-26T10:42:21.599796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:26.922522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.922976+0000","last_change":"2024-02-26T10:41:58.264305+0000","last_active":"2024-02-26T10:42:21.922976+0000","last_peered":"2024-02-26T10:42:21.922976+0000","last_clean":"2024-02-26T10:42:21.922976+0000","last_became_active":"2024-02-26T10:41:58.264021+0000","last_became_peered":"2024-02-26T10:41:58.264021+0000","last_unstale":"2024-02-26T10:42:21.922976+0000","last_undegraded":"2024-02-26T10:42:21.922976+0000","last_fullsized":"2024-02-26T10:42:21.922976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:14.749617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606155+0000","last_change":"2024-02-26T10:42:20.606350+0000","last_active":"2024-02-26T10:42:21.606155+0000","last_peered":"2024-02-26T10:42:21.606155+0000","last_clean":"2024-02-26T10:42:21.606155+0000","last_became_active":"2024-02-26T10:42:20.605955+0000","last_became_peered":"2024-02-26T10:42:20.605955+0000","last_unstale":"2024-02-26T10:42:21.606155+0000","last_undegraded":"2024-02-26T10:42:21.606155+0000","last_fullsized":"2024-02-26T10:42:21.606155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:34:18.543420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919265+0000","last_change":"2024-02-26T10:42:19.551290+0000","last_active":"2024-02-26T10:42:21.919265+0000","last_peered":"2024-02-26T10:42:21.919265+0000","last_clean":"2024-02-26T10:42:21.919265+0000","last_became_active":"2024-02-26T10:42:19.550642+0000","last_became_peered":"2024-02-26T10:42:19.550642+0000","last_unstale":"2024-02-26T10:42:21.919265+0000","last_undegraded":"2024-02-26T10:42:21.919265+0000","last_fullsized":"2024-02-26T10:42:21.919265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:22:49.405962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.731999+0000","last_change":"2024-02-26T10:41:59.296354+0000","last_active":"2024-02-26T10:42:21.731999+0000","last_peered":"2024-02-26T10:42:21.731999+0000","last_clean":"2024-02-26T10:42:21.731999+0000","last_became_active":"2024-02-26T10:41:59.296012+0000","last_became_peered":"2024-02-26T10:41:59.296012+0000","last_unstale":"2024-02-26T10:42:21.731999+0000","last_undegraded":"2024-02-26T10:42:21.731999+0000","last_fullsized":"2024-02-26T10:42:21.731999+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:23:28.972598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596586+0000","last_change":"2024-02-26T10:41:58.270676+0000","last_active":"2024-02-26T10:42:21.596586+0000","last_peered":"2024-02-26T10:42:21.596586+0000","last_clean":"2024-02-26T10:42:21.596586+0000","last_became_active":"2024-02-26T10:41:58.270241+0000","last_became_peered":"2024-02-26T10:41:58.270241+0000","last_unstale":"2024-02-26T10:42:21.596586+0000","last_undegraded":"2024-02-26T10:42:21.596586+0000","last_fullsized":"2024-02-26T10:42:21.596586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:50:59.443811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919511+0000","last_change":"2024-02-26T10:42:20.595281+0000","last_active":"2024-02-26T10:42:21.919511+0000","last_peered":"2024-02-26T10:42:21.919511+0000","last_clean":"2024-02-26T10:42:21.919511+0000","last_became_active":"2024-02-26T10:42:20.593810+0000","last_became_peered":"2024-02-26T10:42:20.593810+0000","last_unstale":"2024-02-26T10:42:21.919511+0000","last_undegraded":"2024-02-26T10:42:21.919511+0000","last_fullsized":"2024-02-26T10:42:21.919511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:05:16.206266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.606336+0000","last_change":"2024-02-26T10:42:19.564800+0000","last_active":"2024-02-26T10:42:21.606336+0000","last_peered":"2024-02-26T10:42:21.606336+0000","last_clean":"2024-02-26T10:42:21.606336+0000","last_became_active":"2024-02-26T10:42:19.564472+0000","last_became_peered":"2024-02-26T10:42:19.564472+0000","last_unstale":"2024-02-26T10:42:21.606336+0000","last_undegraded":"2024-02-26T10:42:21.606336+0000","last_fullsized":"2024-02-26T10:42:21.606336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:19:58.917162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.596513+0000","last_change":"2024-02-26T10:41:59.297360+0000","last_active":"2024-02-26T10:42:21.596513+0000","last_peered":"2024-02-26T10:42:21.596513+0000","last_clean":"2024-02-26T10:42:21.596513+0000","last_became_active":"2024-02-26T10:41:59.297031+0000","last_became_peered":"2024-02-26T10:41:59.297031+0000","last_unstale":"2024-02-26T10:42:21.596513+0000","last_undegraded":"2024-02-26T10:42:21.596513+0000","last_fullsized":"2024-02-26T10:42:21.596513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598011+0000","last_change":"2024-02-26T10:41:58.267445+0000","last_active":"2024-02-26T10:42:21.598011+0000","last_peered":"2024-02-26T10:42:21.598011+0000","last_clean":"2024-02-26T10:42:21.598011+0000","last_became_active":"2024-02-26T10:41:58.264518+0000","last_became_peered":"2024-02-26T10:41:58.264518+0000","last_unstale":"2024-02-26T10:42:21.598011+0000","last_undegraded":"2024-02-26T10:42:21.598011+0000","last_fullsized":"2024-02-26T10:42:21.598011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:58:22.045852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607530+0000","last_change":"2024-02-26T10:42:20.606543+0000","last_active":"2024-02-26T10:42:21.607530+0000","last_peered":"2024-02-26T10:42:21.607530+0000","last_clean":"2024-02-26T10:42:21.607530+0000","last_became_active":"2024-02-26T10:42:20.606202+0000","last_became_peered":"2024-02-26T10:42:20.606202+0000","last_unstale":"2024-02-26T10:42:21.607530+0000","last_undegraded":"2024-02-26T10:42:21.607530+0000","last_fullsized":"2024-02-26T10:42:21.607530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:12:14.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.595742+0000","last_change":"2024-02-26T10:42:19.564857+0000","last_active":"2024-02-26T10:42:21.595742+0000","last_peered":"2024-02-26T10:42:21.595742+0000","last_clean":"2024-02-26T10:42:21.595742+0000","last_became_active":"2024-02-26T10:42:19.564550+0000","last_became_peered":"2024-02-26T10:42:19.564550+0000","last_unstale":"2024-02-26T10:42:21.595742+0000","last_undegraded":"2024-02-26T10:42:21.595742+0000","last_fullsized":"2024-02-26T10:42:21.595742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:06:15.730553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.918939+0000","last_change":"2024-02-26T10:41:59.295832+0000","last_active":"2024-02-26T10:42:21.918939+0000","last_peered":"2024-02-26T10:42:21.918939+0000","last_clean":"2024-02-26T10:42:21.918939+0000","last_became_active":"2024-02-26T10:41:59.295548+0000","last_became_peered":"2024-02-26T10:41:59.295548+0000","last_unstale":"2024-02-26T10:42:21.918939+0000","last_undegraded":"2024-02-26T10:42:21.918939+0000","last_fullsized":"2024-02-26T10:42:21.918939+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T20:20:01.123826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920777+0000","last_change":"2024-02-26T10:41:58.259325+0000","last_active":"2024-02-26T10:42:21.920777+0000","last_peered":"2024-02-26T10:42:21.920777+0000","last_clean":"2024-02-26T10:42:21.920777+0000","last_became_active":"2024-02-26T10:41:58.258993+0000","last_became_peered":"2024-02-26T10:41:58.258993+0000","last_unstale":"2024-02-26T10:42:21.920777+0000","last_undegraded":"2024-02-26T10:42:21.920777+0000","last_fullsized":"2024-02-26T10:42:21.920777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T21:29:23.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.598541+0000","last_change":"2024-02-26T10:42:20.598744+0000","last_active":"2024-02-26T10:42:21.598541+0000","last_peered":"2024-02-26T10:42:21.598541+0000","last_clean":"2024-02-26T10:42:21.598541+0000","last_became_active":"2024-02-26T10:42:20.598453+0000","last_became_peered":"2024-02-26T10:42:20.598453+0000","last_unstale":"2024-02-26T10:42:21.598541+0000","last_undegraded":"2024-02-26T10:42:21.598541+0000","last_fullsized":"2024-02-26T10:42:21.598541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T13:56:54.070139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919646+0000","last_change":"2024-02-26T10:42:19.567123+0000","last_active":"2024-02-26T10:42:21.919646+0000","last_peered":"2024-02-26T10:42:21.919646+0000","last_clean":"2024-02-26T10:42:21.919646+0000","last_became_active":"2024-02-26T10:42:19.566824+0000","last_became_peered":"2024-02-26T10:42:19.566824+0000","last_unstale":"2024-02-26T10:42:21.919646+0000","last_undegraded":"2024-02-26T10:42:21.919646+0000","last_fullsized":"2024-02-26T10:42:21.919646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:25:21.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.920523+0000","last_change":"2024-02-26T10:41:59.300681+0000","last_active":"2024-02-26T10:42:21.920523+0000","last_peered":"2024-02-26T10:42:21.920523+0000","last_clean":"2024-02-26T10:42:21.920523+0000","last_became_active":"2024-02-26T10:41:59.299849+0000","last_became_peered":"2024-02-26T10:41:59.299849+0000","last_unstale":"2024-02-26T10:42:21.920523+0000","last_undegraded":"2024-02-26T10:42:21.920523+0000","last_fullsized":"2024-02-26T10:42:21.920523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T17:51:49.139101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.597947+0000","last_change":"2024-02-26T10:41:58.275362+0000","last_active":"2024-02-26T10:42:21.597947+0000","last_peered":"2024-02-26T10:42:21.597947+0000","last_clean":"2024-02-26T10:42:21.597947+0000","last_became_active":"2024-02-26T10:41:58.275070+0000","last_became_peered":"2024-02-26T10:41:58.275070+0000","last_unstale":"2024-02-26T10:42:21.597947+0000","last_undegraded":"2024-02-26T10:42:21.597947+0000","last_fullsized":"2024-02-26T10:42:21.597947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T15:40:26.347537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.607430+0000","last_change":"2024-02-26T10:42:20.594531+0000","last_active":"2024-02-26T10:42:21.607430+0000","last_peered":"2024-02-26T10:42:21.607430+0000","last_clean":"2024-02-26T10:42:21.607430+0000","last_became_active":"2024-02-26T10:42:20.594237+0000","last_became_peered":"2024-02-26T10:42:20.594237+0000","last_unstale":"2024-02-26T10:42:21.607430+0000","last_undegraded":"2024-02-26T10:42:21.607430+0000","last_fullsized":"2024-02-26T10:42:21.607430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:19.520542+0000","last_clean_scrub_stamp":"2024-02-26T10:42:19.520542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:00:51.720533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921150+0000","last_change":"2024-02-26T10:42:19.574963+0000","last_active":"2024-02-26T10:42:21.921150+0000","last_peered":"2024-02-26T10:42:21.921150+0000","last_clean":"2024-02-26T10:42:21.921150+0000","last_became_active":"2024-02-26T10:42:19.574602+0000","last_became_peered":"2024-02-26T10:42:19.574602+0000","last_unstale":"2024-02-26T10:42:21.921150+0000","last_undegraded":"2024-02-26T10:42:21.921150+0000","last_fullsized":"2024-02-26T10:42:21.921150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:42:18.506123+0000","last_clean_scrub_stamp":"2024-02-26T10:42:18.506123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:02:31.472935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.594291+0000","last_change":"2024-02-26T10:41:59.295485+0000","last_active":"2024-02-26T10:42:21.594291+0000","last_peered":"2024-02-26T10:42:21.594291+0000","last_clean":"2024-02-26T10:42:21.594291+0000","last_became_active":"2024-02-26T10:41:59.295154+0000","last_became_peered":"2024-02-26T10:41:59.295154+0000","last_unstale":"2024-02-26T10:42:21.594291+0000","last_undegraded":"2024-02-26T10:42:21.594291+0000","last_fullsized":"2024-02-26T10:42:21.594291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:58.225047+0000","last_clean_scrub_stamp":"2024-02-26T10:41:58.225047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:20:15.764536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"75'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.921353+0000","last_change":"2024-02-26T10:41:58.272053+0000","last_active":"2024-02-26T10:42:21.921353+0000","last_peered":"2024-02-26T10:42:21.921353+0000","last_clean":"2024-02-26T10:42:21.921353+0000","last_became_active":"2024-02-26T10:41:58.271719+0000","last_became_peered":"2024-02-26T10:41:58.271719+0000","last_unstale":"2024-02-26T10:42:21.921353+0000","last_undegraded":"2024-02-26T10:42:21.921353+0000","last_fullsized":"2024-02-26T10:42:21.921353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:41:57.210128+0000","last_clean_scrub_stamp":"2024-02-26T10:41:57.210128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T18:54:25.907603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-02-26T10:42:21.919456+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:42:21.919456+0000","last_peered":"2024-02-26T10:42:21.919456+0000","last_clean":"2024-02-26T10:42:21.919456+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:42:21.919456+0000","last_undegraded":"2024-02-26T10:42:21.919456+0000","last_fullsized":"2024-02-26T10:42:21.919456+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":3913,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542194,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1585152,"data_stored":1033134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":443930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":445791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542195,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":444320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":447333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":443200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":991232,"data_stored":442778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1581056,"data_stored":1033568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:42:25.634 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-26T10:42:25.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-02-26T10:42:25.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.910+0000 7ffb21400640 1 Processor -- start 2024-02-26T10:42:25.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.911+0000 7ffb21400640 1 -- start start 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.912+0000 7ffb21400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb1c155c50 0x7ffb1c072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.912+0000 7ffb21400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c0c2960 con 0x7ffb1c1592b0 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.912+0000 7ffb21400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c1587d0 con 0x7ffb1c14f160 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.912+0000 7ffb21400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1c158e20 con 0x7ffb1c155c50 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb19e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffb1c1592b0 0x7ffb1c0c58a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47036/0 (socket says 172.21.15.67:47036) 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb19e00640 1 -- 172.21.15.67:0/3932552621 learned_addr learned my addr 172.21.15.67:0/3932552621 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:25.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb1b200640 1 --2- 172.21.15.67:0/3932552621 >> v2:172.21.15.67:3300/0 conn(0x7ffb1c155c50 0x7ffb1c072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb19400640 1 -- 172.21.15.67:0/3932552621 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095945841 0 0) 0x7ffb1c0c2960 con 0x7ffb1c1592b0 2024-02-26T10:42:25.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb19400640 1 -- 172.21.15.67:0/3932552621 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb00003520 con 0x7ffb1c1592b0 2024-02-26T10:42:25.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb1b200640 1 -- 172.21.15.67:0/3932552621 >> v1:172.21.15.82:6789/0 conn(0x7ffb1c14f160 legacy=0x7ffb1c0c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:25.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb1b200640 1 -- 172.21.15.67:0/3932552621 >> v1:172.21.15.67:6789/0 conn(0x7ffb1c1592b0 legacy=0x7ffb1c0c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.913+0000 7ffb1b200640 1 -- 172.21.15.67:0/3932552621 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c158ab0 con 0x7ffb1c155c50 2024-02-26T10:42:25.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.914+0000 7ffb1b200640 1 --2- 172.21.15.67:0/3932552621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c155c50 0x7ffb1c072060 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097d0 tx=0x7ffb1402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d5022e3bb3d6b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb19400640 1 -- 172.21.15.67:0/3932552621 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14035020 con 0x7ffb1c155c50 2024-02-26T10:42:25.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb19400640 1 -- 172.21.15.67:0/3932552621 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14031810 con 0x7ffb1c155c50 2024-02-26T10:42:25.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb19400640 1 -- 172.21.15.67:0/3932552621 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14031be0 con 0x7ffb1c155c50 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb21400640 1 -- 172.21.15.67:0/3932552621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c155c50 msgr2=0x7ffb1c072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb21400640 1 --2- 172.21.15.67:0/3932552621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c155c50 0x7ffb1c072060 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097d0 tx=0x7ffb1402b8b0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb21400640 1 -- 172.21.15.67:0/3932552621 shutdown_connections 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb21400640 1 --2- 172.21.15.67:0/3932552621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c155c50 0x7ffb1c072060 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.915+0000 7ffb21400640 1 -- 172.21.15.67:0/3932552621 >> 172.21.15.67:0/3932552621 conn(0x7ffb1c0c7e40 msgr2=0x7ffb1c0a9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.916+0000 7ffb21400640 1 -- 172.21.15.67:0/3932552621 shutdown_connections 2024-02-26T10:42:25.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.916+0000 7ffb21400640 1 -- 172.21.15.67:0/3932552621 wait complete. 2024-02-26T10:42:25.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.917+0000 7ffb21400640 1 Processor -- start 2024-02-26T10:42:25.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.917+0000 7ffb21400640 1 -- start start 2024-02-26T10:42:25.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.918+0000 7ffb21400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb1c14f160 0x7ffb1c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.918+0000 7ffb21400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1c155c50 0x7ffb1c147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.918+0000 7ffb21400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 0x7ffb1c148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.918+0000 7ffb21400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0725a0 con 0x7ffb1c1592b0 2024-02-26T10:42:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.918+0000 7ffb21400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c14f540 con 0x7ffb1c155c50 2024-02-26T10:42:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.918+0000 7ffb21400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c072720 con 0x7ffb1c14f160 2024-02-26T10:42:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 0x7ffb1c148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 0x7ffb1c148030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50640/0 (socket says 172.21.15.67:50640) 2024-02-26T10:42:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 -- 172.21.15.67:0/2469482157 learned_addr learned my addr 172.21.15.67:0/2469482157 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:25.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb19e00640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1c155c50 0x7ffb1c147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1a800640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb1c14f160 0x7ffb1c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 -- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb1c14f160 msgr2=0x7ffb1c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb1c14f160 0x7ffb1c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 -- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1c155c50 msgr2=0x7ffb1c147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1c155c50 0x7ffb1c147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 -- 172.21.15.67:0/2469482157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c14e940 con 0x7ffb1c1592b0 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb19e00640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1c155c50 0x7ffb1c147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1a800640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb1c14f160 0x7ffb1c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.919+0000 7ffb1b200640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 0x7ffb1c148030 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097d0 tx=0x7ffb140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.920+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14041070 con 0x7ffb1c1592b0 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.920+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1403ac80 con 0x7ffb1c1592b0 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.920+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14035040 con 0x7ffb1c1592b0 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.920+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c148570 con 0x7ffb1c1592b0 2024-02-26T10:42:25.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.920+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c158070 con 0x7ffb1c1592b0 2024-02-26T10:42:25.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.921+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1c074a30 con 0x7ffb1c1592b0 2024-02-26T10:42:25.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.922+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb14038020 con 0x7ffb1c1592b0 2024-02-26T10:42:25.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.923+0000 7ffb12a00640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffaf806b4d0 0x7ffaf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:25.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.923+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ffb140ac120 con 0x7ffb1c1592b0 2024-02-26T10:42:25.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.924+0000 7ffb1a800640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffaf806b4d0 0x7ffaf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:25.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.925+0000 7ffb1a800640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffaf806b4d0 0x7ffaf806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c004520 tx=0x7ffb0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:25.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:25.927+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb1407ac30 con 0x7ffb1c1592b0 2024-02-26T10:42:26.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.195+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7ffb1c073050 con 0x7ffb1c1592b0 2024-02-26T10:42:26.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.200+0000 7ffb12a00640 1 -- 172.21.15.67:0/2469482157 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v18) v1 ==== 185+0+142 (secure 0 0 0) 0x7ffb1407a530 con 0x7ffb1c1592b0 2024-02-26T10:42:26.198 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-02-26T10:42:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQDvadxlZ0bhAxAAj06bCAj8EUvWZoy1CS8Bkw== 2024-02-26T10:42:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow" 2024-02-26T10:42:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:42:26.198 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow rw pool=cephfs2_data" 2024-02-26T10:42:26.199 INFO:teuthology.orchestra.run.smithi067.stderr:updated caps for client.1 2024-02-26T10:42:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffaf806b4d0 msgr2=0x7ffaf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffaf806b4d0 0x7ffaf806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c004520 tx=0x7ffb0c009040 comp rx=0 tx=0).stop 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 msgr2=0x7ffb1c148030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 0x7ffb1c148030 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ffb140097d0 tx=0x7ffb140046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 shutdown_connections 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb1c1592b0 0x7ffb1c148030 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ffaf806b4d0 0x7ffaf806d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1c155c50 0x7ffb1c147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 --2- 172.21.15.67:0/2469482157 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb1c14f160 0x7ffb1c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 >> 172.21.15.67:0/2469482157 conn(0x7ffb1c0c7e40 msgr2=0x7ffb1c0ac620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 shutdown_connections 2024-02-26T10:42:26.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.203+0000 7ffb21400640 1 -- 172.21.15.67:0/2469482157 wait complete. 2024-02-26T10:42:26.220 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:26.220 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:26.278 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:26.278 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:26.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:26.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.557+0000 7f7b56e00640 1 Processor -- start 2024-02-26T10:42:26.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.557+0000 7f7b56e00640 1 -- start start 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.558+0000 7f7b56e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b5006ac20 0x7f7b5006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.558+0000 7f7b56e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b50153630 0x7f7b501524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.558+0000 7f7b56e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b50155f10 con 0x7f7b500c2310 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b50153a60 con 0x7f7b50153630 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b500c28d0 con 0x7f7b5006ac20 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b55000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7b500c2310 0x7f7b50152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47048/0 (socket says 172.21.15.67:47048) 2024-02-26T10:42:26.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b55000640 1 -- 172.21.15.67:0/2156034887 learned_addr learned my addr 172.21.15.67:0/2156034887 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56400640 1 --2- 172.21.15.67:0/2156034887 >> v2:172.21.15.67:3300/0 conn(0x7f7b50153630 0x7f7b501524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b55a00640 1 --2- 172.21.15.67:0/2156034887 >> v2:172.21.15.82:3300/0 conn(0x7f7b5006ac20 0x7f7b5006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56400640 1 -- 172.21.15.67:0/2156034887 >> v2:172.21.15.82:3300/0 conn(0x7f7b5006ac20 msgr2=0x7f7b5006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56400640 1 --2- 172.21.15.67:0/2156034887 >> v2:172.21.15.82:3300/0 conn(0x7f7b5006ac20 0x7f7b5006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56400640 1 -- 172.21.15.67:0/2156034887 >> v1:172.21.15.67:6789/0 conn(0x7f7b500c2310 legacy=0x7f7b50152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.559+0000 7f7b56400640 1 -- 172.21.15.67:0/2156034887 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b50155150 con 0x7f7b50153630 2024-02-26T10:42:26.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.561+0000 7f7b56400640 1 --2- 172.21.15.67:0/2156034887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b501524b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006ee00 tx=0x7f7b4802bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9ab3e8fac9f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:26.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.561+0000 7f7b4fe00640 1 -- 172.21.15.67:0/2156034887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48039070 con 0x7f7b50153630 2024-02-26T10:42:26.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.561+0000 7f7b4fe00640 1 -- 172.21.15.67:0/2156034887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4802cb50 con 0x7f7b50153630 2024-02-26T10:42:26.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.561+0000 7f7b4fe00640 1 -- 172.21.15.67:0/2156034887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48034440 con 0x7f7b50153630 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.561+0000 7f7b56e00640 1 -- 172.21.15.67:0/2156034887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 msgr2=0x7f7b501524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.561+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2156034887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b501524b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006ee00 tx=0x7f7b4802bda0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.562+0000 7f7b56e00640 1 -- 172.21.15.67:0/2156034887 shutdown_connections 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.562+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2156034887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b501524b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.562+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2156034887 >> v2:172.21.15.82:3300/0 conn(0x7f7b5006ac20 0x7f7b5006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.562+0000 7f7b56e00640 1 -- 172.21.15.67:0/2156034887 >> 172.21.15.67:0/2156034887 conn(0x7f7b500c63f0 msgr2=0x7f7b500c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.562+0000 7f7b56e00640 1 -- 172.21.15.67:0/2156034887 shutdown_connections 2024-02-26T10:42:26.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.562+0000 7f7b56e00640 1 -- 172.21.15.67:0/2156034887 wait complete. 2024-02-26T10:42:26.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.563+0000 7f7b56e00640 1 Processor -- start 2024-02-26T10:42:26.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.563+0000 7f7b56e00640 1 -- start start 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.564+0000 7f7b56e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b5006ac20 0x7f7b500c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.564+0000 7f7b56e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b500c2310 0x7f7b50151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.564+0000 7f7b56e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b500ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.564+0000 7f7b56e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b501559f0 con 0x7f7b50153630 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.564+0000 7f7b56e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50155710 con 0x7f7b500c2310 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.564+0000 7f7b56e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50148850 con 0x7f7b5006ac20 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b500ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b500ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50664/0 (socket says 172.21.15.67:50664) 2024-02-26T10:42:26.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 -- 172.21.15.67:0/2504244683 learned_addr learned my addr 172.21.15.67:0/2504244683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:26.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b55000640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b500c2310 0x7f7b50151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:26.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 -- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b5006ac20 msgr2=0x7f7b500c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b5006ac20 0x7f7b500c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 -- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b500c2310 msgr2=0x7f7b50151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b500c2310 0x7f7b50151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 -- 172.21.15.67:0/2504244683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500c38a0 con 0x7f7b50153630 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.565+0000 7f7b56400640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b500ab7b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7b4802c3b0 tx=0x7f7b48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.566+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48039050 con 0x7f7b50153630 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.566+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b48045070 con 0x7f7b50153630 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.566+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4803e830 con 0x7f7b50153630 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.566+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b500c3b80 con 0x7f7b50153630 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.566+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b500c4140 con 0x7f7b50153630 2024-02-26T10:42:26.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.567+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b500c4340 con 0x7f7b50153630 2024-02-26T10:42:26.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.568+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b4803e9d0 con 0x7f7b50153630 2024-02-26T10:42:26.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.569+0000 7f7b4d600640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7b3406b500 0x7f7b3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:26.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.569+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f7b480acc90 con 0x7f7b50153630 2024-02-26T10:42:26.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.570+0000 7f7b55a00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7b3406b500 0x7f7b3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:26.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.571+0000 7f7b55a00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7b3406b500 0x7f7b3406d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7b44006fd0 tx=0x7f7b44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.574+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b500c4340 con 0x7f7b50153630 2024-02-26T10:42:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.771+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b50074a30 con 0x7f7b50153630 2024-02-26T10:42:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.775+0000 7f7b4d600640 1 -- 172.21.15.67:0/2504244683 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7f7b4804e020 con 0x7f7b50153630 2024-02-26T10:42:26.773 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:26.774 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.777+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7b3406b500 msgr2=0x7f7b3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.777+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7b3406b500 0x7f7b3406d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7b44006fd0 tx=0x7f7b44008040 comp rx=0 tx=0).stop 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.777+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 msgr2=0x7f7b500ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.777+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b500ab7b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7b4802c3b0 tx=0x7f7b48004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 shutdown_connections 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b50153630 0x7f7b500ab7b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7b3406b500 0x7f7b3406d9c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b500c2310 0x7f7b50151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 --2- 172.21.15.67:0/2504244683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b5006ac20 0x7f7b500c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:26.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 >> 172.21.15.67:0/2504244683 conn(0x7f7b500c63f0 msgr2=0x7f7b50145460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 shutdown_connections 2024-02-26T10:42:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:26.778+0000 7f7b56e00640 1 -- 172.21.15.67:0/2504244683 wait complete. 2024-02-26T10:42:26.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:26.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:27.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.004+0000 7f9dc7e00640 1 Processor -- start 2024-02-26T10:42:27.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.004+0000 7f9dc7e00640 1 -- start start 2024-02-26T10:42:27.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.005+0000 7f9dc7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9dc81503a0 0x7f9dc80ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc80a9430 0x7f9dc81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc7e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc8153ae0 con 0x7f9dc806e2d0 2024-02-26T10:42:27.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc7e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc8150780 con 0x7f9dc81503a0 2024-02-26T10:42:27.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80a98c0 con 0x7f9dc80a9430 2024-02-26T10:42:27.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc6a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9dc81503a0 0x7f9dc80ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc80a9430 0x7f9dc81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.006+0000 7f9dc6a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9dc81503a0 0x7f9dc80ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34500/0 (socket says 172.21.15.67:34500) 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.007+0000 7f9dc6a00640 1 -- 172.21.15.67:0/4234135525 learned_addr learned my addr 172.21.15.67:0/4234135525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.007+0000 7f9dc6a00640 1 -- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc80a9430 msgr2=0x7f9dc81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.007+0000 7f9dc6a00640 1 --2- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc80a9430 0x7f9dc81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.007+0000 7f9dc6a00640 1 -- 172.21.15.67:0/4234135525 >> v1:172.21.15.82:6789/0 conn(0x7f9dc806e2d0 legacy=0x7f9dc80c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.007+0000 7f9dc6a00640 1 -- 172.21.15.67:0/4234135525 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80a9ba0 con 0x7f9dc81503a0 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.007+0000 7f9dc6000640 1 --2- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc80a9430 0x7f9dc81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.009+0000 7f9dc6a00640 1 --2- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc81503a0 0x7f9dc80ab480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009e50 tx=0x7f9db802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f58d16b630f895 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.009+0000 7f9dc5600640 1 -- 172.21.15.67:0/4234135525 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db802cb00 con 0x7f9dc81503a0 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.009+0000 7f9dc5600640 1 -- 172.21.15.67:0/4234135525 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db802cca0 con 0x7f9dc81503a0 2024-02-26T10:42:27.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.009+0000 7f9dc5600640 1 -- 172.21.15.67:0/4234135525 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8034610 con 0x7f9dc81503a0 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 -- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc81503a0 msgr2=0x7f9dc80ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc81503a0 0x7f9dc80ab480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009e50 tx=0x7f9db802bf60 comp rx=0 tx=0).stop 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 -- 172.21.15.67:0/4234135525 shutdown_connections 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc80a9430 0x7f9dc81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/4234135525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc81503a0 0x7f9dc80ab480 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 -- 172.21.15.67:0/4234135525 >> 172.21.15.67:0/4234135525 conn(0x7f9dc80c6380 msgr2=0x7f9dc80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 -- 172.21.15.67:0/4234135525 shutdown_connections 2024-02-26T10:42:27.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.010+0000 7f9dc7e00640 1 -- 172.21.15.67:0/4234135525 wait complete. 2024-02-26T10:42:27.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.012+0000 7f9dc7e00640 1 Processor -- start 2024-02-26T10:42:27.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.012+0000 7f9dc7e00640 1 -- start start 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 0x7f9dc8151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc81503a0 0x7f9dc81ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc8147cd0 0x7f9dc81eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc806ddb0 con 0x7f9dc806e2d0 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc806dad0 con 0x7f9dc8147cd0 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80c0f80 con 0x7f9dc81503a0 2024-02-26T10:42:27.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 0x7f9dc8151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 0x7f9dc8151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50666/0 (socket says 172.21.15.67:50666) 2024-02-26T10:42:27.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.013+0000 7f9dc6a00640 1 -- 172.21.15.67:0/1111961240 learned_addr learned my addr 172.21.15.67:0/1111961240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc7400640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc8147cd0 0x7f9dc81eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6000640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc81503a0 0x7f9dc81ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6a00640 1 -- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc81503a0 msgr2=0x7f9dc81ea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6a00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc81503a0 0x7f9dc81ea8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6a00640 1 -- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc8147cd0 msgr2=0x7f9dc81eae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6a00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc8147cd0 0x7f9dc81eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6a00640 1 -- 172.21.15.67:0/1111961240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8148350 con 0x7f9dc806e2d0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.014+0000 7f9dc6a00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 0x7f9dc8151d80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9db8000f80 tx=0x7f9db8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.015+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8043020 con 0x7f9dc806e2d0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.015+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db802ce50 con 0x7f9dc806e2d0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.015+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8009510 con 0x7f9dc806e2d0 2024-02-26T10:42:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.015+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc80c5120 con 0x7f9dc806e2d0 2024-02-26T10:42:27.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.015+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc8152690 con 0x7f9dc806e2d0 2024-02-26T10:42:27.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.016+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc80678e0 con 0x7f9dc806e2d0 2024-02-26T10:42:27.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.017+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9db8046020 con 0x7f9dc806e2d0 2024-02-26T10:42:27.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.018+0000 7f9dbea00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9da406b4d0 0x7f9da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.018+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f9db80acbc0 con 0x7f9dc806e2d0 2024-02-26T10:42:27.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.018+0000 7f9dc6000640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9da406b4d0 0x7f9da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.019+0000 7f9dc6000640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9da406b4d0 0x7f9da406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009560 tx=0x7f9db0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.022+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9db804d030 con 0x7f9dc806e2d0 2024-02-26T10:42:27.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.215+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9dc8074a30 con 0x7f9dc806e2d0 2024-02-26T10:42:27.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.216+0000 7f9dbea00640 1 -- 172.21.15.67:0/1111961240 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9db803c350 con 0x7f9dc806e2d0 2024-02-26T10:42:27.214 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.218+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9da406b4d0 msgr2=0x7f9da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.218+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9da406b4d0 0x7f9da406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009560 tx=0x7f9db0008040 comp rx=0 tx=0).stop 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.218+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 msgr2=0x7f9dc8151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.218+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 0x7f9dc8151d80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9db8000f80 tx=0x7f9db8004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 shutdown_connections 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9da406b4d0 0x7f9da406d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc8147cd0 0x7f9dc81eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9dc81503a0 0x7f9dc81ea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 --2- 172.21.15.67:0/1111961240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dc806e2d0 0x7f9dc8151d80 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 >> 172.21.15.67:0/1111961240 conn(0x7f9dc80c6380 msgr2=0x7f9dc814ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 shutdown_connections 2024-02-26T10:42:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.219+0000 7f9dc7e00640 1 -- 172.21.15.67:0/1111961240 wait complete. 2024-02-26T10:42:27.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.455+0000 7f83c0200640 1 Processor -- start 2024-02-26T10:42:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.455+0000 7f83c0200640 1 -- start start 2024-02-26T10:42:27.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83c0200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83b806e860 0x7f83b80c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83c0200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b8155880 0x7f83b80a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83c0200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b8151d70 con 0x7f83b80a6150 2024-02-26T10:42:27.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83c0200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b8152b10 con 0x7f83b806e860 2024-02-26T10:42:27.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83c0200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b8152680 con 0x7f83b8155880 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83bce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83b806e860 0x7f83b80c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83bce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83b806e860 0x7f83b80c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50678/0 (socket says 172.21.15.67:50678) 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83bce00640 1 -- 172.21.15.67:0/786752313 learned_addr learned my addr 172.21.15.67:0/786752313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83be200640 1 --2- 172.21.15.67:0/786752313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b8155880 0x7f83b80a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83bce00640 1 -- 172.21.15.67:0/786752313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b8155880 msgr2=0x7f83b80a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.457+0000 7f83bce00640 1 --2- 172.21.15.67:0/786752313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b8155880 0x7f83b80a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.458+0000 7f83bce00640 1 -- 172.21.15.67:0/786752313 >> v1:172.21.15.82:6789/0 conn(0x7f83b80a6150 legacy=0x7f83b8155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:27.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.458+0000 7f83bce00640 1 -- 172.21.15.67:0/786752313 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8152030 con 0x7f83b806e860 2024-02-26T10:42:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.459+0000 7f83bce00640 1 --2- 172.21.15.67:0/786752313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b806e860 0x7f83b80c25a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f83a400bb10 tx=0x7f83a402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e910579cbbacd56 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.459+0000 7f83b7e00640 1 -- 172.21.15.67:0/786752313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a403b070 con 0x7f83b806e860 2024-02-26T10:42:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.459+0000 7f83b7e00640 1 -- 172.21.15.67:0/786752313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4036440 con 0x7f83b806e860 2024-02-26T10:42:27.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.459+0000 7f83b7e00640 1 -- 172.21.15.67:0/786752313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4036810 con 0x7f83b806e860 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 -- 172.21.15.67:0/786752313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b806e860 msgr2=0x7f83b80c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 --2- 172.21.15.67:0/786752313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b806e860 0x7f83b80c25a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f83a400bb10 tx=0x7f83a402e150 comp rx=0 tx=0).stop 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 -- 172.21.15.67:0/786752313 shutdown_connections 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 --2- 172.21.15.67:0/786752313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b8155880 0x7f83b80a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 --2- 172.21.15.67:0/786752313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b806e860 0x7f83b80c25a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 -- 172.21.15.67:0/786752313 >> 172.21.15.67:0/786752313 conn(0x7f83b80c6da0 msgr2=0x7f83b80c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 -- 172.21.15.67:0/786752313 shutdown_connections 2024-02-26T10:42:27.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.460+0000 7f83c0200640 1 -- 172.21.15.67:0/786752313 wait complete. 2024-02-26T10:42:27.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.461+0000 7f83c0200640 1 Processor -- start 2024-02-26T10:42:27.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 -- start start 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b806e860 0x7f83b80c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 0x7f83b8150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b8155880 0x7f83b81ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b814f3a0 con 0x7f83b80a6150 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80c2220 con 0x7f83b8155880 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.462+0000 7f83c0200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b814f520 con 0x7f83b806e860 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 0x7f83b8150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 0x7f83b8150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50688/0 (socket says 172.21.15.67:50688) 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 -- 172.21.15.67:0/3619018187 learned_addr learned my addr 172.21.15.67:0/3619018187 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bd800640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b806e860 0x7f83b80c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 -- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b806e860 msgr2=0x7f83b80c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b806e860 0x7f83b80c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 -- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b8155880 msgr2=0x7f83b81ea4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b8155880 0x7f83b81ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 -- 172.21.15.67:0/3619018187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8154be0 con 0x7f83b80a6150 2024-02-26T10:42:27.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.463+0000 7f83bce00640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 0x7f83b8150cb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f83a402e190 tx=0x7f83a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.464+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a403b040 con 0x7f83b80a6150 2024-02-26T10:42:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.464+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4046070 con 0x7f83b80a6150 2024-02-26T10:42:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.464+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b8154ec0 con 0x7f83b80a6150 2024-02-26T10:42:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.464+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4034bb0 con 0x7f83b80a6150 2024-02-26T10:42:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.464+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b81e9fa0 con 0x7f83b80a6150 2024-02-26T10:42:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.465+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b8074a30 con 0x7f83b80a6150 2024-02-26T10:42:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.466+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83a404b460 con 0x7f83b80a6150 2024-02-26T10:42:27.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.467+0000 7f83b5600640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f839806b4d0 0x7f839806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.467+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f83a40af300 con 0x7f83b80a6150 2024-02-26T10:42:27.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.468+0000 7f83bd800640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f839806b4d0 0x7f839806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.469+0000 7f83bd800640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f839806b4d0 0x7f839806d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000f60 tx=0x7f83ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.471+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83a4049360 con 0x7f83b80a6150 2024-02-26T10:42:27.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.671+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83b8073280 con 0x7f83b80a6150 2024-02-26T10:42:27.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.672+0000 7f83b5600640 1 -- 172.21.15.67:0/3619018187 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83a407deb0 con 0x7f83b80a6150 2024-02-26T10:42:27.670 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:27.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f839806b4d0 msgr2=0x7f839806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f839806b4d0 0x7f839806d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000f60 tx=0x7f83ac009040 comp rx=0 tx=0).stop 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 msgr2=0x7f83b8150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 0x7f83b8150cb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f83a402e190 tx=0x7f83a40046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 shutdown_connections 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f839806b4d0 0x7f839806d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b8155880 0x7f83b81ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83b80a6150 0x7f83b8150cb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 --2- 172.21.15.67:0/3619018187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83b806e860 0x7f83b80c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.675+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 >> 172.21.15.67:0/3619018187 conn(0x7f83b80c6da0 msgr2=0x7f83b8070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.676+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 shutdown_connections 2024-02-26T10:42:27.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.676+0000 7f83c0200640 1 -- 172.21.15.67:0/3619018187 wait complete. 2024-02-26T10:42:27.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.921+0000 7fecc9600640 1 Processor -- start 2024-02-26T10:42:27.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.922+0000 7fecc9600640 1 -- start start 2024-02-26T10:42:27.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc40c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fecc406e860 0x7fecc40a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc4151d70 con 0x7fecc4155170 2024-02-26T10:42:27.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc4155d10 con 0x7fecc406e860 2024-02-26T10:42:27.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc9600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc4152b10 con 0x7fecc40a6150 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fecc406e860 0x7fecc40a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fecc4155170 0x7fecc4155600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47056/0 (socket says 172.21.15.67:47056) 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.923+0000 7fecc3400640 1 -- 172.21.15.67:0/1064512802 learned_addr learned my addr 172.21.15.67:0/1064512802 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.924+0000 7fecc2a00640 1 --2- 172.21.15.67:0/1064512802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc40c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.924+0000 7fecc2000640 1 -- 172.21.15.67:0/1064512802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 msgr2=0x7fecc40c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.924+0000 7fecc2000640 1 --2- 172.21.15.67:0/1064512802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc40c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.924+0000 7fecc2000640 1 -- 172.21.15.67:0/1064512802 >> v1:172.21.15.67:6789/0 conn(0x7fecc4155170 legacy=0x7fecc4155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.924+0000 7fecc2000640 1 -- 172.21.15.67:0/1064512802 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4152030 con 0x7fecc406e860 2024-02-26T10:42:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.924+0000 7fecc1600640 1 -- 172.21.15.67:0/1064512802 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331094685 0 0) 0x7fecc4151d70 con 0x7fecc4155170 2024-02-26T10:42:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.925+0000 7fecc2000640 1 --2- 172.21.15.67:0/1064512802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc406e860 0x7fecc40a6530 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fecb400ba40 tx=0x7fecb402dff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5f6effbd669b00 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.925+0000 7fecc1600640 1 -- 172.21.15.67:0/1064512802 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb403b070 con 0x7fecc406e860 2024-02-26T10:42:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc1600640 1 -- 172.21.15.67:0/1064512802 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb402eda0 con 0x7fecc406e860 2024-02-26T10:42:27.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc1600640 1 -- 172.21.15.67:0/1064512802 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4036660 con 0x7fecc406e860 2024-02-26T10:42:27.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 -- 172.21.15.67:0/1064512802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc406e860 msgr2=0x7fecc40a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 --2- 172.21.15.67:0/1064512802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc406e860 0x7fecc40a6530 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fecb400ba40 tx=0x7fecb402dff0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 -- 172.21.15.67:0/1064512802 shutdown_connections 2024-02-26T10:42:27.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 --2- 172.21.15.67:0/1064512802 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc406e860 0x7fecc40a6530 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 --2- 172.21.15.67:0/1064512802 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc40c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 -- 172.21.15.67:0/1064512802 >> 172.21.15.67:0/1064512802 conn(0x7fecc40c6da0 msgr2=0x7fecc40c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:27.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 -- 172.21.15.67:0/1064512802 shutdown_connections 2024-02-26T10:42:27.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.926+0000 7fecc9600640 1 -- 172.21.15.67:0/1064512802 wait complete. 2024-02-26T10:42:27.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.928+0000 7fecc9600640 1 Processor -- start 2024-02-26T10:42:27.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.928+0000 7fecc9600640 1 -- start start 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc406e860 0x7fecc414a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc414d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 0x7fecc41e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40c2220 con 0x7fecc4155170 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc9600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc4152680 con 0x7fecc406e860 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc9600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc414f3a0 con 0x7fecc40a6150 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 0x7fecc41e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 0x7fecc41e9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50720/0 (socket says 172.21.15.67:50720) 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.929+0000 7fecc3400640 1 -- 172.21.15.67:0/1968216771 learned_addr learned my addr 172.21.15.67:0/1968216771 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc2000640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc414d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc2a00640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc406e860 0x7fecc414a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc3400640 1 -- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 msgr2=0x7fecc414d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc3400640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc414d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc3400640 1 -- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc406e860 msgr2=0x7fecc414a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:27.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc3400640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc406e860 0x7fecc414a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:27.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc3400640 1 -- 172.21.15.67:0/1968216771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc41db550 con 0x7fecc4155170 2024-02-26T10:42:27.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecc3400640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 0x7fecc41e9e90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c700 tx=0x7fecbc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.930+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc018070 con 0x7fecc4155170 2024-02-26T10:42:27.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.931+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc00ce10 con 0x7fecc4155170 2024-02-26T10:42:27.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.931+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00a860 con 0x7fecc4155170 2024-02-26T10:42:27.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.931+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc41ea8f0 con 0x7fecc4155170 2024-02-26T10:42:27.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.931+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc40719b0 con 0x7fecc4155170 2024-02-26T10:42:27.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.932+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc414d1e0 con 0x7fecc4155170 2024-02-26T10:42:27.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.933+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecbc01b070 con 0x7fecc4155170 2024-02-26T10:42:27.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.934+0000 7fecbaa00640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feca006b4d0 0x7feca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:27.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.934+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fecbc089ad0 con 0x7fecc4155170 2024-02-26T10:42:27.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.935+0000 7fecc2a00640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feca006b4d0 0x7feca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:27.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.936+0000 7fecc2a00640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feca006b4d0 0x7feca006d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fecac006fd0 tx=0x7fecac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:27.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:27.938+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecbc058680 con 0x7fecc4155170 2024-02-26T10:42:28.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.138+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecc4074a30 con 0x7fecc4155170 2024-02-26T10:42:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.139+0000 7fecbaa00640 1 -- 172.21.15.67:0/1968216771 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecbc057f80 con 0x7fecc4155170 2024-02-26T10:42:28.137 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.141+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feca006b4d0 msgr2=0x7feca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feca006b4d0 0x7feca006d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fecac006fd0 tx=0x7fecac008040 comp rx=0 tx=0).stop 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 msgr2=0x7fecc41e9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 0x7fecc41e9e90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c700 tx=0x7fecbc00cbc0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 shutdown_connections 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecc4155170 0x7fecc41e9e90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecc40a6150 0x7fecc414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7feca006b4d0 0x7feca006d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 --2- 172.21.15.67:0/1968216771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc406e860 0x7fecc414a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 >> 172.21.15.67:0/1968216771 conn(0x7fecc40c6da0 msgr2=0x7fecc4151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 shutdown_connections 2024-02-26T10:42:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.142+0000 7fecc9600640 1 -- 172.21.15.67:0/1968216771 wait complete. 2024-02-26T10:42:28.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:42:28.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.388+0000 7f7344800640 1 Processor -- start 2024-02-26T10:42:28.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.388+0000 7f7344800640 1 -- start start 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.389+0000 7f7344800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f733c0a6150 0x7f733c0c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7344800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f733c0a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7344800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c151d70 con 0x7f733c06e860 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7344800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c155d10 con 0x7f733c0a6150 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7344800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c152b10 con 0x7f733c155880 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7341400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f733c06e860 0x7f733c155170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47070/0 (socket says 172.21.15.67:47070) 2024-02-26T10:42:28.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7341400640 1 -- 172.21.15.67:0/2674772051 learned_addr learned my addr 172.21.15.67:0/2674772051 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7341e00640 1 --2- 172.21.15.67:0/2674772051 >> v2:172.21.15.82:3300/0 conn(0x7f733c0a6150 0x7f733c0c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7342800640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f733c0a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.390+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180682023 0 0) 0x7f733c151d70 con 0x7f733c06e860 2024-02-26T10:42:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7324005730 con 0x7f733c06e860 2024-02-26T10:42:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468991282 0 0) 0x7f7324005730 con 0x7f733c06e860 2024-02-26T10:42:28.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 msgr2=0x7f733c0a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f733c0a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 >> v2:172.21.15.82:3300/0 conn(0x7f733c0a6150 msgr2=0x7f733c0c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 --2- 172.21.15.67:0/2674772051 >> v2:172.21.15.82:3300/0 conn(0x7f733c0a6150 0x7f733c0c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c152030 con 0x7f733c06e860 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f732c001670 con 0x7f733c06e860 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 >> v1:172.21.15.67:6789/0 conn(0x7f733c06e860 legacy=0x7f733c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f7324005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324006140 0x7f7324006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7324006f50 0x7f73240073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c152b10 con 0x7f7324006f50 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732400dbd0 con 0x7f7324006140 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732400dda0 con 0x7f733c155880 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.391+0000 7f7342800640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7324006f50 0x7f73240073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7341e00640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f7324005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7341400640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324006140 0x7f7324006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7342800640 1 -- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 msgr2=0x7f7324005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7342800640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f7324005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7342800640 1 -- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324006140 msgr2=0x7f7324006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7342800640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324006140 0x7f7324006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7342800640 1 -- 172.21.15.67:0/2674772051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324005730 con 0x7f7324006f50 2024-02-26T10:42:28.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.392+0000 7f7341e00640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c155880 0x7f7324005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.393+0000 7f7342800640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7324006f50 0x7f73240073d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f733800e170 tx=0x7f7324007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efd4680f04bbbe9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.393+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338039070 con 0x7f7324006f50 2024-02-26T10:42:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.393+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733802cbf0 con 0x7f7324006f50 2024-02-26T10:42:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.393+0000 7f7340a00640 1 -- 172.21.15.67:0/2674772051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338009c10 con 0x7f7324006f50 2024-02-26T10:42:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7344800640 1 -- 172.21.15.67:0/2674772051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7324006f50 msgr2=0x7f73240073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7344800640 1 --2- 172.21.15.67:0/2674772051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7324006f50 0x7f73240073d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f733800e170 tx=0x7f7324007a40 comp rx=0 tx=0).stop 2024-02-26T10:42:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7342800640 1 -- 172.21.15.67:0/2674772051 reap_dead start 2024-02-26T10:42:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7344800640 1 -- 172.21.15.67:0/2674772051 shutdown_connections 2024-02-26T10:42:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7344800640 1 -- 172.21.15.67:0/2674772051 >> 172.21.15.67:0/2674772051 conn(0x7f733c0c6da0 msgr2=0x7f733c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7344800640 1 -- 172.21.15.67:0/2674772051 shutdown_connections 2024-02-26T10:42:28.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.394+0000 7f7344800640 1 -- 172.21.15.67:0/2674772051 wait complete. 2024-02-26T10:42:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.396+0000 7f7344800640 1 Processor -- start 2024-02-26T10:42:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.396+0000 7f7344800640 1 -- start start 2024-02-26T10:42:28.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7344800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f733c1dd7b0 0x7f733c147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7344800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 0x7f733c147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7344800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c148b60 0x7f733c1db670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7344800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0c2220 con 0x7f733c0a5d10 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7344800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c152680 con 0x7f733c1dd7b0 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7344800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c14f3a0 con 0x7f733c148b60 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7341400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 0x7f733c147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7341400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 0x7f733c147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50742/0 (socket says 172.21.15.67:50742) 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.397+0000 7f7341400640 1 -- 172.21.15.67:0/1847894467 learned_addr learned my addr 172.21.15.67:0/1847894467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341e00640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f733c1dd7b0 0x7f733c147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7342800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c148b60 0x7f733c1db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341400640 1 -- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c148b60 msgr2=0x7f733c1db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341400640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c148b60 0x7f733c1db670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341400640 1 -- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f733c1dd7b0 msgr2=0x7f733c147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341400640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f733c1dd7b0 0x7f733c147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341400640 1 -- 172.21.15.67:0/1847894467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c148910 con 0x7f733c0a5d10 2024-02-26T10:42:28.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341e00640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f733c1dd7b0 0x7f733c147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7341400640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 0x7f733c147e90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f732c00eaf0 tx=0x7f732c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00c900 con 0x7f733c0a5d10 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c018070 con 0x7f733c0a5d10 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c148630 con 0x7f733c0a5d10 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.398+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00cd00 con 0x7f733c0a5d10 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.399+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c1e7410 con 0x7f733c0a5d10 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.400+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c074a30 con 0x7f733c0a5d10 2024-02-26T10:42:28.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.401+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f732c01d460 con 0x7f733c0a5d10 2024-02-26T10:42:28.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.402+0000 7f7336000640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f732006b4d0 0x7f732006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.402+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f732c08a560 con 0x7f733c0a5d10 2024-02-26T10:42:28.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.403+0000 7f7341e00640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f732006b4d0 0x7f732006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.404+0000 7f7341e00640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f732006b4d0 0x7f732006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f733c0a8350 tx=0x7f7330008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:28.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.406+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f732c059110 con 0x7f733c0a5d10 2024-02-26T10:42:28.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.610+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f733c073280 con 0x7f733c0a5d10 2024-02-26T10:42:28.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.611+0000 7f7336000640 1 -- 172.21.15.67:0/1847894467 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7f732c058a10 con 0x7f733c0a5d10 2024-02-26T10:42:28.609 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:28.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f732006b4d0 msgr2=0x7f732006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f732006b4d0 0x7f732006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f733c0a8350 tx=0x7f7330008040 comp rx=0 tx=0).stop 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 msgr2=0x7f733c147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 0x7f733c147e90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f732c00eaf0 tx=0x7f732c001770 comp rx=0 tx=0).stop 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 shutdown_connections 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f733c148b60 0x7f733c1db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f733c0a5d10 0x7f733c147e90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f732006b4d0 0x7f732006d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 --2- 172.21.15.67:0/1847894467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f733c1dd7b0 0x7f733c147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 >> 172.21.15.67:0/1847894467 conn(0x7f733c0c6da0 msgr2=0x7f733c070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.614+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 shutdown_connections 2024-02-26T10:42:28.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.615+0000 7f7344800640 1 -- 172.21.15.67:0/1847894467 wait complete. 2024-02-26T10:42:28.633 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:42:28.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.855+0000 7f08d6800640 1 Processor -- start 2024-02-26T10:42:28.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.855+0000 7f08d6800640 1 -- start start 2024-02-26T10:42:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.856+0000 7f08d6800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 0x7f08d00c7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d6800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08d0158fe0 0x7f08d015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d6800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d0155c90 con 0x7f08d00c7940 2024-02-26T10:42:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d6800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d00c08a0 con 0x7f08d0158fe0 2024-02-26T10:42:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d6800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d00c0b80 con 0x7f08d014ee90 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d5400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 0x7f08d00c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d5e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08d0158fe0 0x7f08d015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d5400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 0x7f08d00c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:58648/0 (socket says 172.21.15.67:58648) 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.857+0000 7f08d5400640 1 -- 172.21.15.67:0/572513303 learned_addr learned my addr 172.21.15.67:0/572513303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.858+0000 7f08d5400640 1 -- 172.21.15.67:0/572513303 >> v2:172.21.15.82:3300/0 conn(0x7f08d0158fe0 msgr2=0x7f08d015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.858+0000 7f08d5400640 1 --2- 172.21.15.67:0/572513303 >> v2:172.21.15.82:3300/0 conn(0x7f08d0158fe0 0x7f08d015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.858+0000 7f08d5400640 1 -- 172.21.15.67:0/572513303 >> v1:172.21.15.82:6789/0 conn(0x7f08d00c7940 legacy=0x7f08d01588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.858+0000 7f08d5400640 1 -- 172.21.15.67:0/572513303 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d01559b0 con 0x7f08d014ee90 2024-02-26T10:42:28.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.859+0000 7f08d5400640 1 --2- 172.21.15.67:0/572513303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 0x7f08d00c7400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009e50 tx=0x7f08c002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa2d917482ee2d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:28.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.860+0000 7f08cfe00640 1 -- 172.21.15.67:0/572513303 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c002ca80 con 0x7f08d014ee90 2024-02-26T10:42:28.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.860+0000 7f08cfe00640 1 -- 172.21.15.67:0/572513303 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c002cc20 con 0x7f08d014ee90 2024-02-26T10:42:28.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.860+0000 7f08cfe00640 1 -- 172.21.15.67:0/572513303 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00343f0 con 0x7f08d014ee90 2024-02-26T10:42:28.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 -- 172.21.15.67:0/572513303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 msgr2=0x7f08d00c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 --2- 172.21.15.67:0/572513303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 0x7f08d00c7400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009e50 tx=0x7f08c002bee0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 -- 172.21.15.67:0/572513303 shutdown_connections 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 --2- 172.21.15.67:0/572513303 >> v2:172.21.15.82:3300/0 conn(0x7f08d0158fe0 0x7f08d015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 --2- 172.21.15.67:0/572513303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d014ee90 0x7f08d00c7400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 -- 172.21.15.67:0/572513303 >> 172.21.15.67:0/572513303 conn(0x7f08d006dc20 msgr2=0x7f08d006e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 -- 172.21.15.67:0/572513303 shutdown_connections 2024-02-26T10:42:28.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.861+0000 7f08d6800640 1 -- 172.21.15.67:0/572513303 wait complete. 2024-02-26T10:42:28.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.862+0000 7f08d6800640 1 Processor -- start 2024-02-26T10:42:28.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.863+0000 7f08d6800640 1 -- start start 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.863+0000 7f08d6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 0x7f08d01ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.863+0000 7f08d6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d014ee90 0x7f08d013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.863+0000 7f08d6800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d0158fe0 0x7f08d0140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.863+0000 7f08d6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d006ac20 con 0x7f08d00c7940 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d6800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0072450 con 0x7f08d014ee90 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d6800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d006ada0 con 0x7f08d0158fe0 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 0x7f08d01ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 0x7f08d01ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50746/0 (socket says 172.21.15.67:50746) 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 -- 172.21.15.67:0/2139389943 learned_addr learned my addr 172.21.15.67:0/2139389943 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d4a00640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d014ee90 0x7f08d013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5e00640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d0158fe0 0x7f08d0140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 -- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d0158fe0 msgr2=0x7f08d0140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d0158fe0 0x7f08d0140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 -- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d014ee90 msgr2=0x7f08d013fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d014ee90 0x7f08d013fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.864+0000 7f08d5400640 1 -- 172.21.15.67:0/2139389943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d014f960 con 0x7f08d00c7940 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.865+0000 7f08d5400640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 0x7f08d01ee300 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000f80 tx=0x7f08c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.865+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0043020 con 0x7f08d00c7940 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.865+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c003e450 con 0x7f08d00c7940 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.865+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0009510 con 0x7f08d00c7940 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.865+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d01eebd0 con 0x7f08d00c7940 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.865+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d014f310 con 0x7f08d00c7940 2024-02-26T10:42:28.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.866+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d0074a30 con 0x7f08d00c7940 2024-02-26T10:42:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.867+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08c0046020 con 0x7f08d00c7940 2024-02-26T10:42:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.868+0000 7f08cd600640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f08b006b4d0 0x7f08b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.868+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f08c00acd60 con 0x7f08d00c7940 2024-02-26T10:42:28.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.869+0000 7f08d4a00640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f08b006b4d0 0x7f08b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:28.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.870+0000 7f08d4a00640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f08b006b4d0 0x7f08b006d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f08d006c640 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:28.873+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08c004d030 con 0x7f08d00c7940 2024-02-26T10:42:29.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.072+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f08d00c4a30 con 0x7f08d00c7940 2024-02-26T10:42:29.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.072+0000 7f08cd600640 1 -- 172.21.15.67:0/2139389943 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7f08c007b7f0 con 0x7f08d00c7940 2024-02-26T10:42:29.070 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f08b006b4d0 msgr2=0x7f08b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f08b006b4d0 0x7f08b006d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f08d006c640 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 msgr2=0x7f08d01ee300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 0x7f08d01ee300 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000f80 tx=0x7f08c0004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 shutdown_connections 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08d0158fe0 0x7f08d0140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f08b006b4d0 0x7f08b006d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d014ee90 0x7f08d013fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 --2- 172.21.15.67:0/2139389943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f08d00c7940 0x7f08d01ee300 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.075+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 >> 172.21.15.67:0/2139389943 conn(0x7f08d006dc20 msgr2=0x7f08d0156d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.076+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 shutdown_connections 2024-02-26T10:42:29.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.076+0000 7f08d6800640 1 -- 172.21.15.67:0/2139389943 wait complete. 2024-02-26T10:42:29.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:42:29.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.307+0000 7fc670c00640 1 Processor -- start 2024-02-26T10:42:29.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.308+0000 7fc670c00640 1 -- start start 2024-02-26T10:42:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.309+0000 7fc670c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c152400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.309+0000 7fc670c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc66c152940 0x7fc66c070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.309+0000 7fc670c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c0c3750 con 0x7fc66c0c7140 2024-02-26T10:42:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.309+0000 7fc670c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0c39f0 con 0x7fc66c152940 2024-02-26T10:42:29.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.309+0000 7fc670c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c3cd0 con 0x7fc66c0713e0 2024-02-26T10:42:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.310+0000 7fc669e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc66c0c7140 0x7fc66c0c64c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:44362/0 (socket says 172.21.15.67:44362) 2024-02-26T10:42:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.310+0000 7fc669e00640 1 -- 172.21.15.67:0/2405558911 learned_addr learned my addr 172.21.15.67:0/2405558911 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.310+0000 7fc669400640 1 --2- 172.21.15.67:0/2405558911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:29.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.310+0000 7fc66a800640 1 --2- 172.21.15.67:0/2405558911 >> v2:172.21.15.82:3300/0 conn(0x7fc66c152940 0x7fc66c070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.311+0000 7fc668a00640 1 -- 172.21.15.67:0/2405558911 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123521671 0 0) 0x7fc66c0c3750 con 0x7fc66c0c7140 2024-02-26T10:42:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.311+0000 7fc669400640 1 -- 172.21.15.67:0/2405558911 >> v2:172.21.15.82:3300/0 conn(0x7fc66c152940 msgr2=0x7fc66c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.311+0000 7fc669400640 1 --2- 172.21.15.67:0/2405558911 >> v2:172.21.15.82:3300/0 conn(0x7fc66c152940 0x7fc66c070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.311+0000 7fc669400640 1 -- 172.21.15.67:0/2405558911 >> v1:172.21.15.82:6789/0 conn(0x7fc66c0c7140 legacy=0x7fc66c0c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.311+0000 7fc669400640 1 -- 172.21.15.67:0/2405558911 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c0c3fb0 con 0x7fc66c0713e0 2024-02-26T10:42:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.312+0000 7fc669400640 1 --2- 172.21.15.67:0/2405558911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c152400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009d90 tx=0x7fc65c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652e68adfefcfc54 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc668a00640 1 -- 172.21.15.67:0/2405558911 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c039070 con 0x7fc66c0713e0 2024-02-26T10:42:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc668a00640 1 -- 172.21.15.67:0/2405558911 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c02cbc0 con 0x7fc66c0713e0 2024-02-26T10:42:29.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc668a00640 1 -- 172.21.15.67:0/2405558911 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c034440 con 0x7fc66c0713e0 2024-02-26T10:42:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 -- 172.21.15.67:0/2405558911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 msgr2=0x7fc66c152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 --2- 172.21.15.67:0/2405558911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c152400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009d90 tx=0x7fc65c02be10 comp rx=0 tx=0).stop 2024-02-26T10:42:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 -- 172.21.15.67:0/2405558911 shutdown_connections 2024-02-26T10:42:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 --2- 172.21.15.67:0/2405558911 >> v2:172.21.15.82:3300/0 conn(0x7fc66c152940 0x7fc66c070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 --2- 172.21.15.67:0/2405558911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c152400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 -- 172.21.15.67:0/2405558911 >> 172.21.15.67:0/2405558911 conn(0x7fc66c06b3b0 msgr2=0x7fc66c0aa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 -- 172.21.15.67:0/2405558911 shutdown_connections 2024-02-26T10:42:29.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.313+0000 7fc670c00640 1 -- 172.21.15.67:0/2405558911 wait complete. 2024-02-26T10:42:29.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.315+0000 7fc670c00640 1 Processor -- start 2024-02-26T10:42:29.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.315+0000 7fc670c00640 1 -- start start 2024-02-26T10:42:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc670c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc670c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 0x7fc66c14a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc670c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc66c152940 0x7fc66c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc670c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c76a0 con 0x7fc66c0c7140 2024-02-26T10:42:29.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc670c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c7520 con 0x7fc66c152940 2024-02-26T10:42:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc670c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c070770 con 0x7fc66c0713e0 2024-02-26T10:42:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc669400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 0x7fc66c14a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc669400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 0x7fc66c14a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50748/0 (socket says 172.21.15.67:50748) 2024-02-26T10:42:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.316+0000 7fc669400640 1 -- 172.21.15.67:0/1489558485 learned_addr learned my addr 172.21.15.67:0/1489558485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669e00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:29.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc66a800640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc66c152940 0x7fc66c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669400640 1 -- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 msgr2=0x7fc66c151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669400640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669400640 1 -- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc66c152940 msgr2=0x7fc66c14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669400640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc66c152940 0x7fc66c14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669400640 1 -- 172.21.15.67:0/1489558485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c0c4830 con 0x7fc66c0c7140 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.317+0000 7fc669400640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 0x7fc66c14a3a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009d90 tx=0x7fc65c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.318+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c039050 con 0x7fc66c0c7140 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.318+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c03a040 con 0x7fc66c0c7140 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.318+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c009450 con 0x7fc66c0c7140 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.318+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c0ad5c0 con 0x7fc66c0c7140 2024-02-26T10:42:29.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.318+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c0ad2e0 con 0x7fc66c0c7140 2024-02-26T10:42:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.319+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c0c4e20 con 0x7fc66c0c7140 2024-02-26T10:42:29.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.320+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc65c047020 con 0x7fc66c0c7140 2024-02-26T10:42:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.321+0000 7fc662000640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc64806b4d0 0x7fc64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.321+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc65c0ac790 con 0x7fc66c0c7140 2024-02-26T10:42:29.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.322+0000 7fc669e00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc64806b4d0 0x7fc64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:29.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.323+0000 7fc669e00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc64806b4d0 0x7fc64806d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc654000f60 tx=0x7fc654009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:29.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.325+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc65c04e020 con 0x7fc66c0c7140 2024-02-26T10:42:29.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.537+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc66c074a30 con 0x7fc66c0c7140 2024-02-26T10:42:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.538+0000 7fc662000640 1 -- 172.21.15.67:0/1489558485 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc66c0c4830 con 0x7fc66c0c7140 2024-02-26T10:42:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.540+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc64806b4d0 msgr2=0x7fc64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.540+0000 7fc670c00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc64806b4d0 0x7fc64806d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc654000f60 tx=0x7fc654009040 comp rx=0 tx=0).stop 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.540+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 msgr2=0x7fc66c14a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 0x7fc66c14a3a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009d90 tx=0x7fc65c004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 shutdown_connections 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc64806b4d0 0x7fc64806d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc66c152940 0x7fc66c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc66c0c7140 0x7fc66c14a3a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 --2- 172.21.15.67:0/1489558485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc66c0713e0 0x7fc66c151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 >> 172.21.15.67:0/1489558485 conn(0x7fc66c06b3b0 msgr2=0x7fc66c0a9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:29.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 shutdown_connections 2024-02-26T10:42:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:29.541+0000 7fc670c00640 1 -- 172.21.15.67:0/1489558485 wait complete. 2024-02-26T10:42:29.559 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:42:29.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:29.560 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:29.673 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:42:29.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:29.673 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:42:29.722 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:29.722 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:29.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:42:29.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:42:29.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.002+0000 7f5692400640 1 Processor -- start 2024-02-26T10:42:30.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.002+0000 7f5692400640 1 -- start start 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.003+0000 7f5692400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f568c0c1c10 0x7f568c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.003+0000 7f5692400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f568c06e180 0x7f568c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.003+0000 7f5692400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c155cf0 con 0x7f568c0c0850 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.003+0000 7f5692400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c06dca0 con 0x7f568c0c1c10 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.003+0000 7f5692400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0c2a70 con 0x7f568c06e180 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.004+0000 7f5691000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f568c0c1c10 0x7f568c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.004+0000 7f5691000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f568c0c1c10 0x7f568c1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36998/0 (socket says 172.21.15.67:36998) 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.004+0000 7f5691000640 1 -- 172.21.15.67:0/2293507742 learned_addr learned my addr 172.21.15.67:0/2293507742 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.004+0000 7f5691a00640 1 --2- 172.21.15.67:0/2293507742 >> v2:172.21.15.82:3300/0 conn(0x7f568c06e180 0x7f568c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.004+0000 7f5691000640 1 -- 172.21.15.67:0/2293507742 >> v2:172.21.15.82:3300/0 conn(0x7f568c06e180 msgr2=0x7f568c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.004+0000 7f5691000640 1 --2- 172.21.15.67:0/2293507742 >> v2:172.21.15.82:3300/0 conn(0x7f568c06e180 0x7f568c06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.005+0000 7f5691000640 1 -- 172.21.15.67:0/2293507742 >> v1:172.21.15.82:6789/0 conn(0x7f568c0c0850 legacy=0x7f568c06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.005+0000 7f5691000640 1 -- 172.21.15.67:0/2293507742 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0c21d0 con 0x7f568c0c1c10 2024-02-26T10:42:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.006+0000 7f5691000640 1 --2- 172.21.15.67:0/2293507742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c0c1c10 0x7f568c1557b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f568c0bf8b0 tx=0x7f567402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea5680d400fbff6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f568be00640 1 -- 172.21.15.67:0/2293507742 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567402e070 con 0x7f568c0c1c10 2024-02-26T10:42:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f568be00640 1 -- 172.21.15.67:0/2293507742 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674037070 con 0x7f568c0c1c10 2024-02-26T10:42:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f568be00640 1 -- 172.21.15.67:0/2293507742 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674031a00 con 0x7f568c0c1c10 2024-02-26T10:42:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f5692400640 1 -- 172.21.15.67:0/2293507742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c0c1c10 msgr2=0x7f568c1557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f5692400640 1 --2- 172.21.15.67:0/2293507742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c0c1c10 0x7f568c1557b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f568c0bf8b0 tx=0x7f567402b880 comp rx=0 tx=0).stop 2024-02-26T10:42:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f5692400640 1 -- 172.21.15.67:0/2293507742 shutdown_connections 2024-02-26T10:42:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f5692400640 1 --2- 172.21.15.67:0/2293507742 >> v2:172.21.15.82:3300/0 conn(0x7f568c06e180 0x7f568c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f5692400640 1 --2- 172.21.15.67:0/2293507742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c0c1c10 0x7f568c1557b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.007+0000 7f5692400640 1 -- 172.21.15.67:0/2293507742 >> 172.21.15.67:0/2293507742 conn(0x7f568c0c63f0 msgr2=0x7f568c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.008+0000 7f5692400640 1 -- 172.21.15.67:0/2293507742 shutdown_connections 2024-02-26T10:42:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.008+0000 7f5692400640 1 -- 172.21.15.67:0/2293507742 wait complete. 2024-02-26T10:42:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.010+0000 7f5692400640 1 Processor -- start 2024-02-26T10:42:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.010+0000 7f5692400640 1 -- start start 2024-02-26T10:42:30.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5692400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 0x7f568c1eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5692400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f568c0c0850 0x7f568c1eb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5692400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568c0c1c10 0x7f568c14b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5692400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c06b330 con 0x7f568c06e180 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5692400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0c2790 con 0x7f568c0c1c10 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5692400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c06b4b0 con 0x7f568c0c0850 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5691000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 0x7f568c1eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5691000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 0x7f568c1eed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37004/0 (socket says 172.21.15.67:37004) 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5691000640 1 -- 172.21.15.67:0/3805853488 learned_addr learned my addr 172.21.15.67:0/3805853488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.011+0000 7f5691a00640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568c0c1c10 0x7f568c14b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5683e00640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f568c0c0850 0x7f568c1eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5691000640 1 -- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f568c0c0850 msgr2=0x7f568c1eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5691000640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f568c0c0850 0x7f568c1eb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5691000640 1 -- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568c0c1c10 msgr2=0x7f568c14b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5691000640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568c0c1c10 0x7f568c14b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5691000640 1 -- 172.21.15.67:0/3805853488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c14bb10 con 0x7f568c06e180 2024-02-26T10:42:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5691000640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 0x7f568c1eed00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f567402b8c0 tx=0x7f56740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.012+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674032880 con 0x7f568c06e180 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.013+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674032a20 con 0x7f568c06e180 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.013+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674032e10 con 0x7f568c06e180 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.013+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c14bf10 con 0x7f568c06e180 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.013+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c14fb60 con 0x7f568c06e180 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.014+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c071870 con 0x7f568c06e180 2024-02-26T10:42:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.015+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56740453f0 con 0x7f568c06e180 2024-02-26T10:42:30.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.016+0000 7f5689600640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f566806b500 0x7f566806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:30.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.016+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f56740ac4e0 con 0x7f568c06e180 2024-02-26T10:42:30.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.016+0000 7f5683e00640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f566806b500 0x7f566806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:30.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.017+0000 7f5683e00640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f566806b500 0x7f566806d9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f568c0c0fa0 tx=0x7f567c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.020+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f568c071870 con 0x7f568c06e180 2024-02-26T10:42:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.261+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f568c06bea0 con 0x7f568c06e180 2024-02-26T10:42:30.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.262+0000 7f5689600640 1 -- 172.21.15.67:0/3805853488 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+149 (secure 0 0 0) 0x7f568c06bea0 con 0x7f568c06e180 2024-02-26T10:42:30.260 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:42:30.260 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f566806b500 msgr2=0x7f566806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f566806b500 0x7f566806d9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f568c0c0fa0 tx=0x7f567c009040 comp rx=0 tx=0).stop 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 msgr2=0x7f568c1eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 0x7f568c1eed00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f567402b8c0 tx=0x7f56740046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 shutdown_connections 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f566806b500 0x7f566806d9c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568c0c1c10 0x7f568c14b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f568c0c0850 0x7f568c1eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 --2- 172.21.15.67:0/3805853488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f568c06e180 0x7f568c1eed00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:30.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.265+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 >> 172.21.15.67:0/3805853488 conn(0x7f568c0c63f0 msgr2=0x7f568c06ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:30.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.266+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 shutdown_connections 2024-02-26T10:42:30.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:30.266+0000 7f5692400640 1 -- 172.21.15.67:0/3805853488 wait complete. 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-26T10:42:30.285 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:30.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.286 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:42:30.341 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:42:30.341 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:42:30.341 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:42:30.341 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:30.341 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:42:30.341 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:42:30.342 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:42:30.343 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:30.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.344 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:42:30.434 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:42:30.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.438 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:42:30.487 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:42:30.487 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:42:30.487 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:42:30.487 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:42:30.487 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:42:30.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.488 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:42:30.488 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:42:30.488 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:42:30.488 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:42:30.488 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:42:30.488 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:42:30.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.659 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:42:30.704 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:42:30.705 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:30.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:42:30.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:30.839 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:30.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:42:30.890 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:30.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:30.890 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:30.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:42:30.938 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs2=/ 2024-02-26T10:42:30.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:30.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:31.114 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:31.114 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:31.150 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:31.150 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:31.150 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:42:31.150 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 1@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs2=/ 2024-02-26T10:42:31.150 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:42:31.152 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:42:31.152 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:42:31.152 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:42:31.153 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:42:31.153 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:42:31.157 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:42:31.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:31.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:31.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:31.243 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:42:31.320 INFO:teuthology.orchestra.run.smithi082.stdout:client6798 2024-02-26T10:42:31.321 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:31.321 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:31.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:31.376 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/testdir) 2024-02-26T10:42:31.444 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:31.444 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/mnt.0/testdir/testfile 2024-02-26T10:42:31.510 INFO:tasks.cephfs.caps_helper:Test file has been created on FS "cephfs" at path "/home/ubuntu/cephtest/mnt.0/testdir/testfile" with the following data - self.path = /home/ubuntu/cephtest/mnt.0/testdir/testfile cephfs_name = cephfs cephfs_mntpt = / hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:31.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:31.511 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec mkdir /home/ubuntu/cephtest/mnt.1/testdir) 2024-02-26T10:42:31.584 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:31.584 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/mnt.1/testdir/testfile 2024-02-26T10:42:31.647 INFO:tasks.cephfs.caps_helper:Test file has been created on FS "cephfs2" at path "/home/ubuntu/cephtest/mnt.1/testdir/testfile" with the following data - self.path = /home/ubuntu/cephtest/mnt.1/testdir/testfile cephfs_name = cephfs2 cephfs_mntpt = / hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:31.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth add client.testuser mon 'allow r' osd 'allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2' mds 'allow r fsname=cephfs, allow r fsname=cephfs2' 2024-02-26T10:42:31.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.891+0000 7fc7c7e00640 1 Processor -- start 2024-02-26T10:42:31.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.891+0000 7fc7c7e00640 1 -- start start 2024-02-26T10:42:31.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c813c000 0x7fc7c8072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:31.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc7c806e460 0x7fc7c80c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:31.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c7e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c806e8c0 con 0x7fc7c80c3670 2024-02-26T10:42:31.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c806eb60 con 0x7fc7c806e460 2024-02-26T10:42:31.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81564b0 con 0x7fc7c813c000 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc7c806e460 0x7fc7c80c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc7c806e460 0x7fc7c80c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37014/0 (socket says 172.21.15.67:37014) 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c6000640 1 -- 172.21.15.67:0/1220895565 learned_addr learned my addr 172.21.15.67:0/1220895565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.893+0000 7fc7c6a00640 1 --2- 172.21.15.67:0/1220895565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c813c000 0x7fc7c8072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.894+0000 7fc7c6000640 1 -- 172.21.15.67:0/1220895565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c813c000 msgr2=0x7fc7c8072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.894+0000 7fc7c6000640 1 --2- 172.21.15.67:0/1220895565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c813c000 0x7fc7c8072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.894+0000 7fc7c6000640 1 -- 172.21.15.67:0/1220895565 >> v1:172.21.15.82:6789/0 conn(0x7fc7c80c3670 legacy=0x7fc7c8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:31.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.894+0000 7fc7c6000640 1 -- 172.21.15.67:0/1220895565 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c8156790 con 0x7fc7c806e460 2024-02-26T10:42:31.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.895+0000 7fc7c6000640 1 --2- 172.21.15.67:0/1220895565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c806e460 0x7fc7c80c3130 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009870 tx=0x7fc7b402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718613006c7c7c56 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:31.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.895+0000 7fc7c5600640 1 -- 172.21.15.67:0/1220895565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40317c0 con 0x7fc7c806e460 2024-02-26T10:42:31.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.895+0000 7fc7c5600640 1 -- 172.21.15.67:0/1220895565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b4037070 con 0x7fc7c806e460 2024-02-26T10:42:31.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.895+0000 7fc7c5600640 1 -- 172.21.15.67:0/1220895565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4031b80 con 0x7fc7c806e460 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 -- 172.21.15.67:0/1220895565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c806e460 msgr2=0x7fc7c80c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/1220895565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c806e460 0x7fc7c80c3130 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009870 tx=0x7fc7b402b860 comp rx=0 tx=0).stop 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 -- 172.21.15.67:0/1220895565 shutdown_connections 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/1220895565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c806e460 0x7fc7c80c3130 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/1220895565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c813c000 0x7fc7c8072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 -- 172.21.15.67:0/1220895565 >> 172.21.15.67:0/1220895565 conn(0x7fc7c80a84a0 msgr2=0x7fc7c80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 -- 172.21.15.67:0/1220895565 shutdown_connections 2024-02-26T10:42:31.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.896+0000 7fc7c7e00640 1 -- 172.21.15.67:0/1220895565 wait complete. 2024-02-26T10:42:31.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.899+0000 7fc7c7e00640 1 Processor -- start 2024-02-26T10:42:31.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.899+0000 7fc7c7e00640 1 -- start start 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c806e460 0x7fc7c80c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c80c3670 0x7fc7c8151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 0x7fc7c8137830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8157270 con 0x7fc7c813c000 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8156d50 con 0x7fc7c80c3670 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c813c3e0 con 0x7fc7c806e460 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.900+0000 7fc7c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 0x7fc7c8137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 0x7fc7c8137830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37022/0 (socket says 172.21.15.67:37022) 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 -- 172.21.15.67:0/13200820 learned_addr learned my addr 172.21.15.67:0/13200820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c6a00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c806e460 0x7fc7c80c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c6000640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c80c3670 0x7fc7c8151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 -- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c806e460 msgr2=0x7fc7c80c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:31.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c806e460 0x7fc7c80c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:31.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 -- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c80c3670 msgr2=0x7fc7c8151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:31.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c80c3670 0x7fc7c8151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:31.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 -- 172.21.15.67:0/13200820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80c4580 con 0x7fc7c813c000 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.901+0000 7fc7c7400640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 0x7fc7c8137830 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c780 tx=0x7fc7c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.902+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004b60 con 0x7fc7c813c000 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.902+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0004d00 con 0x7fc7c813c000 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.902+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000aa90 con 0x7fc7c813c000 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.902+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c80c4860 con 0x7fc7c813c000 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.902+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c80c4e20 con 0x7fc7c813c000 2024-02-26T10:42:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.903+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8154000 con 0x7fc7c813c000 2024-02-26T10:42:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.904+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7c000ac30 con 0x7fc7c813c000 2024-02-26T10:42:31.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.905+0000 7fc7bea00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7a806b4d0 0x7fc7a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:31.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.905+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc7c008a950 con 0x7fc7c813c000 2024-02-26T10:42:31.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.906+0000 7fc7c6a00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7a806b4d0 0x7fc7a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:31.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.907+0000 7fc7c6a00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7a806b4d0 0x7fc7a806d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8006fd0 tx=0x7fc7b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:31.910+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7c8154000 con 0x7fc7c813c000 2024-02-26T10:42:32.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.181+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow r", "osd", "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2", "mds", "allow r fsname=cephfs, allow r fsname=cephfs2"]} v 0) v1 -- 0x7fc7c8074a30 con 0x7fc7c813c000 2024-02-26T10:42:32.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.186+0000 7fc7bea00640 1 -- 172.21.15.67:0/13200820 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow r", "osd", "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2", "mds", "allow r fsname=cephfs, allow r fsname=cephfs2"]}]=0 added key for client.testuser v19) v1 ==== 272+0+0 (secure 0 0 0) 0x7fc7c0020070 con 0x7fc7c813c000 2024-02-26T10:42:32.184 INFO:teuthology.orchestra.run.smithi067.stderr:added key for client.testuser 2024-02-26T10:42:32.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7a806b4d0 msgr2=0x7fc7a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7a806b4d0 0x7fc7a806d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8006fd0 tx=0x7fc7b8008040 comp rx=0 tx=0).stop 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 msgr2=0x7fc7c8137830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 0x7fc7c8137830 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c780 tx=0x7fc7c000cc40 comp rx=0 tx=0).stop 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 shutdown_connections 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7c813c000 0x7fc7c8137830 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc7a806b4d0 0x7fc7a806d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c80c3670 0x7fc7c8151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 --2- 172.21.15.67:0/13200820 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7c806e460 0x7fc7c80c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 >> 172.21.15.67:0/13200820 conn(0x7fc7c80a84a0 msgr2=0x7fc7c80bf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:32.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 shutdown_connections 2024-02-26T10:42:32.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.189+0000 7fc7c7e00640 1 -- 172.21.15.67:0/13200820 wait complete. 2024-02-26T10:42:32.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.testuser 2024-02-26T10:42:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.431+0000 7fd054400640 1 Processor -- start 2024-02-26T10:42:32.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.431+0000 7fd054400640 1 -- start start 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd054400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd04c156810 0x7fd04c06e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd054400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd04c159210 0x7fd04c159660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd054400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c15c8a0 con 0x7fd04c072fe0 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd054400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c15cb80 con 0x7fd04c156810 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd054400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c15ce60 con 0x7fd04c159210 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd051000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd04c072fe0 0x7fd04c0c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47018/0 (socket says 172.21.15.67:47018) 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd051000640 1 -- 172.21.15.67:0/1139237764 learned_addr learned my addr 172.21.15.67:0/1139237764 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.433+0000 7fd051a00640 1 --2- 172.21.15.67:0/1139237764 >> v2:172.21.15.67:3300/0 conn(0x7fd04c156810 0x7fd04c06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd052400640 1 --2- 172.21.15.67:0/1139237764 >> v2:172.21.15.82:3300/0 conn(0x7fd04c159210 0x7fd04c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd04be00640 1 -- 172.21.15.67:0/1139237764 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429929213 0 0) 0x7fd04c15c8a0 con 0x7fd04c072fe0 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd04be00640 1 -- 172.21.15.67:0/1139237764 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd034003520 con 0x7fd04c072fe0 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd051a00640 1 -- 172.21.15.67:0/1139237764 >> v2:172.21.15.82:3300/0 conn(0x7fd04c159210 msgr2=0x7fd04c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd051a00640 1 --2- 172.21.15.67:0/1139237764 >> v2:172.21.15.82:3300/0 conn(0x7fd04c159210 0x7fd04c159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd051a00640 1 -- 172.21.15.67:0/1139237764 >> v1:172.21.15.67:6789/0 conn(0x7fd04c072fe0 legacy=0x7fd04c0c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.434+0000 7fd051a00640 1 -- 172.21.15.67:0/1139237764 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c15d140 con 0x7fd04c156810 2024-02-26T10:42:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.435+0000 7fd051a00640 1 --2- 172.21.15.67:0/1139237764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c06e660 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd04c06e410 tx=0x7fd04002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268413cb384f447c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.435+0000 7fd04be00640 1 -- 172.21.15.67:0/1139237764 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400315a0 con 0x7fd04c156810 2024-02-26T10:42:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.435+0000 7fd04be00640 1 -- 172.21.15.67:0/1139237764 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040037070 con 0x7fd04c156810 2024-02-26T10:42:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.435+0000 7fd04be00640 1 -- 172.21.15.67:0/1139237764 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040031960 con 0x7fd04c156810 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 -- 172.21.15.67:0/1139237764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 msgr2=0x7fd04c06e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 --2- 172.21.15.67:0/1139237764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c06e660 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd04c06e410 tx=0x7fd04002b840 comp rx=0 tx=0).stop 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 -- 172.21.15.67:0/1139237764 shutdown_connections 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 --2- 172.21.15.67:0/1139237764 >> v2:172.21.15.82:3300/0 conn(0x7fd04c159210 0x7fd04c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 --2- 172.21.15.67:0/1139237764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c06e660 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 -- 172.21.15.67:0/1139237764 >> 172.21.15.67:0/1139237764 conn(0x7fd04c0a84a0 msgr2=0x7fd04c0a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 -- 172.21.15.67:0/1139237764 shutdown_connections 2024-02-26T10:42:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.436+0000 7fd054400640 1 -- 172.21.15.67:0/1139237764 wait complete. 2024-02-26T10:42:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.438+0000 7fd054400640 1 Processor -- start 2024-02-26T10:42:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.438+0000 7fd054400640 1 -- start start 2024-02-26T10:42:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd054400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd04c072fe0 0x7fd04c14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd054400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd054400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd04c159210 0x7fd04c14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd054400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c06ec40 con 0x7fd04c156810 2024-02-26T10:42:32.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd054400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15d660 con 0x7fd04c159210 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd054400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c159ba0 con 0x7fd04c072fe0 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd051000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd051000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c0bb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37052/0 (socket says 172.21.15.67:37052) 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd051000640 1 -- 172.21.15.67:0/3081063450 learned_addr learned my addr 172.21.15.67:0/3081063450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd051a00640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd04c072fe0 0x7fd04c14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd051000640 1 -- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd04c072fe0 msgr2=0x7fd04c14e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.439+0000 7fd051000640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd04c072fe0 0x7fd04c14e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd051000640 1 -- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd04c159210 msgr2=0x7fd04c14c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd052400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd04c159210 0x7fd04c14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd051000640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd04c159210 0x7fd04c14c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd051000640 1 -- 172.21.15.67:0/3081063450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c1f2c60 con 0x7fd04c156810 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd052400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd04c159210 0x7fd04c14c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd051000640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c0bb890 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd03800c430 tx=0x7fd03800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038017070 con 0x7fd04c156810 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038005ce0 con 0x7fd04c156810 2024-02-26T10:42:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c14eb30 con 0x7fd04c156810 2024-02-26T10:42:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380072f0 con 0x7fd04c156810 2024-02-26T10:42:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.440+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c1f2180 con 0x7fd04c156810 2024-02-26T10:42:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.441+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c074a30 con 0x7fd04c156810 2024-02-26T10:42:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.442+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd03801a070 con 0x7fd04c156810 2024-02-26T10:42:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.443+0000 7fd049600640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd02c06b500 0x7fd02c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.443+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd038089980 con 0x7fd04c156810 2024-02-26T10:42:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.444+0000 7fd051a00640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd02c06b500 0x7fd02c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:32.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.445+0000 7fd051a00640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd02c06b500 0x7fd02c06d9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd040036000 tx=0x7fd040008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:32.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.448+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd038058490 con 0x7fd04c156810 2024-02-26T10:42:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.710+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.testuser"} v 0) v1 -- 0x7fd04c06e440 con 0x7fd04c156810 2024-02-26T10:42:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.711+0000 7fd049600640 1 -- 172.21.15.67:0/3081063450 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.testuser"}]=0 v19) v1 ==== 85+0+226 (secure 0 0 0) 0x7fd038057d90 con 0x7fd04c156810 2024-02-26T10:42:32.709 INFO:teuthology.orchestra.run.smithi067.stdout:[client.testuser] 2024-02-26T10:42:32.709 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ== 2024-02-26T10:42:32.709 INFO:teuthology.orchestra.run.smithi067.stdout: caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" 2024-02-26T10:42:32.709 INFO:teuthology.orchestra.run.smithi067.stdout: caps mon = "allow r" 2024-02-26T10:42:32.709 INFO:teuthology.orchestra.run.smithi067.stdout: caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd02c06b500 msgr2=0x7fd02c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd02c06b500 0x7fd02c06d9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd040036000 tx=0x7fd040008030 comp rx=0 tx=0).stop 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 msgr2=0x7fd04c0bb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c0bb890 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd03800c430 tx=0x7fd03800c8f0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 shutdown_connections 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd02c06b500 0x7fd02c06d9c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd04c159210 0x7fd04c14c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd04c156810 0x7fd04c0bb890 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 --2- 172.21.15.67:0/3081063450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd04c072fe0 0x7fd04c14e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 >> 172.21.15.67:0/3081063450 conn(0x7fd04c0a84a0 msgr2=0x7fd04c0a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 shutdown_connections 2024-02-26T10:42:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:32.714+0000 7fd054400640 1 -- 172.21.15.67:0/3081063450 wait complete. 2024-02-26T10:42:32.731 INFO:tasks.cephfs.test_multifs_auth:keyring generated for testing is - [client.testuser] key = AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ== caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" caps mon = "allow r" caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-02-26T10:42:32.732 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:42:32.782 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.CCp4ORC6xu 2024-02-26T10:42:32.782 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:32.782 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/tmp/tmp.CCp4ORC6xu 2024-02-26T10:42:32.832 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:32.833 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:32.929 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:32.929 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:32.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:33.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.205+0000 7fab1d600640 1 Processor -- start 2024-02-26T10:42:33.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.206+0000 7fab1d600640 1 -- start start 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab1d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab1806db00 0x7fab180abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab1d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1806e0e0 con 0x7fab180ac230 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab1d600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1806e3c0 con 0x7fab180c2710 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab1d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab18070c50 con 0x7fab1806db00 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab1cc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fab180ac230 0x7fab180716a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47034/0 (socket says 172.21.15.67:47034) 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab1cc00640 1 -- 172.21.15.67:0/2214568166 learned_addr learned my addr 172.21.15.67:0/2214568166 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:33.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.207+0000 7fab0fe00640 1 --2- 172.21.15.67:0/2214568166 >> v2:172.21.15.67:3300/0 conn(0x7fab1806db00 0x7fab180abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.208+0000 7fab17400640 1 -- 172.21.15.67:0/2214568166 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083590487 0 0) 0x7fab1806e0e0 con 0x7fab180ac230 2024-02-26T10:42:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.208+0000 7fab17400640 1 -- 172.21.15.67:0/2214568166 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faafc003520 con 0x7fab180ac230 2024-02-26T10:42:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.208+0000 7fab0fe00640 1 -- 172.21.15.67:0/2214568166 >> v1:172.21.15.82:6789/0 conn(0x7fab180c2710 legacy=0x7fab18151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.208+0000 7fab0fe00640 1 -- 172.21.15.67:0/2214568166 >> v1:172.21.15.67:6789/0 conn(0x7fab180ac230 legacy=0x7fab180716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.208+0000 7fab0fe00640 1 -- 172.21.15.67:0/2214568166 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18070ea0 con 0x7fab1806db00 2024-02-26T10:42:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.209+0000 7fab0fe00640 1 --2- 172.21.15.67:0/2214568166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1806db00 0x7fab180abcf0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fab000097d0 tx=0x7fab0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c87271f0816dcef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.209+0000 7fab17400640 1 -- 172.21.15.67:0/2214568166 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000315a0 con 0x7fab1806db00 2024-02-26T10:42:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.209+0000 7fab17400640 1 -- 172.21.15.67:0/2214568166 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00037070 con 0x7fab1806db00 2024-02-26T10:42:33.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.209+0000 7fab17400640 1 -- 172.21.15.67:0/2214568166 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00031960 con 0x7fab1806db00 2024-02-26T10:42:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 -- 172.21.15.67:0/2214568166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1806db00 msgr2=0x7fab180abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 --2- 172.21.15.67:0/2214568166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1806db00 0x7fab180abcf0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fab000097d0 tx=0x7fab0002b840 comp rx=0 tx=0).stop 2024-02-26T10:42:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 -- 172.21.15.67:0/2214568166 shutdown_connections 2024-02-26T10:42:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 --2- 172.21.15.67:0/2214568166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1806db00 0x7fab180abcf0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 -- 172.21.15.67:0/2214568166 >> 172.21.15.67:0/2214568166 conn(0x7fab180c63f0 msgr2=0x7fab180c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 -- 172.21.15.67:0/2214568166 shutdown_connections 2024-02-26T10:42:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.210+0000 7fab1d600640 1 -- 172.21.15.67:0/2214568166 wait complete. 2024-02-26T10:42:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.212+0000 7fab1d600640 1 Processor -- start 2024-02-26T10:42:33.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.212+0000 7fab1d600640 1 -- start start 2024-02-26T10:42:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1806db00 0x7fab181edb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab180ac230 0x7fab180c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 0x7fab18151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18071db0 con 0x7fab180c2710 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180c2af0 con 0x7fab1806db00 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab18071f30 con 0x7fab180ac230 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 0x7fab18151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 0x7fab18151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37064/0 (socket says 172.21.15.67:37064) 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab1cc00640 1 -- 172.21.15.67:0/2752290826 learned_addr learned my addr 172.21.15.67:0/2752290826 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.213+0000 7fab17e00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1806db00 0x7fab181edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab0fe00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab180ac230 0x7fab180c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1cc00640 1 -- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab180ac230 msgr2=0x7fab180c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1cc00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab180ac230 0x7fab180c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1cc00640 1 -- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1806db00 msgr2=0x7fab181edb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1cc00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1806db00 0x7fab181edb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1cc00640 1 -- 172.21.15.67:0/2752290826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab181ee0a0 con 0x7fab180c2710 2024-02-26T10:42:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab0fe00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab180ac230 0x7fab180c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1cc00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 0x7fab18151560 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fab1000cb40 tx=0x7fab1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10017070 con 0x7fab180c2710 2024-02-26T10:42:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1000aca0 con 0x7fab180c2710 2024-02-26T10:42:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10012620 con 0x7fab180c2710 2024-02-26T10:42:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180c0380 con 0x7fab180c2710 2024-02-26T10:42:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.214+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180bfae0 con 0x7fab180c2710 2024-02-26T10:42:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.215+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18074a30 con 0x7fab180c2710 2024-02-26T10:42:33.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.216+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab1001a070 con 0x7fab180c2710 2024-02-26T10:42:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.217+0000 7fab14c00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faaf406b4d0 0x7faaf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.217+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fab1008a330 con 0x7fab180c2710 2024-02-26T10:42:33.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.218+0000 7fab17e00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faaf406b4d0 0x7faaf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.219+0000 7fab17e00640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faaf406b4d0 0x7faaf406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab08007fd0 tx=0x7fab08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:33.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.221+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab10058ee0 con 0x7fab180c2710 2024-02-26T10:42:33.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.427+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab1806ea20 con 0x7fab180c2710 2024-02-26T10:42:33.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.432+0000 7fab14c00640 1 -- 172.21.15.67:0/2752290826 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7fab100587e0 con 0x7fab180c2710 2024-02-26T10:42:33.430 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:33.430 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faaf406b4d0 msgr2=0x7faaf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faaf406b4d0 0x7faaf406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab08007fd0 tx=0x7fab08009040 comp rx=0 tx=0).stop 2024-02-26T10:42:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 msgr2=0x7fab18151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 0x7fab18151560 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fab1000cb40 tx=0x7fab1000a060 comp rx=0 tx=0).stop 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 shutdown_connections 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab180c2710 0x7fab18151560 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab180ac230 0x7fab180c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7faaf406b4d0 0x7faaf406d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 --2- 172.21.15.67:0/2752290826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1806db00 0x7fab181edb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.434+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 >> 172.21.15.67:0/2752290826 conn(0x7fab180c63f0 msgr2=0x7fab180c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.435+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 shutdown_connections 2024-02-26T10:42:33.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.435+0000 7fab1d600640 1 -- 172.21.15.67:0/2752290826 wait complete. 2024-02-26T10:42:33.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:33.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.681+0000 7f6382a00640 1 Processor -- start 2024-02-26T10:42:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.682+0000 7f6382a00640 1 -- start start 2024-02-26T10:42:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.683+0000 7f6382a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f637c1503d0 0x7f637c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.683+0000 7f6382a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c0a6eb0 con 0x7f637c06e390 2024-02-26T10:42:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.683+0000 7f6382a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c0a7190 con 0x7f637c153af0 2024-02-26T10:42:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.683+0000 7f6382a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0a7470 con 0x7f637c1503d0 2024-02-26T10:42:33.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.683+0000 7f637b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f637c06e390 0x7f637c1533e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47048/0 (socket says 172.21.15.67:47048) 2024-02-26T10:42:33.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.683+0000 7f637b400640 1 -- 172.21.15.67:0/30008030 learned_addr learned my addr 172.21.15.67:0/30008030 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637be00640 1 --2- 172.21.15.67:0/30008030 >> v2:172.21.15.82:3300/0 conn(0x7f637c1503d0 0x7f637c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992628749 0 0) 0x7f637c0a6eb0 con 0x7f637c06e390 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6360003520 con 0x7f637c06e390 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597456562 0 0) 0x7f6360003520 con 0x7f637c06e390 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 >> v2:172.21.15.82:3300/0 conn(0x7f637c1503d0 msgr2=0x7f637c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 --2- 172.21.15.67:0/30008030 >> v2:172.21.15.82:3300/0 conn(0x7f637c1503d0 0x7f637c06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 >> v1:172.21.15.82:6789/0 conn(0x7f637c153af0 legacy=0x7f637c15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0a7750 con 0x7f637c06e390 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f6364001670 con 0x7f637c06e390 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 >> v1:172.21.15.67:6789/0 conn(0x7f637c06e390 legacy=0x7f637c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.684+0000 7f637aa00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63600039b0 0x7f6360003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637aa00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6360004800 0x7f6360004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637aa00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600052e0 0x7f636000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0a7470 con 0x7f63600052e0 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636000c1b0 con 0x7f6360004800 2024-02-26T10:42:33.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636000c3b0 con 0x7f63600039b0 2024-02-26T10:42:33.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f6380a00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600052e0 0x7f636000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637be00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63600039b0 0x7f6360003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f637b400640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6360004800 0x7f6360004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f6380a00640 1 -- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63600039b0 msgr2=0x7f6360003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f6380a00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63600039b0 0x7f6360003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f6380a00640 1 -- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6360004800 msgr2=0x7f6360004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f6380a00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6360004800 0x7f6360004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.685+0000 7f6380a00640 1 -- 172.21.15.67:0/30008030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6360003520 con 0x7f63600052e0 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.686+0000 7f6380a00640 1 -- 172.21.15.67:0/30008030 reap_dead start 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.687+0000 7f6380a00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600052e0 0x7f636000b9c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6374001720 tx=0x7f6360005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d2e42efbd6def server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.687+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374001370 con 0x7f63600052e0 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.687+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63740052d0 con 0x7f63600052e0 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.687+0000 7f637aa00640 1 -- 172.21.15.67:0/30008030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374003810 con 0x7f63600052e0 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.687+0000 7f6382a00640 1 -- 172.21.15.67:0/30008030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600052e0 msgr2=0x7f636000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.687+0000 7f6382a00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600052e0 0x7f636000b9c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6374001720 tx=0x7f6360005830 comp rx=0 tx=0).stop 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.688+0000 7f6382a00640 1 -- 172.21.15.67:0/30008030 shutdown_connections 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.688+0000 7f6382a00640 1 --2- 172.21.15.67:0/30008030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63600052e0 0x7f636000b9c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.688+0000 7f6382a00640 1 -- 172.21.15.67:0/30008030 >> 172.21.15.67:0/30008030 conn(0x7f637c0c6da0 msgr2=0x7f637c0c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:33.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.688+0000 7f6382a00640 1 -- 172.21.15.67:0/30008030 shutdown_connections 2024-02-26T10:42:33.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.688+0000 7f6382a00640 1 -- 172.21.15.67:0/30008030 wait complete. 2024-02-26T10:42:33.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.689+0000 7f6382a00640 1 Processor -- start 2024-02-26T10:42:33.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.690+0000 7f6382a00640 1 -- start start 2024-02-26T10:42:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6382a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63600052e0 0x7f637c148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6382a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f637c1439e0 0x7f637c142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6382a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 0x7f637c142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6382a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1507b0 con 0x7f637c156720 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6382a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c06ef00 con 0x7f63600052e0 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6382a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0c0ef0 con 0x7f637c1439e0 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6380a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 0x7f637c142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6380a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 0x7f637c142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37092/0 (socket says 172.21.15.67:37092) 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f6380a00640 1 -- 172.21.15.67:0/759605856 learned_addr learned my addr 172.21.15.67:0/759605856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f637b400640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f637c1439e0 0x7f637c142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.691+0000 7f637be00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63600052e0 0x7f637c148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6380a00640 1 -- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f637c1439e0 msgr2=0x7f637c142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6380a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f637c1439e0 0x7f637c142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6380a00640 1 -- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63600052e0 msgr2=0x7f637c148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6380a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63600052e0 0x7f637c148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6380a00640 1 -- 172.21.15.67:0/759605856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c154120 con 0x7f637c156720 2024-02-26T10:42:33.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6380a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 0x7f637c142dd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6374001840 tx=0x7f6374001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374007a30 con 0x7f637c156720 2024-02-26T10:42:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6374036070 con 0x7f637c156720 2024-02-26T10:42:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374005480 con 0x7f637c156720 2024-02-26T10:42:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c143790 con 0x7f637c156720 2024-02-26T10:42:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.692+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c072910 con 0x7f637c156720 2024-02-26T10:42:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.694+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c1542f0 con 0x7f637c156720 2024-02-26T10:42:33.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.695+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6374042020 con 0x7f637c156720 2024-02-26T10:42:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.696+0000 7f6373e00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f635806b500 0x7f635806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.696+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f63740ab2a0 con 0x7f637c156720 2024-02-26T10:42:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.697+0000 7f637be00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f635806b500 0x7f635806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.698+0000 7f637be00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f635806b500 0x7f635806d9c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f637c0c6420 tx=0x7f636c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:33.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.701+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6374079e50 con 0x7f637c156720 2024-02-26T10:42:33.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.899+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f637c074a30 con 0x7f637c156720 2024-02-26T10:42:33.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.900+0000 7f6373e00640 1 -- 172.21.15.67:0/759605856 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6374079750 con 0x7f637c156720 2024-02-26T10:42:33.898 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.902+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f635806b500 msgr2=0x7f635806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.902+0000 7f6382a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f635806b500 0x7f635806d9c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f637c0c6420 tx=0x7f636c00b040 comp rx=0 tx=0).stop 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.902+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 msgr2=0x7f637c142dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.902+0000 7f6382a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 0x7f637c142dd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6374001840 tx=0x7f6374001120 comp rx=0 tx=0).stop 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 shutdown_connections 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f637c156720 0x7f637c142dd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f637c1439e0 0x7f637c142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f635806b500 0x7f635806d9c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 --2- 172.21.15.67:0/759605856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63600052e0 0x7f637c148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 >> 172.21.15.67:0/759605856 conn(0x7f637c0c6da0 msgr2=0x7f637c1463c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 shutdown_connections 2024-02-26T10:42:33.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:33.903+0000 7f6382a00640 1 -- 172.21.15.67:0/759605856 wait complete. 2024-02-26T10:42:33.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:34.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.154+0000 7f85ecc00640 1 Processor -- start 2024-02-26T10:42:34.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.154+0000 7f85ecc00640 1 -- start start 2024-02-26T10:42:34.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.155+0000 7f85ecc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85e80a9430 0x7f85e80ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.155+0000 7f85ecc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85e806e2d0 0x7f85e80c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85ecc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e8150780 con 0x7f85e81503a0 2024-02-26T10:42:34.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85ecc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e80a98c0 con 0x7f85e80a9430 2024-02-26T10:42:34.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85ecc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e80a9ba0 con 0x7f85e806e2d0 2024-02-26T10:42:34.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85e80a9430 0x7f85e80ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85e80a9430 0x7f85e80ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37096/0 (socket says 172.21.15.67:37096) 2024-02-26T10:42:34.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85dfe00640 1 -- 172.21.15.67:0/3571643435 learned_addr learned my addr 172.21.15.67:0/3571643435 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:34.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.156+0000 7f85e7e00640 1 --2- 172.21.15.67:0/3571643435 >> v2:172.21.15.82:3300/0 conn(0x7f85e806e2d0 0x7f85e80c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.157+0000 7f85dfe00640 1 -- 172.21.15.67:0/3571643435 >> v2:172.21.15.82:3300/0 conn(0x7f85e806e2d0 msgr2=0x7f85e80c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:34.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.157+0000 7f85dfe00640 1 --2- 172.21.15.67:0/3571643435 >> v2:172.21.15.82:3300/0 conn(0x7f85e806e2d0 0x7f85e80c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.157+0000 7f85dfe00640 1 -- 172.21.15.67:0/3571643435 >> v1:172.21.15.82:6789/0 conn(0x7f85e81503a0 legacy=0x7f85e81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:34.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.157+0000 7f85dfe00640 1 -- 172.21.15.67:0/3571643435 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80a9e80 con 0x7f85e80a9430 2024-02-26T10:42:34.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.158+0000 7f85dfe00640 1 --2- 172.21.15.67:0/3571643435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e80a9430 0x7f85e80ab480 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f85e80c3f40 tx=0x7f85d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee6ae7279f46c8a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:34.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.158+0000 7f85e6a00640 1 -- 172.21.15.67:0/3571643435 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0034070 con 0x7f85e80a9430 2024-02-26T10:42:34.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.158+0000 7f85e6a00640 1 -- 172.21.15.67:0/3571643435 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0030860 con 0x7f85e80a9430 2024-02-26T10:42:34.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.158+0000 7f85e6a00640 1 -- 172.21.15.67:0/3571643435 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0030c30 con 0x7f85e80a9430 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 -- 172.21.15.67:0/3571643435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e80a9430 msgr2=0x7f85e80ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 --2- 172.21.15.67:0/3571643435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e80a9430 0x7f85e80ab480 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f85e80c3f40 tx=0x7f85d002b9a0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 -- 172.21.15.67:0/3571643435 shutdown_connections 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 --2- 172.21.15.67:0/3571643435 >> v2:172.21.15.82:3300/0 conn(0x7f85e806e2d0 0x7f85e80c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 --2- 172.21.15.67:0/3571643435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e80a9430 0x7f85e80ab480 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 -- 172.21.15.67:0/3571643435 >> 172.21.15.67:0/3571643435 conn(0x7f85e80c6380 msgr2=0x7f85e80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 -- 172.21.15.67:0/3571643435 shutdown_connections 2024-02-26T10:42:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.159+0000 7f85ecc00640 1 -- 172.21.15.67:0/3571643435 wait complete. 2024-02-26T10:42:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.161+0000 7f85ecc00640 1 Processor -- start 2024-02-26T10:42:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.161+0000 7f85ecc00640 1 -- start start 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85ecc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e806e2d0 0x7f85e80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85ecc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85e81503a0 0x7f85e814a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85ecc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 0x7f85e81ea940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85ecc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80c0f80 con 0x7f85e8147f70 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85ecc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e806ddb0 con 0x7f85e806e2d0 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85ecc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e806e720 con 0x7f85e81503a0 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 0x7f85e81ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 0x7f85e81ea940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37110/0 (socket says 172.21.15.67:37110) 2024-02-26T10:42:34.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85e7e00640 1 -- 172.21.15.67:0/2437115591 learned_addr learned my addr 172.21.15.67:0/2437115591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.162+0000 7f85e7400640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e806e2d0 0x7f85e80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85dfe00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85e81503a0 0x7f85e814a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7e00640 1 -- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85e81503a0 msgr2=0x7f85e814a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7e00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85e81503a0 0x7f85e814a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7e00640 1 -- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e806e2d0 msgr2=0x7f85e80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7e00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e806e2d0 0x7f85e80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7e00640 1 -- 172.21.15.67:0/2437115591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8071e80 con 0x7f85e8147f70 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7400640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e806e2d0 0x7f85e80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85e7e00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 0x7f85e81ea940 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f85e000efc0 tx=0x7f85e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0018020 con 0x7f85e8147f70 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0014480 con 0x7f85e8147f70 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0014870 con 0x7f85e8147f70 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.163+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e8071a00 con 0x7f85e8147f70 2024-02-26T10:42:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.164+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e8072120 con 0x7f85e8147f70 2024-02-26T10:42:34.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.165+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e8074a30 con 0x7f85e8147f70 2024-02-26T10:42:34.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.166+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85e0005d10 con 0x7f85e8147f70 2024-02-26T10:42:34.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.166+0000 7f85df400640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c406b4d0 0x7f85c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.167+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f85e008b3b0 con 0x7f85e8147f70 2024-02-26T10:42:34.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.167+0000 7f85e7400640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c406b4d0 0x7f85c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.168+0000 7f85e7400640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c406b4d0 0x7f85c406d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f85e81eba70 tx=0x7f85d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:34.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.171+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85e0022030 con 0x7f85e8147f70 2024-02-26T10:42:34.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.369+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85e80c3ad0 con 0x7f85e8147f70 2024-02-26T10:42:34.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.370+0000 7f85df400640 1 -- 172.21.15.67:0/2437115591 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85e0010040 con 0x7f85e8147f70 2024-02-26T10:42:34.368 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c406b4d0 msgr2=0x7f85c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c406b4d0 0x7f85c406d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f85e81eba70 tx=0x7f85d8009040 comp rx=0 tx=0).stop 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 msgr2=0x7f85e81ea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 0x7f85e81ea940 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f85e000efc0 tx=0x7f85e000c490 comp rx=0 tx=0).stop 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 shutdown_connections 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85e8147f70 0x7f85e81ea940 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85e81503a0 0x7f85e814a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f85c406b4d0 0x7f85c406d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 --2- 172.21.15.67:0/2437115591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85e806e2d0 0x7f85e80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 >> 172.21.15.67:0/2437115591 conn(0x7f85e80c6380 msgr2=0x7f85e80c1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:34.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 shutdown_connections 2024-02-26T10:42:34.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.373+0000 7f85ecc00640 1 -- 172.21.15.67:0/2437115591 wait complete. 2024-02-26T10:42:34.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.620+0000 7efcdc000640 1 Processor -- start 2024-02-26T10:42:34.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.620+0000 7efcdc000640 1 -- start start 2024-02-26T10:42:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcdc000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efcd40a6150 0x7efcd40c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcdc000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efcd406e860 0x7efcd40a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcdc000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd4151d70 con 0x7efcd4155170 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcdc000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd4152b10 con 0x7efcd406e860 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcdc000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd4152680 con 0x7efcd40a6150 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcd8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efcd406e860 0x7efcd40a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcd8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efcd406e860 0x7efcd40a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37112/0 (socket says 172.21.15.67:37112) 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.622+0000 7efcd8c00640 1 -- 172.21.15.67:0/2464886440 learned_addr learned my addr 172.21.15.67:0/2464886440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.623+0000 7efcd9600640 1 --2- 172.21.15.67:0/2464886440 >> v2:172.21.15.82:3300/0 conn(0x7efcd40a6150 0x7efcd40c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.623+0000 7efcd8c00640 1 -- 172.21.15.67:0/2464886440 >> v2:172.21.15.82:3300/0 conn(0x7efcd40a6150 msgr2=0x7efcd40c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.623+0000 7efcd8c00640 1 --2- 172.21.15.67:0/2464886440 >> v2:172.21.15.82:3300/0 conn(0x7efcd40a6150 0x7efcd40c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.623+0000 7efcd8c00640 1 -- 172.21.15.67:0/2464886440 >> v1:172.21.15.82:6789/0 conn(0x7efcd4155170 legacy=0x7efcd41555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.623+0000 7efcd8c00640 1 -- 172.21.15.67:0/2464886440 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4152030 con 0x7efcd406e860 2024-02-26T10:42:34.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.624+0000 7efcd8c00640 1 --2- 172.21.15.67:0/2464886440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd406e860 0x7efcd40a6530 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efcd406ef20 tx=0x7efcc402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993f0a9a8ea888e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:34.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.624+0000 7efcd3e00640 1 -- 172.21.15.67:0/2464886440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc402e070 con 0x7efcd406e860 2024-02-26T10:42:34.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcd3e00640 1 -- 172.21.15.67:0/2464886440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc4037070 con 0x7efcd406e860 2024-02-26T10:42:34.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcd3e00640 1 -- 172.21.15.67:0/2464886440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4031a00 con 0x7efcd406e860 2024-02-26T10:42:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 -- 172.21.15.67:0/2464886440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd406e860 msgr2=0x7efcd40a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 --2- 172.21.15.67:0/2464886440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd406e860 0x7efcd40a6530 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efcd406ef20 tx=0x7efcc402b880 comp rx=0 tx=0).stop 2024-02-26T10:42:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 -- 172.21.15.67:0/2464886440 shutdown_connections 2024-02-26T10:42:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 --2- 172.21.15.67:0/2464886440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd406e860 0x7efcd40a6530 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 --2- 172.21.15.67:0/2464886440 >> v2:172.21.15.82:3300/0 conn(0x7efcd40a6150 0x7efcd40c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 -- 172.21.15.67:0/2464886440 >> 172.21.15.67:0/2464886440 conn(0x7efcd40c6da0 msgr2=0x7efcd40c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:34.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 -- 172.21.15.67:0/2464886440 shutdown_connections 2024-02-26T10:42:34.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.625+0000 7efcdc000640 1 -- 172.21.15.67:0/2464886440 wait complete. 2024-02-26T10:42:34.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.627+0000 7efcdc000640 1 Processor -- start 2024-02-26T10:42:34.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.627+0000 7efcdc000640 1 -- start start 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.628+0000 7efcdc000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efcd406e860 0x7efcd4150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.628+0000 7efcdc000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcd40a6150 0x7efcd4143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.628+0000 7efcdc000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 0x7efcd4143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.628+0000 7efcdc000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40c2220 con 0x7efcd4155170 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.628+0000 7efcdc000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4155cf0 con 0x7efcd40a6150 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.628+0000 7efcdc000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd414f3a0 con 0x7efcd406e860 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 0x7efcd4143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 0x7efcd4143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37126/0 (socket says 172.21.15.67:37126) 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 -- 172.21.15.67:0/4130612557 learned_addr learned my addr 172.21.15.67:0/4130612557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcd8c00640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcd40a6150 0x7efcd4143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcd9600640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efcd406e860 0x7efcd4150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 -- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efcd406e860 msgr2=0x7efcd4150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efcd406e860 0x7efcd4150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 -- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcd40a6150 msgr2=0x7efcd4143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcd40a6150 0x7efcd4143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 -- 172.21.15.67:0/4130612557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd41441f0 con 0x7efcd4155170 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcd8c00640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcd40a6150 0x7efcd4143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:34.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.629+0000 7efcda000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 0x7efcd4143cb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b970 tx=0x7efccc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.630+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc019070 con 0x7efcd4155170 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.630+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc00cae0 con 0x7efcd4155170 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.630+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc014440 con 0x7efcd4155170 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.630+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd414ebc0 con 0x7efcd4155170 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.630+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd414e8e0 con 0x7efcd4155170 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.631+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4074a30 con 0x7efcd4155170 2024-02-26T10:42:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.632+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efccc01c070 con 0x7efcd4155170 2024-02-26T10:42:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.633+0000 7efcd1600640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcb806b4d0 0x7efcb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:34.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.633+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7efccc08a1a0 con 0x7efcd4155170 2024-02-26T10:42:34.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.633+0000 7efcd9600640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcb806b4d0 0x7efcb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:34.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.635+0000 7efcd9600640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcb806b4d0 0x7efcb806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006fd0 tx=0x7efcc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.637+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efccc058cb0 con 0x7efcd4155170 2024-02-26T10:42:34.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.839+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcd4073280 con 0x7efcd4155170 2024-02-26T10:42:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.840+0000 7efcd1600640 1 -- 172.21.15.67:0/4130612557 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7efccc0585b0 con 0x7efcd4155170 2024-02-26T10:42:34.838 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcb806b4d0 msgr2=0x7efcb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcb806b4d0 0x7efcb806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efcc8006fd0 tx=0x7efcc8008040 comp rx=0 tx=0).stop 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 msgr2=0x7efcd4143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 0x7efcd4143cb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b970 tx=0x7efccc00be30 comp rx=0 tx=0).stop 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 shutdown_connections 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcd4155170 0x7efcd4143cb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7efcb806b4d0 0x7efcb806d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcd40a6150 0x7efcd4143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 --2- 172.21.15.67:0/4130612557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efcd406e860 0x7efcd4150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 >> 172.21.15.67:0/4130612557 conn(0x7efcd40c6da0 msgr2=0x7efcd40c2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 shutdown_connections 2024-02-26T10:42:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:34.843+0000 7efcdc000640 1 -- 172.21.15.67:0/4130612557 wait complete. 2024-02-26T10:42:34.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:42:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.091+0000 7fa347c00640 1 Processor -- start 2024-02-26T10:42:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.091+0000 7fa347c00640 1 -- start start 2024-02-26T10:42:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.092+0000 7fa347c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 0x7fa34006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.092+0000 7fa347c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400a6eb0 con 0x7fa340153af0 2024-02-26T10:42:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa347c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400a7190 con 0x7fa34006e390 2024-02-26T10:42:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa347c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400a7470 con 0x7fa3401503d0 2024-02-26T10:42:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa345c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa340153af0 0x7fa34015d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47054/0 (socket says 172.21.15.67:47054) 2024-02-26T10:42:35.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa345c00640 1 -- 172.21.15.67:0/3328654297 learned_addr learned my addr 172.21.15.67:0/3328654297 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa345200640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 0x7fa34006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539658219 0 0) 0x7fa3400a6eb0 con 0x7fa340153af0 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.093+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa328003520 con 0x7fa340153af0 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837261397 0 0) 0x7fa328003520 con 0x7fa340153af0 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 msgr2=0x7fa34006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 0x7fa34006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 >> v1:172.21.15.82:6789/0 conn(0x7fa34006e390 legacy=0x7fa3401533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400a7750 con 0x7fa340153af0 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fa3380019f0 con 0x7fa340153af0 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 >> v1:172.21.15.67:6789/0 conn(0x7fa340153af0 legacy=0x7fa34015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 0x7fa3280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328003fb0 0x7fa328004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004d40 0x7fa3280051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400a7470 con 0x7fa328004d40 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32800bc50 con 0x7fa328003fb0 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32800be50 con 0x7fa3401503d0 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.094+0000 7fa345c00640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004d40 0x7fa3280051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.095+0000 7fa345200640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 0x7fa3280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.095+0000 7fa345c00640 1 -- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 msgr2=0x7fa3280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.095+0000 7fa345c00640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3401503d0 0x7fa3280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.095+0000 7fa345c00640 1 -- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328003fb0 msgr2=0x7fa328004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.095+0000 7fa345c00640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328003fb0 0x7fa328004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.095+0000 7fa345c00640 1 -- 172.21.15.67:0/3328654297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa328003520 con 0x7fa328004d40 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.096+0000 7fa345c00640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004d40 0x7fa3280051a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa34006d060 tx=0x7fa328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e764411d318c17 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.096+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33800a390 con 0x7fa328004d40 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.096+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa338034650 con 0x7fa328004d40 2024-02-26T10:42:35.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.096+0000 7fa33f400640 1 -- 172.21.15.67:0/3328654297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338034a80 con 0x7fa328004d40 2024-02-26T10:42:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa347c00640 1 -- 172.21.15.67:0/3328654297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004d40 msgr2=0x7fa3280051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa347c00640 1 --2- 172.21.15.67:0/3328654297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004d40 0x7fa3280051a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa34006d060 tx=0x7fa328005830 comp rx=0 tx=0).stop 2024-02-26T10:42:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa345c00640 1 -- 172.21.15.67:0/3328654297 reap_dead start 2024-02-26T10:42:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa347c00640 1 -- 172.21.15.67:0/3328654297 shutdown_connections 2024-02-26T10:42:35.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa347c00640 1 -- 172.21.15.67:0/3328654297 >> 172.21.15.67:0/3328654297 conn(0x7fa3400c6da0 msgr2=0x7fa3400c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa347c00640 1 -- 172.21.15.67:0/3328654297 shutdown_connections 2024-02-26T10:42:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.097+0000 7fa347c00640 1 -- 172.21.15.67:0/3328654297 wait complete. 2024-02-26T10:42:35.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.099+0000 7fa347c00640 1 Processor -- start 2024-02-26T10:42:35.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.099+0000 7fa347c00640 1 -- start start 2024-02-26T10:42:35.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa347c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328004d40 0x7fa340071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa347c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3400bfb90 0x7fa3401eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa347c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 0x7fa3401495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa347c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3401507b0 con 0x7fa3401eb4b0 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa347c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34006ef00 con 0x7fa328004d40 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa347c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400c0ef0 con 0x7fa3400bfb90 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa345c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 0x7fa3401495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa345c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 0x7fa3401495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37150/0 (socket says 172.21.15.67:37150) 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.100+0000 7fa345c00640 1 -- 172.21.15.67:0/2778308812 learned_addr learned my addr 172.21.15.67:0/2778308812 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345200640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328004d40 0x7fa340071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa33fe00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3400bfb90 0x7fa3401eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345c00640 1 -- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3400bfb90 msgr2=0x7fa3401eaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3400bfb90 0x7fa3401eaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345c00640 1 -- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328004d40 msgr2=0x7fa340071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328004d40 0x7fa340071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345c00640 1 -- 172.21.15.67:0/2778308812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400c0ba0 con 0x7fa3401eb4b0 2024-02-26T10:42:35.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa345c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 0x7fa3401495d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa33800a830 tx=0x7fa3400a7080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.101+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338034800 con 0x7fa3401eb4b0 2024-02-26T10:42:35.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.102+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa338044070 con 0x7fa3401eb4b0 2024-02-26T10:42:35.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.102+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338034ba0 con 0x7fa3401eb4b0 2024-02-26T10:42:35.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.102+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3401ea790 con 0x7fa3401eb4b0 2024-02-26T10:42:35.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.102+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3401edea0 con 0x7fa3401eb4b0 2024-02-26T10:42:35.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.103+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa340074a30 con 0x7fa3401eb4b0 2024-02-26T10:42:35.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.104+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa338049460 con 0x7fa3401eb4b0 2024-02-26T10:42:35.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.105+0000 7fa33cc00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32006b500 0x7fa32006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.105+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa3380acab0 con 0x7fa3401eb4b0 2024-02-26T10:42:35.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.105+0000 7fa345200640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32006b500 0x7fa32006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.106+0000 7fa345200640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32006b500 0x7fa32006d9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa334009a10 tx=0x7fa334008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.110+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa33803e350 con 0x7fa3401eb4b0 2024-02-26T10:42:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.308+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa34006ec50 con 0x7fa3401eb4b0 2024-02-26T10:42:35.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.309+0000 7fa33cc00640 1 -- 172.21.15.67:0/2778308812 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7fa33807b760 con 0x7fa3401eb4b0 2024-02-26T10:42:35.307 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32006b500 msgr2=0x7fa32006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32006b500 0x7fa32006d9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa334009a10 tx=0x7fa334008040 comp rx=0 tx=0).stop 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 msgr2=0x7fa3401495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 0x7fa3401495d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa33800a830 tx=0x7fa3400a7080 comp rx=0 tx=0).stop 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 shutdown_connections 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3401eb4b0 0x7fa3401495d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3400bfb90 0x7fa3401eaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32006b500 0x7fa32006d9c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 --2- 172.21.15.67:0/2778308812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa328004d40 0x7fa340071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 >> 172.21.15.67:0/2778308812 conn(0x7fa3400c6da0 msgr2=0x7fa340151ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 shutdown_connections 2024-02-26T10:42:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.312+0000 7fa347c00640 1 -- 172.21.15.67:0/2778308812 wait complete. 2024-02-26T10:42:35.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:42:35.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.563+0000 7f1fbb200640 1 Processor -- start 2024-02-26T10:42:35.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.563+0000 7f1fbb200640 1 -- start start 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fbb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fb4071710 0x7f1fb40c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fbb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fb406e3d0 0x7f1fb415c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fbb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb415d0b0 con 0x7f1fb415cba0 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fbb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb40c2a80 con 0x7f1fb4071710 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fbb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb415d390 con 0x7f1fb406e3d0 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fb9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fb4071710 0x7f1fb40c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fba800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1fb415cba0 0x7f1fb4156320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47062/0 (socket says 172.21.15.67:47062) 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fba800640 1 -- 172.21.15.67:0/3222011757 learned_addr learned my addr 172.21.15.67:0/3222011757 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:35.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.565+0000 7f1fb9400640 1 --2- 172.21.15.67:0/3222011757 >> v2:172.21.15.82:3300/0 conn(0x7f1fb406e3d0 0x7f1fb415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.566+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3222011757 >> v2:172.21.15.82:3300/0 conn(0x7f1fb406e3d0 msgr2=0x7f1fb415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.566+0000 7f1fb9e00640 1 --2- 172.21.15.67:0/3222011757 >> v2:172.21.15.82:3300/0 conn(0x7f1fb406e3d0 0x7f1fb415c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.566+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3222011757 >> v1:172.21.15.67:6789/0 conn(0x7f1fb415cba0 legacy=0x7f1fb4156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.566+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3222011757 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb406dbd0 con 0x7f1fb4071710 2024-02-26T10:42:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.567+0000 7f1fb9e00640 1 --2- 172.21.15.67:0/3222011757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb4071710 0x7f1fb40c2540 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40c71d0 tx=0x7f1fa802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f2d849e8e16e0a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.567+0000 7f1fb8a00640 1 -- 172.21.15.67:0/3222011757 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80315a0 con 0x7f1fb4071710 2024-02-26T10:42:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.567+0000 7f1fb8a00640 1 -- 172.21.15.67:0/3222011757 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa8037070 con 0x7f1fb4071710 2024-02-26T10:42:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.567+0000 7f1fb8a00640 1 -- 172.21.15.67:0/3222011757 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8031960 con 0x7f1fb4071710 2024-02-26T10:42:35.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 -- 172.21.15.67:0/3222011757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb4071710 msgr2=0x7f1fb40c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3222011757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb4071710 0x7f1fb40c2540 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40c71d0 tx=0x7f1fa802b840 comp rx=0 tx=0).stop 2024-02-26T10:42:35.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 -- 172.21.15.67:0/3222011757 shutdown_connections 2024-02-26T10:42:35.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3222011757 >> v2:172.21.15.82:3300/0 conn(0x7f1fb406e3d0 0x7f1fb415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3222011757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb4071710 0x7f1fb40c2540 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 -- 172.21.15.67:0/3222011757 >> 172.21.15.67:0/3222011757 conn(0x7f1fb40c6380 msgr2=0x7f1fb40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:35.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 -- 172.21.15.67:0/3222011757 shutdown_connections 2024-02-26T10:42:35.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.568+0000 7f1fbb200640 1 -- 172.21.15.67:0/3222011757 wait complete. 2024-02-26T10:42:35.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.570+0000 7f1fbb200640 1 Processor -- start 2024-02-26T10:42:35.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.570+0000 7f1fbb200640 1 -- start start 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.571+0000 7f1fbb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 0x7f1fb415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.571+0000 7f1fbb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fb4071710 0x7f1fb414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.571+0000 7f1fbb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fb415cba0 0x7f1fb414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.571+0000 7f1fbb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4159410 con 0x7f1fb406e3d0 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.571+0000 7f1fbb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4159290 con 0x7f1fb415cba0 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.571+0000 7f1fbb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb406e860 con 0x7f1fb4071710 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 0x7f1fb415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 0x7f1fb415b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37170/0 (socket says 172.21.15.67:37170) 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3167394189 learned_addr learned my addr 172.21.15.67:0/3167394189 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fba800640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fb415cba0 0x7f1fb414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fb4071710 msgr2=0x7f1fb414f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9400640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fb4071710 0x7f1fb414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fb4071710 0x7f1fb414f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fb415cba0 msgr2=0x7f1fb414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fb415cba0 0x7f1fb414fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 -- 172.21.15.67:0/3167394189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb41e4c70 con 0x7f1fb406e3d0 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fba800640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fb415cba0 0x7f1fb414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.572+0000 7f1fb9e00640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 0x7f1fb415b6d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1fa802b880 tx=0x7f1fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.573+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8032640 con 0x7f1fb406e3d0 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.573+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa8032800 con 0x7f1fb406e3d0 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.573+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8032bf0 con 0x7f1fb406e3d0 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.573+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb4150080 con 0x7f1fb406e3d0 2024-02-26T10:42:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.573+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb41e51f0 con 0x7f1fb406e3d0 2024-02-26T10:42:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.574+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb40678e0 con 0x7f1fb406e3d0 2024-02-26T10:42:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.575+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fa804e430 con 0x7f1fb406e3d0 2024-02-26T10:42:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.576+0000 7f1fae000640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f9806b500 0x7f1f9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.576+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1fa802f030 con 0x7f1fb406e3d0 2024-02-26T10:42:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.576+0000 7f1fb9400640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f9806b500 0x7f1f9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.578+0000 7f1fb9400640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f9806b500 0x7f1f9806d9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006fd0 tx=0x7f1fa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.581+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fa8040070 con 0x7f1fb406e3d0 2024-02-26T10:42:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.779+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f1fb4158680 con 0x7f1fb406e3d0 2024-02-26T10:42:35.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.779+0000 7f1fae000640 1 -- 172.21.15.67:0/3167394189 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7f1fa807b000 con 0x7f1fb406e3d0 2024-02-26T10:42:35.777 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f9806b500 msgr2=0x7f1f9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f9806b500 0x7f1f9806d9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006fd0 tx=0x7f1fa4008040 comp rx=0 tx=0).stop 2024-02-26T10:42:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 msgr2=0x7f1fb415b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 0x7f1fb415b6d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1fa802b880 tx=0x7f1fa80046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 shutdown_connections 2024-02-26T10:42:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1f9806b500 0x7f1f9806d9c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fb415cba0 0x7f1fb414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fb4071710 0x7f1fb414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 --2- 172.21.15.67:0/3167394189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb406e3d0 0x7f1fb415b6d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:35.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 >> 172.21.15.67:0/3167394189 conn(0x7f1fb40c6380 msgr2=0x7f1fb40c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:35.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 shutdown_connections 2024-02-26T10:42:35.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:35.782+0000 7f1fbb200640 1 -- 172.21.15.67:0/3167394189 wait complete. 2024-02-26T10:42:35.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:42:36.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.021+0000 7fb079c00640 1 Processor -- start 2024-02-26T10:42:36.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.022+0000 7fb079c00640 1 -- start start 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.023+0000 7fb079c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406ac20 0x7fb0740a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.023+0000 7fb079c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb07406db00 0x7fb0740abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.023+0000 7fb079c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07406b460 con 0x7fb07406b000 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.023+0000 7fb079c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07406dfb0 con 0x7fb07406db00 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.023+0000 7fb079c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740a9a80 con 0x7fb07406ac20 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb07406db00 0x7fb0740abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb07406db00 0x7fb0740abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37186/0 (socket says 172.21.15.67:37186) 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 -- 172.21.15.67:0/1366518632 learned_addr learned my addr 172.21.15.67:0/1366518632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:36.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb073e00640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406ac20 0x7fb0740a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 -- 172.21.15.67:0/1366518632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406ac20 msgr2=0x7fb0740a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:36.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406ac20 0x7fb0740a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 -- 172.21.15.67:0/1366518632 >> v1:172.21.15.82:6789/0 conn(0x7fb07406b000 legacy=0x7fb074153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:36.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb079200640 1 -- 172.21.15.67:0/1366518632 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0740a9d60 con 0x7fb07406db00 2024-02-26T10:42:36.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.024+0000 7fb073e00640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406ac20 0x7fb0740a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.025+0000 7fb079200640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406db00 0x7fb0740abf50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009e50 tx=0x7fb06c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3dabe864ce76ef server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb073400640 1 -- 172.21.15.67:0/1366518632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c02ca80 con 0x7fb07406db00 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb073400640 1 -- 172.21.15.67:0/1366518632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c02cc20 con 0x7fb07406db00 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb073400640 1 -- 172.21.15.67:0/1366518632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0343f0 con 0x7fb07406db00 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb079c00640 1 -- 172.21.15.67:0/1366518632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406db00 msgr2=0x7fb0740abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb079c00640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406db00 0x7fb0740abf50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009e50 tx=0x7fb06c02bee0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb079c00640 1 -- 172.21.15.67:0/1366518632 shutdown_connections 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb079c00640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406db00 0x7fb0740abf50 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb079c00640 1 --2- 172.21.15.67:0/1366518632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406ac20 0x7fb0740a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.026+0000 7fb079c00640 1 -- 172.21.15.67:0/1366518632 >> 172.21.15.67:0/1366518632 conn(0x7fb0740c63f0 msgr2=0x7fb0740c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.027+0000 7fb079c00640 1 -- 172.21.15.67:0/1366518632 shutdown_connections 2024-02-26T10:42:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.027+0000 7fb079c00640 1 -- 172.21.15.67:0/1366518632 wait complete. 2024-02-26T10:42:36.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.028+0000 7fb079c00640 1 Processor -- start 2024-02-26T10:42:36.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.028+0000 7fb079c00640 1 -- start start 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.029+0000 7fb079c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 0x7fb0740c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.029+0000 7fb079c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb07406b000 0x7fb0741ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.029+0000 7fb079c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406db00 0x7fb0741eada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.029+0000 7fb079c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740ac490 con 0x7fb07406ac20 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.029+0000 7fb079c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740aba30 con 0x7fb07406b000 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.029+0000 7fb079c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb074150410 con 0x7fb07406db00 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 0x7fb0740c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 0x7fb0740c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37196/0 (socket says 172.21.15.67:37196) 2024-02-26T10:42:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 -- 172.21.15.67:0/1373104349 learned_addr learned my addr 172.21.15.67:0/1373104349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb06be00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb07406b000 0x7fb0741ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 -- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406db00 msgr2=0x7fb0741eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb079200640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406db00 0x7fb0741eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406db00 0x7fb0741eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 -- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb07406b000 msgr2=0x7fb0741ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb07406b000 0x7fb0741ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 -- 172.21.15.67:0/1373104349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0741eb2e0 con 0x7fb07406ac20 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb06be00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb07406b000 0x7fb0741ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.030+0000 7fb073e00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 0x7fb0740c55b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb06400c7f0 tx=0x7fb06400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.031+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064015070 con 0x7fb07406ac20 2024-02-26T10:42:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.031+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb064007a30 con 0x7fb07406ac20 2024-02-26T10:42:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.031+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064007e00 con 0x7fb07406ac20 2024-02-26T10:42:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.031+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb074143fe0 con 0x7fb07406ac20 2024-02-26T10:42:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.031+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0741ee390 con 0x7fb07406ac20 2024-02-26T10:42:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.032+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb074074a30 con 0x7fb07406ac20 2024-02-26T10:42:36.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.033+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb064018070 con 0x7fb07406ac20 2024-02-26T10:42:36.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.034+0000 7fb070c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb05006b4d0 0x7fb05006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.034+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb06408a570 con 0x7fb07406ac20 2024-02-26T10:42:36.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.035+0000 7fb06be00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb05006b4d0 0x7fb05006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.036+0000 7fb06be00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb05006b4d0 0x7fb05006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0741eb990 tx=0x7fb05c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:36.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.039+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0741eb2e0 con 0x7fb07406ac20 2024-02-26T10:42:36.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.252+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb07414ed00 con 0x7fb07406ac20 2024-02-26T10:42:36.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.252+0000 7fb070c00640 1 -- 172.21.15.67:0/1373104349 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb064058ff0 con 0x7fb07406ac20 2024-02-26T10:42:36.250 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:42:36.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb05006b4d0 msgr2=0x7fb05006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb05006b4d0 0x7fb05006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0741eb990 tx=0x7fb05c009040 comp rx=0 tx=0).stop 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 msgr2=0x7fb0740c55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 0x7fb0740c55b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb06400c7f0 tx=0x7fb06400cbf0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 shutdown_connections 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb07406db00 0x7fb0741eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb05006b4d0 0x7fb05006d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb07406b000 0x7fb0741ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 --2- 172.21.15.67:0/1373104349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb07406ac20 0x7fb0740c55b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.255+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 >> 172.21.15.67:0/1373104349 conn(0x7fb0740c63f0 msgr2=0x7fb0740bf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.256+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 shutdown_connections 2024-02-26T10:42:36.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.256+0000 7fb079c00640 1 -- 172.21.15.67:0/1373104349 wait complete. 2024-02-26T10:42:36.273 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-26T10:42:36.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:36.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:36.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:42:36.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:36.409 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T10:42:36.463 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:36.464 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:36.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:42:36.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:42:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.745+0000 7f7662000640 1 Processor -- start 2024-02-26T10:42:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.745+0000 7f7662000640 1 -- start start 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f7662000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f765c06ac20 0x7f765c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f7662000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f765c0c2310 0x7f765c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f7662000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c152f20 0x7f765c153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f7662000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c155f10 con 0x7f765c0c2310 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f7662000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0c28d0 con 0x7f765c06ac20 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f7662000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c155150 con 0x7f765c152f20 2024-02-26T10:42:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f765be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f765c0c2310 0x7f765c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f765be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f765c0c2310 0x7f765c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37202/0 (socket says 172.21.15.67:37202) 2024-02-26T10:42:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.747+0000 7f765be00640 1 -- 172.21.15.67:0/2880310101 learned_addr learned my addr 172.21.15.67:0/2880310101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f7661600640 1 --2- 172.21.15.67:0/2880310101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c152f20 0x7f765c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f7660c00640 1 --2- 172.21.15.67:0/2880310101 >> v2:172.21.15.82:3300/0 conn(0x7f765c06ac20 0x7f765c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f765be00640 1 -- 172.21.15.67:0/2880310101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c152f20 msgr2=0x7f765c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f765be00640 1 --2- 172.21.15.67:0/2880310101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c152f20 0x7f765c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f765be00640 1 -- 172.21.15.67:0/2880310101 >> v2:172.21.15.82:3300/0 conn(0x7f765c06ac20 msgr2=0x7f765c06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f765be00640 1 --2- 172.21.15.67:0/2880310101 >> v2:172.21.15.82:3300/0 conn(0x7f765c06ac20 0x7f765c06b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.748+0000 7f765be00640 1 -- 172.21.15.67:0/2880310101 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c155430 con 0x7f765c0c2310 2024-02-26T10:42:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.749+0000 7f765be00640 1 --2- 172.21.15.67:0/2880310101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c0c2310 0x7f765c1524b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f765c06ee00 tx=0x7f764802dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4be45d2fcc9aa0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.750+0000 7f765b400640 1 -- 172.21.15.67:0/2880310101 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764803b070 con 0x7f765c0c2310 2024-02-26T10:42:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.750+0000 7f765b400640 1 -- 172.21.15.67:0/2880310101 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764802ed70 con 0x7f765c0c2310 2024-02-26T10:42:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.750+0000 7f765b400640 1 -- 172.21.15.67:0/2880310101 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648036660 con 0x7f765c0c2310 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.750+0000 7f7662000640 1 -- 172.21.15.67:0/2880310101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c0c2310 msgr2=0x7f765c1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.750+0000 7f7662000640 1 --2- 172.21.15.67:0/2880310101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c0c2310 0x7f765c1524b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f765c06ee00 tx=0x7f764802dfc0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 -- 172.21.15.67:0/2880310101 shutdown_connections 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 --2- 172.21.15.67:0/2880310101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c152f20 0x7f765c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 --2- 172.21.15.67:0/2880310101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c0c2310 0x7f765c1524b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 --2- 172.21.15.67:0/2880310101 >> v2:172.21.15.82:3300/0 conn(0x7f765c06ac20 0x7f765c06b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 -- 172.21.15.67:0/2880310101 >> 172.21.15.67:0/2880310101 conn(0x7f765c0c63f0 msgr2=0x7f765c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 -- 172.21.15.67:0/2880310101 shutdown_connections 2024-02-26T10:42:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.751+0000 7f7662000640 1 -- 172.21.15.67:0/2880310101 wait complete. 2024-02-26T10:42:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.752+0000 7f7662000640 1 Processor -- start 2024-02-26T10:42:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 -- start start 2024-02-26T10:42:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 0x7f765c13ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c0c2310 0x7f765c147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f765c152f20 0x7f765c142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c148850 con 0x7f765c06ac20 2024-02-26T10:42:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c1559f0 con 0x7f765c152f20 2024-02-26T10:42:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.753+0000 7f7662000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c1489d0 con 0x7f765c0c2310 2024-02-26T10:42:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 0x7f765c13ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 0x7f765c13ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37218/0 (socket says 172.21.15.67:37218) 2024-02-26T10:42:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 -- 172.21.15.67:0/3336403330 learned_addr learned my addr 172.21.15.67:0/3336403330 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 -- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c0c2310 msgr2=0x7f765c147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c0c2310 0x7f765c147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 -- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f765c152f20 msgr2=0x7f765c142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f765c152f20 0x7f765c142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.754+0000 7f7660c00640 1 -- 172.21.15.67:0/3336403330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c143030 con 0x7f765c06ac20 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.755+0000 7f7660c00640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 0x7f765c13ce40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f765000d830 tx=0x7f765000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.755+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650018030 con 0x7f765c06ac20 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.755+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765000be10 con 0x7f765c06ac20 2024-02-26T10:42:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.755+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500098b0 con 0x7f765c06ac20 2024-02-26T10:42:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.755+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c13f800 con 0x7f765c06ac20 2024-02-26T10:42:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.755+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c1e7290 con 0x7f765c06ac20 2024-02-26T10:42:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.756+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c1480a0 con 0x7f765c06ac20 2024-02-26T10:42:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.757+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f765001b070 con 0x7f765c06ac20 2024-02-26T10:42:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.758+0000 7f7658c00640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f763c06b4d0 0x7f763c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.758+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f765008a7f0 con 0x7f765c06ac20 2024-02-26T10:42:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.759+0000 7f765be00640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f763c06b4d0 0x7f763c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:36.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.760+0000 7f765be00640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f763c06b4d0 0x7f763c06d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7648038000 tx=0x7f7648030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:36.762+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7650059270 con 0x7f765c06ac20 2024-02-26T10:42:37.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.003+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f765c074a30 con 0x7f765c06ac20 2024-02-26T10:42:37.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.004+0000 7f7658c00640 1 -- 172.21.15.67:0/3336403330 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+149 (secure 0 0 0) 0x7f7650058b70 con 0x7f765c06ac20 2024-02-26T10:42:37.002 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:42:37.002 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:42:37.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.006+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f763c06b4d0 msgr2=0x7f763c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.006+0000 7f7662000640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f763c06b4d0 0x7f763c06d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7648038000 tx=0x7f7648030000 comp rx=0 tx=0).stop 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.006+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 msgr2=0x7f765c13ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.006+0000 7f7662000640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 0x7f765c13ce40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f765000d830 tx=0x7f765000dcf0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 shutdown_connections 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f763c06b4d0 0x7f763c06d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f765c152f20 0x7f765c142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f765c0c2310 0x7f765c147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 --2- 172.21.15.67:0/3336403330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f765c06ac20 0x7f765c13ce40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 >> 172.21.15.67:0/3336403330 conn(0x7f765c0c63f0 msgr2=0x7f765c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 shutdown_connections 2024-02-26T10:42:37.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:37.007+0000 7f7662000640 1 -- 172.21.15.67:0/3336403330 wait complete. 2024-02-26T10:42:37.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-26T10:42:37.024 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:42:37.024 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-02-26T10:42:37.025 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-02-26T10:42:37.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:37.025 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-26T10:42:37.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:42:37.025 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.CCp4ORC6xu 2024-02-26T10:42:37.025 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:42:37.076 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.RkVLlYEDIk 2024-02-26T10:42:37.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /tmp/tmp.CCp4ORC6xu /tmp/tmp.RkVLlYEDIk 2024-02-26T10:42:37.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.RkVLlYEDIk 2024-02-26T10:42:37.344 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.RkVLlYEDIk is 226B 2024-02-26T10:42:37.405 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.RkVLlYEDIk 2024-02-26T10:42:37.461 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ== caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" caps mon = "allow r" caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-02-26T10:42:37.462 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:42:37.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:37.462 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-02-26T10:42:37.517 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:42:37.518 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42293sec preferred_lft 42293sec 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::288e:93ff:fe9f:f95d/64 scope link 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 3e:fc:46:71:84:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::3cfc:46ff:fe71:84e3/64 scope link 2024-02-26T10:42:37.519 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:37.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:37.521 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:42:37.610 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-26T10:42:37.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:37.614 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-02-26T10:42:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-02-26T10:42:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:42:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:42:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:42:37.666 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:42:37.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:37.667 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-02-26T10:42:37.668 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-02-26T10:42:37.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:42:37.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:42:37.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:42:37.668 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-02-26T10:42:37.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:37.854 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:42:37.906 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:42:37.907 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:42:37.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:37.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-02-26T10:42:38.005 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:38.005 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:38.063 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-26T10:42:38.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:38.063 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:38.116 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:38.116 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/tmp/tmp.CCp4ORC6xu of=/dev/stdout 2024-02-26T10:42:38.170 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs=/ 2024-02-26T10:42:38.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:38.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:38.323 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,secret=AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:38.323 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:38.323 INFO:teuthology.orchestra.run.smithi067.stdout:keyring.get_secret failed 2024-02-26T10:42:38.360 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,secret=AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:38.360 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:38.360 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:42:38.361 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with new device syntax: testuser@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs=/ 2024-02-26T10:42:38.361 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:42:38.363 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-26T10:42:38.363 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:42:38.363 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:42:38.363 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:42:38.363 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:42:38.369 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:42:38.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:38.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:38.458 INFO:teuthology.orchestra.run.smithi067.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.0': Permission denied 2024-02-26T10:42:38.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:42:38.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:38.465 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:42:38.524 INFO:teuthology.orchestra.run.smithi067.stdout:client5839 2024-02-26T10:42:38.524 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:38.524 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:38.576 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-02-26T10:42:38.632 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.rk0n2Hqx1w 2024-02-26T10:42:38.633 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:38.633 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/tmp/tmp.rk0n2Hqx1w 2024-02-26T10:42:38.686 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:38.686 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:38.782 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:38.782 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:38.833 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.065+0000 7f8803000640 1 Processor -- start 2024-02-26T10:42:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.065+0000 7f8803000640 1 -- start start 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.066+0000 7f8803000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06ac20 0x7f87fc0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.066+0000 7f8803000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87fc06db00 0x7f87fc0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.066+0000 7f8803000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc06b460 con 0x7f87fc06b000 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.066+0000 7f8803000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87fc06dfb0 con 0x7f87fc06db00 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.066+0000 7f8803000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0a9a80 con 0x7f87fc06ac20 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.066+0000 7f8802600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87fc06db00 0x7f87fc0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8802600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87fc06db00 0x7f87fc0abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37224/0 (socket says 172.21.15.67:37224) 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8802600640 1 -- 172.21.15.67:0/882215844 learned_addr learned my addr 172.21.15.67:0/882215844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8801200640 1 --1- 172.21.15.67:0/882215844 >> v1:172.21.15.67:6789/0 conn(0x7f87fc06b000 0x7f87fc153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47076/0 (socket says 172.21.15.67:47076) 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8801c00640 1 --2- 172.21.15.67:0/882215844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06ac20 0x7f87fc0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8802600640 1 -- 172.21.15.67:0/882215844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06ac20 msgr2=0x7f87fc0a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8802600640 1 --2- 172.21.15.67:0/882215844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06ac20 0x7f87fc0a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8802600640 1 -- 172.21.15.67:0/882215844 >> v1:172.21.15.67:6789/0 conn(0x7f87fc06b000 legacy=0x7f87fc153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f8802600640 1 -- 172.21.15.67:0/882215844 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0a9d60 con 0x7f87fc06db00 2024-02-26T10:42:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.067+0000 7f87fbe00640 1 -- 172.21.15.67:0/882215844 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618177695 0 0) 0x7f87fc06b460 con 0x7f87fc06b000 2024-02-26T10:42:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.068+0000 7f8802600640 1 --2- 172.21.15.67:0/882215844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06db00 0x7f87fc0abf50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009e50 tx=0x7f87f402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bef11698905bbf4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.068+0000 7f87fbe00640 1 -- 172.21.15.67:0/882215844 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f402cb00 con 0x7f87fc06db00 2024-02-26T10:42:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f87fbe00640 1 -- 172.21.15.67:0/882215844 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f402cca0 con 0x7f87fc06db00 2024-02-26T10:42:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f87fbe00640 1 -- 172.21.15.67:0/882215844 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4034610 con 0x7f87fc06db00 2024-02-26T10:42:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 -- 172.21.15.67:0/882215844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06db00 msgr2=0x7f87fc0abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 --2- 172.21.15.67:0/882215844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06db00 0x7f87fc0abf50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009e50 tx=0x7f87f402bf60 comp rx=0 tx=0).stop 2024-02-26T10:42:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 -- 172.21.15.67:0/882215844 shutdown_connections 2024-02-26T10:42:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 --2- 172.21.15.67:0/882215844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06db00 0x7f87fc0abf50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 --2- 172.21.15.67:0/882215844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06ac20 0x7f87fc0a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 -- 172.21.15.67:0/882215844 >> 172.21.15.67:0/882215844 conn(0x7f87fc0c63f0 msgr2=0x7f87fc0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 -- 172.21.15.67:0/882215844 shutdown_connections 2024-02-26T10:42:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.069+0000 7f8803000640 1 -- 172.21.15.67:0/882215844 wait complete. 2024-02-26T10:42:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.071+0000 7f8803000640 1 Processor -- start 2024-02-26T10:42:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.071+0000 7f8803000640 1 -- start start 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8803000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 0x7f87fc1dc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8803000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06b000 0x7f87fc1dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8803000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc06db00 0x7f87fc1edd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8803000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0ac490 con 0x7f87fc06ac20 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8803000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0aba30 con 0x7f87fc06db00 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8803000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc150410 con 0x7f87fc06b000 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8801c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 0x7f87fc1dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8801c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 0x7f87fc1dc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37230/0 (socket says 172.21.15.67:37230) 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.072+0000 7f8801c00640 1 -- 172.21.15.67:0/634748610 learned_addr learned my addr 172.21.15.67:0/634748610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801200640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06b000 0x7f87fc1dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801c00640 1 -- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06b000 msgr2=0x7f87fc1dcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8802600640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc06db00 0x7f87fc1edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801c00640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06b000 0x7f87fc1dcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801c00640 1 -- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc06db00 msgr2=0x7f87fc1edd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801c00640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc06db00 0x7f87fc1edd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801c00640 1 -- 172.21.15.67:0/634748610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc1deb40 con 0x7f87fc06ac20 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8801c00640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 0x7f87fc1dc460 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f87f000d800 tx=0x7f87f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000b6c0 con 0x7f87fc06ac20 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0019070 con 0x7f87fc06ac20 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000ba80 con 0x7f87fc06ac20 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc1eb460 con 0x7f87fc06ac20 2024-02-26T10:42:39.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.073+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc1eb740 con 0x7f87fc06ac20 2024-02-26T10:42:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.075+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc074a30 con 0x7f87fc06ac20 2024-02-26T10:42:39.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.075+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87f000bc20 con 0x7f87fc06ac20 2024-02-26T10:42:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.076+0000 7f87f9600640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.076+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f87f008aab0 con 0x7f87fc06ac20 2024-02-26T10:42:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.077+0000 7f8801200640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.078+0000 7f8801200640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87e8007fd0 tx=0x7f87e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.081+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87f0059530 con 0x7f87fc06ac20 2024-02-26T10:42:39.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.283+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87fc06b770 con 0x7f87fc06ac20 2024-02-26T10:42:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.286+0000 7f87f9600640 1 -- 172.21.15.67:0/634748610 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7f87f0058e30 con 0x7f87fc06ac20 2024-02-26T10:42:39.284 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:39.284 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.288+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87dc06b4d0 msgr2=0x7f87dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.288+0000 7f8803000640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87e8007fd0 tx=0x7f87e8009040 comp rx=0 tx=0).stop 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 msgr2=0x7f87fc1dc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 0x7f87fc1dc460 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f87f000d800 tx=0x7f87f000dcc0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 shutdown_connections 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87e8007fd0 tx=0x7f87e8009040 comp rx=0 tx=0).stop 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc06db00 0x7f87fc1edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc06b000 0x7f87fc1dcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 --2- 172.21.15.67:0/634748610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87fc06ac20 0x7f87fc1dc460 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 >> 172.21.15.67:0/634748610 conn(0x7f87fc0c63f0 msgr2=0x7f87fc14b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 shutdown_connections 2024-02-26T10:42:39.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.289+0000 7f8803000640 1 -- 172.21.15.67:0/634748610 wait complete. 2024-02-26T10:42:39.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:39.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:39.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.537+0000 7f53e5800640 1 Processor -- start 2024-02-26T10:42:39.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.537+0000 7f53e5800640 1 -- start start 2024-02-26T10:42:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53e5800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53e01503a0 0x7f53e00ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53e5800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f53e00a9430 0x7f53e01535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53e5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e006e2d0 0x7f53e00c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53e5800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e0153ae0 con 0x7f53e01503a0 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53e5800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e0150780 con 0x7f53e00a9430 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53e5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00a98c0 con 0x7f53e006e2d0 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53e01503a0 0x7f53e00ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53dfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53e01503a0 0x7f53e00ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39414/0 (socket says 172.21.15.67:39414) 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53dfe00640 1 -- 172.21.15.67:0/1938169940 learned_addr learned my addr 172.21.15.67:0/1938169940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.539+0000 7f53df400640 1 --2- 172.21.15.67:0/1938169940 >> v2:172.21.15.82:3300/0 conn(0x7f53e00a9430 0x7f53e01535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53e4e00640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e006e2d0 0x7f53e00c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53dfe00640 1 -- 172.21.15.67:0/1938169940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e006e2d0 msgr2=0x7f53e00c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53dfe00640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e006e2d0 0x7f53e00c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53dfe00640 1 -- 172.21.15.67:0/1938169940 >> v2:172.21.15.82:3300/0 conn(0x7f53e00a9430 msgr2=0x7f53e01535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53dfe00640 1 --2- 172.21.15.67:0/1938169940 >> v2:172.21.15.82:3300/0 conn(0x7f53e00a9430 0x7f53e01535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53dfe00640 1 -- 172.21.15.67:0/1938169940 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00a9ba0 con 0x7f53e01503a0 2024-02-26T10:42:39.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53e4e00640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e006e2d0 0x7f53e00c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.540+0000 7f53df400640 1 --2- 172.21.15.67:0/1938169940 >> v2:172.21.15.82:3300/0 conn(0x7f53e00a9430 0x7f53e01535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.541+0000 7f53dfe00640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e01503a0 0x7f53e00ab480 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f53e00c73e0 tx=0x7f53d002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c527fce6750e8c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.541+0000 7f53dea00640 1 -- 172.21.15.67:0/1938169940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0035020 con 0x7f53e01503a0 2024-02-26T10:42:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.541+0000 7f53dea00640 1 -- 172.21.15.67:0/1938169940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d0031880 con 0x7f53e01503a0 2024-02-26T10:42:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.541+0000 7f53dea00640 1 -- 172.21.15.67:0/1938169940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0031c50 con 0x7f53e01503a0 2024-02-26T10:42:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 -- 172.21.15.67:0/1938169940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e01503a0 msgr2=0x7f53e00ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e01503a0 0x7f53e00ab480 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f53e00c73e0 tx=0x7f53d002b920 comp rx=0 tx=0).stop 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 -- 172.21.15.67:0/1938169940 shutdown_connections 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e006e2d0 0x7f53e00c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 --2- 172.21.15.67:0/1938169940 >> v2:172.21.15.82:3300/0 conn(0x7f53e00a9430 0x7f53e01535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 --2- 172.21.15.67:0/1938169940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e01503a0 0x7f53e00ab480 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 -- 172.21.15.67:0/1938169940 >> 172.21.15.67:0/1938169940 conn(0x7f53e00c6380 msgr2=0x7f53e00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 -- 172.21.15.67:0/1938169940 shutdown_connections 2024-02-26T10:42:39.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.542+0000 7f53e5800640 1 -- 172.21.15.67:0/1938169940 wait complete. 2024-02-26T10:42:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.544+0000 7f53e5800640 1 Processor -- start 2024-02-26T10:42:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.544+0000 7f53e5800640 1 -- start start 2024-02-26T10:42:39.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53e5800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 0x7f53e014dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53e5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e01503a0 0x7f53e00c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53e5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e01de920 0x7f53e0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53e5800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e006ddb0 con 0x7f53e006e2d0 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53e5800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e006dad0 con 0x7f53e01503a0 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53e5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c0f80 con 0x7f53e01de920 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 0x7f53e014dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 0x7f53e014dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39416/0 (socket says 172.21.15.67:39416) 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.545+0000 7f53dfe00640 1 -- 172.21.15.67:0/1488616267 learned_addr learned my addr 172.21.15.67:0/1488616267 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53df400640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e01503a0 0x7f53e00c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53dfe00640 1 -- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e01de920 msgr2=0x7f53e0151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53e4e00640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e01de920 0x7f53e0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53dfe00640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e01de920 0x7f53e0151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53dfe00640 1 -- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e01503a0 msgr2=0x7f53e00c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53dfe00640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e01503a0 0x7f53e00c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53dfe00640 1 -- 172.21.15.67:0/1488616267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e014e1a0 con 0x7f53e006e2d0 2024-02-26T10:42:39.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53df400640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e01503a0 0x7f53e00c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:39.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53dfe00640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 0x7f53e014dbc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f53d002b960 tx=0x7f53d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.546+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0040070 con 0x7f53e006e2d0 2024-02-26T10:42:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.547+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d003a420 con 0x7f53e006e2d0 2024-02-26T10:42:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.547+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0035040 con 0x7f53e006e2d0 2024-02-26T10:42:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.547+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e00c5b30 con 0x7f53e006e2d0 2024-02-26T10:42:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.547+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e0152560 con 0x7f53e006e2d0 2024-02-26T10:42:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.548+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e00678e0 con 0x7f53e006e2d0 2024-02-26T10:42:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.549+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53d004b060 con 0x7f53e006e2d0 2024-02-26T10:42:39.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.550+0000 7f53d7e00640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f53c006b500 0x7f53c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.550+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f53d00ac460 con 0x7f53e006e2d0 2024-02-26T10:42:39.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.550+0000 7f53df400640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f53c006b500 0x7f53c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.551+0000 7f53df400640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f53c006b500 0x7f53c006d9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f53e00bf670 tx=0x7f53cc007530 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.554+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53d007af80 con 0x7f53e006e2d0 2024-02-26T10:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.753+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53e0074a30 con 0x7f53e006e2d0 2024-02-26T10:42:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.754+0000 7f53d7e00640 1 -- 172.21.15.67:0/1488616267 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53d0030e90 con 0x7f53e006e2d0 2024-02-26T10:42:39.751 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f53c006b500 msgr2=0x7f53c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f53c006b500 0x7f53c006d9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f53e00bf670 tx=0x7f53cc007530 comp rx=0 tx=0).stop 2024-02-26T10:42:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 msgr2=0x7f53e014dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 0x7f53e014dbc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f53d002b960 tx=0x7f53d00046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 shutdown_connections 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53e01de920 0x7f53e0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f53c006b500 0x7f53c006d9c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e01503a0 0x7f53e00c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 --2- 172.21.15.67:0/1488616267 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53e006e2d0 0x7f53e014dbc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.756+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 >> 172.21.15.67:0/1488616267 conn(0x7f53e00c6380 msgr2=0x7f53e00c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.757+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 shutdown_connections 2024-02-26T10:42:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.757+0000 7f53e5800640 1 -- 172.21.15.67:0/1488616267 wait complete. 2024-02-26T10:42:39.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.996+0000 7f8584c00640 1 Processor -- start 2024-02-26T10:42:39.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.996+0000 7f8584c00640 1 -- start start 2024-02-26T10:42:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.997+0000 7f8584c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85800c5290 0x7f85801562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.997+0000 7f8584c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85800af8a0 0x7f85800afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.997+0000 7f8584c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858006c110 con 0x7f8580155880 2024-02-26T10:42:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.997+0000 7f8584c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858006d000 con 0x7f85800af8a0 2024-02-26T10:42:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.997+0000 7f8584c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800c72c0 con 0x7f85800c5290 2024-02-26T10:42:39.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f857fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85800af8a0 0x7f85800afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f8577e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8580155880 0x7f858006c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37810/0 (socket says 172.21.15.67:37810) 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f8577e00640 1 -- 172.21.15.67:0/3965432998 learned_addr learned my addr 172.21.15.67:0/3965432998 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f857f400640 1 --2- 172.21.15.67:0/3965432998 >> v2:172.21.15.82:3300/0 conn(0x7f85800c5290 0x7f85801562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f857fe00640 1 -- 172.21.15.67:0/3965432998 >> v2:172.21.15.82:3300/0 conn(0x7f85800c5290 msgr2=0x7f85801562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f857fe00640 1 --2- 172.21.15.67:0/3965432998 >> v2:172.21.15.82:3300/0 conn(0x7f85800c5290 0x7f85801562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f857fe00640 1 -- 172.21.15.67:0/3965432998 >> v1:172.21.15.67:6789/0 conn(0x7f8580155880 legacy=0x7f858006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:39.998+0000 7f857fe00640 1 -- 172.21.15.67:0/3965432998 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800c75a0 con 0x7f85800af8a0 2024-02-26T10:42:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.000+0000 7f857fe00640 1 --2- 172.21.15.67:0/3965432998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85800af8a0 0x7f85800afd30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f85800c7080 tx=0x7f857802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bf2441a2d1ceab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.000+0000 7f857ea00640 1 -- 172.21.15.67:0/3965432998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578039070 con 0x7f85800af8a0 2024-02-26T10:42:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.000+0000 7f857ea00640 1 -- 172.21.15.67:0/3965432998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857802cbe0 con 0x7f85800af8a0 2024-02-26T10:42:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.000+0000 7f857ea00640 1 -- 172.21.15.67:0/3965432998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578034440 con 0x7f85800af8a0 2024-02-26T10:42:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.000+0000 7f8584c00640 1 -- 172.21.15.67:0/3965432998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85800af8a0 msgr2=0x7f85800afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.000+0000 7f8584c00640 1 --2- 172.21.15.67:0/3965432998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85800af8a0 0x7f85800afd30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f85800c7080 tx=0x7f857802be30 comp rx=0 tx=0).stop 2024-02-26T10:42:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.001+0000 7f8584c00640 1 -- 172.21.15.67:0/3965432998 shutdown_connections 2024-02-26T10:42:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.001+0000 7f8584c00640 1 --2- 172.21.15.67:0/3965432998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85800af8a0 0x7f85800afd30 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.001+0000 7f8584c00640 1 --2- 172.21.15.67:0/3965432998 >> v2:172.21.15.82:3300/0 conn(0x7f85800c5290 0x7f85801562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.001+0000 7f8584c00640 1 -- 172.21.15.67:0/3965432998 >> 172.21.15.67:0/3965432998 conn(0x7f85800ac230 msgr2=0x7f85800ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.001+0000 7f8584c00640 1 -- 172.21.15.67:0/3965432998 shutdown_connections 2024-02-26T10:42:39.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.001+0000 7f8584c00640 1 -- 172.21.15.67:0/3965432998 wait complete. 2024-02-26T10:42:40.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.002+0000 7f8584c00640 1 Processor -- start 2024-02-26T10:42:40.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.003+0000 7f8584c00640 1 -- start start 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8584c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85800af8a0 0x7f85800c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8584c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85800c5290 0x7f85800c0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8584c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 0x7f85800c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8584c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8580155cf0 con 0x7f8580155880 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8584c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858006c3d0 con 0x7f85800af8a0 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8584c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85800c5670 con 0x7f85800c5290 2024-02-26T10:42:40.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 0x7f85800c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 0x7f85800c06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39440/0 (socket says 172.21.15.67:39440) 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 -- 172.21.15.67:0/2162347234 learned_addr learned my addr 172.21.15.67:0/2162347234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857f400640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85800af8a0 0x7f85800c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f8577e00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85800c5290 0x7f85800c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 -- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85800c5290 msgr2=0x7f85800c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85800c5290 0x7f85800c0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 -- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85800af8a0 msgr2=0x7f85800c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85800af8a0 0x7f85800c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857fe00640 1 -- 172.21.15.67:0/2162347234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85801eb4e0 con 0x7f8580155880 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.004+0000 7f857f400640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85800af8a0 0x7f85800c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:40.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.005+0000 7f857fe00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 0x7f85800c06d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8578009e80 tx=0x7f8578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.005+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578039050 con 0x7f8580155880 2024-02-26T10:42:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.005+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578045070 con 0x7f8580155880 2024-02-26T10:42:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.005+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85800be180 con 0x7f8580155880 2024-02-26T10:42:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.005+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857803e830 con 0x7f8580155880 2024-02-26T10:42:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.005+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85800c9170 con 0x7f8580155880 2024-02-26T10:42:40.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.007+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8580074a30 con 0x7f8580155880 2024-02-26T10:42:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.007+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f857803e9d0 con 0x7f8580155880 2024-02-26T10:42:40.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.008+0000 7f8577400640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f855c06b4d0 0x7f855c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.009+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f85780ad4a0 con 0x7f8580155880 2024-02-26T10:42:40.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.009+0000 7f857f400640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f855c06b4d0 0x7f855c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.010+0000 7f857f400640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f855c06b4d0 0x7f855c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8570006fd0 tx=0x7f8570008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.013+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f857807c170 con 0x7f8580155880 2024-02-26T10:42:40.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.211+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f858006d950 con 0x7f8580155880 2024-02-26T10:42:40.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.212+0000 7f8577400640 1 -- 172.21.15.67:0/2162347234 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f857807ba70 con 0x7f8580155880 2024-02-26T10:42:40.210 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f855c06b4d0 msgr2=0x7f855c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f855c06b4d0 0x7f855c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8570006fd0 tx=0x7f8570008040 comp rx=0 tx=0).stop 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 msgr2=0x7f85800c06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 0x7f85800c06d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8578009e80 tx=0x7f8578004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 shutdown_connections 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580155880 0x7f85800c06d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85800c5290 0x7f85800c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f855c06b4d0 0x7f855c06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 --2- 172.21.15.67:0/2162347234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85800af8a0 0x7f85800c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 >> 172.21.15.67:0/2162347234 conn(0x7f85800ac230 msgr2=0x7f85800ac610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 shutdown_connections 2024-02-26T10:42:40.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.215+0000 7f8584c00640 1 -- 172.21.15.67:0/2162347234 wait complete. 2024-02-26T10:42:40.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:40.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.457+0000 7f3eee600640 1 Processor -- start 2024-02-26T10:42:40.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.457+0000 7f3eee600640 1 -- start start 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3eee600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee8158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3eee600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3ee80c0ef0 0x7f3ee8070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3eee600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ee8071410 0x7f3ee806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3eee600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee8073010 con 0x7f3ee8071410 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3eee600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee814f490 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3eee600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee8072ce0 con 0x7f3ee81591b0 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3ee7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ee8071410 0x7f3ee806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3ee7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ee8071410 0x7f3ee806de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39452/0 (socket says 172.21.15.67:39452) 2024-02-26T10:42:40.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.459+0000 7f3ee7e00640 1 -- 172.21.15.67:0/1582011438 learned_addr learned my addr 172.21.15.67:0/1582011438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:40.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3ee7400640 1 --2- 172.21.15.67:0/1582011438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee8158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3edea00640 1 --2- 172.21.15.67:0/1582011438 >> v2:172.21.15.82:3300/0 conn(0x7f3ee80c0ef0 0x7f3ee8070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3ee7e00640 1 -- 172.21.15.67:0/1582011438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 msgr2=0x7f3ee8158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:40.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3ee7e00640 1 --2- 172.21.15.67:0/1582011438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee8158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3ee7e00640 1 -- 172.21.15.67:0/1582011438 >> v2:172.21.15.82:3300/0 conn(0x7f3ee80c0ef0 msgr2=0x7f3ee8070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:40.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3ee7e00640 1 --2- 172.21.15.67:0/1582011438 >> v2:172.21.15.82:3300/0 conn(0x7f3ee80c0ef0 0x7f3ee8070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.460+0000 7f3ee7e00640 1 -- 172.21.15.67:0/1582011438 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8155bf0 con 0x7f3ee8071410 2024-02-26T10:42:40.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.461+0000 7f3ee7e00640 1 --2- 172.21.15.67:0/1582011438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8071410 0x7f3ee806de50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3ee806f280 tx=0x7f3ee002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c020e9ac6119e548 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.461+0000 7f3ee6a00640 1 -- 172.21.15.67:0/1582011438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee002c9e0 con 0x7f3ee8071410 2024-02-26T10:42:40.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.461+0000 7f3ee6a00640 1 -- 172.21.15.67:0/1582011438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee002cb80 con 0x7f3ee8071410 2024-02-26T10:42:40.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3ee6a00640 1 -- 172.21.15.67:0/1582011438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00343f0 con 0x7f3ee8071410 2024-02-26T10:42:40.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 -- 172.21.15.67:0/1582011438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8071410 msgr2=0x7f3ee806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 --2- 172.21.15.67:0/1582011438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8071410 0x7f3ee806de50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3ee806f280 tx=0x7f3ee002be40 comp rx=0 tx=0).stop 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 -- 172.21.15.67:0/1582011438 shutdown_connections 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 --2- 172.21.15.67:0/1582011438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8071410 0x7f3ee806de50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 --2- 172.21.15.67:0/1582011438 >> v2:172.21.15.82:3300/0 conn(0x7f3ee80c0ef0 0x7f3ee8070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 --2- 172.21.15.67:0/1582011438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee8158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 -- 172.21.15.67:0/1582011438 >> 172.21.15.67:0/1582011438 conn(0x7f3ee80c6da0 msgr2=0x7f3ee80c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.462+0000 7f3eee600640 1 -- 172.21.15.67:0/1582011438 shutdown_connections 2024-02-26T10:42:40.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.463+0000 7f3eee600640 1 -- 172.21.15.67:0/1582011438 wait complete. 2024-02-26T10:42:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.464+0000 7f3eee600640 1 Processor -- start 2024-02-26T10:42:40.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.464+0000 7f3eee600640 1 -- start start 2024-02-26T10:42:40.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3eee600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee8071410 0x7f3ee80a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3eee600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 0x7f3ee80aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3eee600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3eee600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80a8d20 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3eee600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80a8ba0 con 0x7f3ee8071410 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3eee600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee806e390 con 0x7f3ee81591b0 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3edea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 0x7f3ee80aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.465+0000 7f3edea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 0x7f3ee80aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39454/0 (socket says 172.21.15.67:39454) 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 -- 172.21.15.67:0/3228416639 learned_addr learned my addr 172.21.15.67:0/3228416639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3ee7400640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee8071410 0x7f3ee80a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 -- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 msgr2=0x7f3ee814d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3ee7e00640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee814d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 -- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee8071410 msgr2=0x7f3ee80a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee8071410 0x7f3ee80a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 -- 172.21.15.67:0/3228416639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8150b30 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.466+0000 7f3edea00640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 0x7f3ee80aacd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000d7d0 tx=0x7f3ed000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.467+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0018030 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.467+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0004020 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.467+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0004230 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.467+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee81eea90 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.467+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee814f7b0 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.468+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8074a30 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.469+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ed001b070 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.470+0000 7f3edfe00640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec406b4d0 0x7f3ec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.470+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3ed008a720 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.471+0000 7f3ee7400640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec406b4d0 0x7f3ec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.472+0000 7f3ee7400640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec406b4d0 0x7f3ec406d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8006fd0 tx=0x7f3ed8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.475+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ed0059240 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.672+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ee80a9980 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.673+0000 7f3edfe00640 1 -- 172.21.15.67:0/3228416639 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ed0058b40 con 0x7f3ee80c0ef0 2024-02-26T10:42:40.671 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec406b4d0 msgr2=0x7f3ec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec406b4d0 0x7f3ec406d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8006fd0 tx=0x7f3ed8008040 comp rx=0 tx=0).stop 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 msgr2=0x7f3ee80aacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 0x7f3ee80aacd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000d7d0 tx=0x7f3ed000dc90 comp rx=0 tx=0).stop 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 shutdown_connections 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee81591b0 0x7f3ee814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c0ef0 0x7f3ee80aacd0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec406b4d0 0x7f3ec406d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 --2- 172.21.15.67:0/3228416639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee8071410 0x7f3ee80a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 >> 172.21.15.67:0/3228416639 conn(0x7f3ee80c6da0 msgr2=0x7f3ee80c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 shutdown_connections 2024-02-26T10:42:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.676+0000 7f3eee600640 1 -- 172.21.15.67:0/3228416639 wait complete. 2024-02-26T10:42:40.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:42:40.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.924+0000 7fbaa7200640 1 Processor -- start 2024-02-26T10:42:40.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.925+0000 7fbaa7200640 1 -- start start 2024-02-26T10:42:40.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.926+0000 7fbaa7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbaa8071710 0x7fbaa80c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.926+0000 7fbaa7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa815d0b0 con 0x7fbaa8156320 2024-02-26T10:42:40.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.926+0000 7fbaa7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa815d390 con 0x7fbaa806e3d0 2024-02-26T10:42:40.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.926+0000 7fbaa7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa80c2a80 con 0x7fbaa8071710 2024-02-26T10:42:40.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.927+0000 7fbaa6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbaa8156320 0x7fbaa8156790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37824/0 (socket says 172.21.15.67:37824) 2024-02-26T10:42:40.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.927+0000 7fbaa6800640 1 -- 172.21.15.67:0/3947749346 learned_addr learned my addr 172.21.15.67:0/3947749346 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.927+0000 7fbaa5e00640 1 --2- 172.21.15.67:0/3947749346 >> v2:172.21.15.82:3300/0 conn(0x7fbaa8071710 0x7fbaa80c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.927+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034550912 0 0) 0x7fbaa815d0b0 con 0x7fbaa8156320 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.927+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba8c003520 con 0x7fbaa8156320 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1400011352 0 0) 0x7fba8c003520 con 0x7fbaa8156320 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 >> v2:172.21.15.82:3300/0 conn(0x7fbaa8071710 msgr2=0x7fbaa80c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 --2- 172.21.15.67:0/3947749346 >> v2:172.21.15.82:3300/0 conn(0x7fbaa8071710 0x7fbaa80c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 >> v1:172.21.15.82:6789/0 conn(0x7fbaa806e3d0 legacy=0x7fbaa815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa806dbd0 con 0x7fbaa8156320 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fbaa00019f0 con 0x7fbaa8156320 2024-02-26T10:42:40.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 >> v1:172.21.15.67:6789/0 conn(0x7fbaa8156320 legacy=0x7fbaa8156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c0051c0 0x7fba8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c00c130 con 0x7fba8c004800 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c00c2e0 con 0x7fba8c0039b0 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.928+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c00c4e0 con 0x7fba8c0051c0 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fba9fe00640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fbaa5e00640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fba9fe00640 1 -- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c0051c0 msgr2=0x7fba8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fba9fe00640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c0051c0 0x7fba8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fba9fe00640 1 -- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba8c0039b0 msgr2=0x7fba8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fba9fe00640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fba9fe00640 1 -- 172.21.15.67:0/3947749346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c003520 con 0x7fba8c004800 2024-02-26T10:42:40.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.929+0000 7fbaa6800640 1 -- 172.21.15.67:0/3947749346 reap_dead start 2024-02-26T10:42:40.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.930+0000 7fba9fe00640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fba90001750 tx=0x7fba8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17025ef9dd49bb85 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.930+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90001370 con 0x7fba8c004800 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.930+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba900052d0 con 0x7fba8c004800 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.930+0000 7fbaa5400640 1 -- 172.21.15.67:0/3947749346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90003830 con 0x7fba8c004800 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 -- 172.21.15.67:0/3947749346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c004800 msgr2=0x7fba8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fba90001750 tx=0x7fba8c005830 comp rx=0 tx=0).stop 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 -- 172.21.15.67:0/3947749346 shutdown_connections 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 --2- 172.21.15.67:0/3947749346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 -- 172.21.15.67:0/3947749346 >> 172.21.15.67:0/3947749346 conn(0x7fbaa80c6380 msgr2=0x7fbaa80c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 -- 172.21.15.67:0/3947749346 shutdown_connections 2024-02-26T10:42:40.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.931+0000 7fbaa7200640 1 -- 172.21.15.67:0/3947749346 wait complete. 2024-02-26T10:42:40.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.933+0000 7fbaa7200640 1 Processor -- start 2024-02-26T10:42:40.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.933+0000 7fbaa7200640 1 -- start start 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c004800 0x7fbaa80becf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa80abd50 0x7fbaa81ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 0x7fbaa80bf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8159410 con 0x7fbaa81530a0 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa8159290 con 0x7fbaa80abd50 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa806e860 con 0x7fba8c004800 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 0x7fbaa80bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 0x7fbaa80bf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39472/0 (socket says 172.21.15.67:39472) 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa6800640 1 -- 172.21.15.67:0/4286226561 learned_addr learned my addr 172.21.15.67:0/4286226561 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fbaa5e00640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c004800 0x7fbaa80becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.934+0000 7fba9fe00640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa80abd50 0x7fbaa81ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa6800640 1 -- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c004800 msgr2=0x7fbaa80becf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa6800640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c004800 0x7fbaa80becf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa6800640 1 -- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa80abd50 msgr2=0x7fbaa81ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa6800640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa80abd50 0x7fbaa81ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa6800640 1 -- 172.21.15.67:0/4286226561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa80bf230 con 0x7fbaa81530a0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fba9fe00640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa80abd50 0x7fbaa81ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa6800640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 0x7fbaa80bf930 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00013d0 tx=0x7fbaa000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa000a610 con 0x7fbaa81530a0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa000a7b0 con 0x7fbaa81530a0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa000aba0 con 0x7fbaa81530a0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa80bdea0 con 0x7fbaa81530a0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.935+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa8152c80 con 0x7fbaa81530a0 2024-02-26T10:42:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.937+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa8074a30 con 0x7fbaa81530a0 2024-02-26T10:42:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.938+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaa000ad40 con 0x7fbaa81530a0 2024-02-26T10:42:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.938+0000 7fba9e000640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba8406b4d0 0x7fba8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:40.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.939+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fbaa0089970 con 0x7fbaa81530a0 2024-02-26T10:42:40.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.939+0000 7fbaa5e00640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba8406b4d0 0x7fba8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.940+0000 7fbaa5e00640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba8406b4d0 0x7fba8406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fba98009a90 tx=0x7fba98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:40.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:40.943+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaa001a030 con 0x7fbaa81530a0 2024-02-26T10:42:41.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.138+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbaa80c0480 con 0x7fbaa81530a0 2024-02-26T10:42:41.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.139+0000 7fba9e000640 1 -- 172.21.15.67:0/4286226561 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7fbaa0057ed0 con 0x7fbaa81530a0 2024-02-26T10:42:41.136 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba8406b4d0 msgr2=0x7fba8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba8406b4d0 0x7fba8406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fba98009a90 tx=0x7fba98008040 comp rx=0 tx=0).stop 2024-02-26T10:42:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 msgr2=0x7fbaa80bf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 0x7fbaa80bf930 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00013d0 tx=0x7fbaa000ccd0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 shutdown_connections 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbaa81530a0 0x7fbaa80bf930 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fba8406b4d0 0x7fba8406d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa80abd50 0x7fbaa81ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 --2- 172.21.15.67:0/4286226561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8c004800 0x7fbaa80becf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.141+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 >> 172.21.15.67:0/4286226561 conn(0x7fbaa80c6380 msgr2=0x7fbaa8159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.142+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 shutdown_connections 2024-02-26T10:42:41.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.142+0000 7fbaa7200640 1 -- 172.21.15.67:0/4286226561 wait complete. 2024-02-26T10:42:41.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:42:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.381+0000 7f8a78c00640 1 Processor -- start 2024-02-26T10:42:41.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.381+0000 7f8a78c00640 1 -- start start 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a78c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8a741503d0 0x7f8a7406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a78c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 0x7f8a741533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a78c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740a6eb0 con 0x7f8a740c2950 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a78c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a740a7190 con 0x7f8a741503d0 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a78c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740a7470 con 0x7f8a7406e390 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a72800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8a740c2950 0x7f8a74156320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37834/0 (socket says 172.21.15.67:37834) 2024-02-26T10:42:41.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a72800640 1 -- 172.21.15.67:0/2573496785 learned_addr learned my addr 172.21.15.67:0/2573496785 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:41.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.383+0000 7f8a6be00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 0x7f8a741533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71e00640 1 --2- 172.21.15.67:0/2573496785 >> v2:172.21.15.82:3300/0 conn(0x7f8a741503d0 0x7f8a7406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242614560 0 0) 0x7f8a740a6eb0 con 0x7f8a740c2950 2024-02-26T10:42:41.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a58003520 con 0x7f8a740c2950 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166150510 0 0) 0x7f8a58003520 con 0x7f8a740c2950 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 msgr2=0x7f8a741533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 0x7f8a741533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 >> v2:172.21.15.82:3300/0 conn(0x7f8a741503d0 msgr2=0x7f8a7406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 --2- 172.21.15.67:0/2573496785 >> v2:172.21.15.82:3300/0 conn(0x7f8a741503d0 0x7f8a7406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740a7750 con 0x7f8a740c2950 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f8a6c0019f0 con 0x7f8a740c2950 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 >> v1:172.21.15.67:6789/0 conn(0x7f8a740c2950 legacy=0x7f8a74156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 0x7f8a580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a58004d40 0x7f8a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740a7190 con 0x7f8a58003f80 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740a7470 con 0x7f8a58004d40 2024-02-26T10:42:41.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.384+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5800baf0 con 0x7f8a7406e390 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a6be00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a71e00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 0x7f8a580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a72800640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a58004d40 0x7f8a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a6be00640 1 -- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 msgr2=0x7f8a580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a6be00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a7406e390 0x7f8a580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a6be00640 1 -- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a58004d40 msgr2=0x7f8a580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a6be00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a58004d40 0x7f8a580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a6be00640 1 -- 172.21.15.67:0/2573496785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58003520 con 0x7f8a58003f80 2024-02-26T10:42:41.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.385+0000 7f8a72800640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a58004d40 0x7f8a580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:41.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.386+0000 7f8a6be00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a58004800 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009ac0 tx=0x7f8a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f8e2585dc72090 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c039070 con 0x7f8a58003f80 2024-02-26T10:42:41.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c02c6b0 con 0x7f8a58003f80 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a71400640 1 -- 172.21.15.67:0/2573496785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c009690 con 0x7f8a58003f80 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a78c00640 1 -- 172.21.15.67:0/2573496785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 msgr2=0x7f8a58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a78c00640 1 --2- 172.21.15.67:0/2573496785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a58004800 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009ac0 tx=0x7f8a58005830 comp rx=0 tx=0).stop 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a72800640 1 -- 172.21.15.67:0/2573496785 reap_dead start 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a78c00640 1 -- 172.21.15.67:0/2573496785 shutdown_connections 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.387+0000 7f8a78c00640 1 -- 172.21.15.67:0/2573496785 >> 172.21.15.67:0/2573496785 conn(0x7f8a740c6da0 msgr2=0x7f8a740c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.388+0000 7f8a78c00640 1 -- 172.21.15.67:0/2573496785 shutdown_connections 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.388+0000 7f8a78c00640 1 -- 172.21.15.67:0/2573496785 wait complete. 2024-02-26T10:42:41.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.389+0000 7f8a78c00640 1 Processor -- start 2024-02-26T10:42:41.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.389+0000 7f8a78c00640 1 -- start start 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a78c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a74156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a78c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74155250 0x7f8a7414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a78c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a741e8f10 0x7f8a7414bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a78c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a741507b0 con 0x7f8a58003f80 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a78c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7406ef00 con 0x7f8a74155250 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a78c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740c0ef0 con 0x7f8a741e8f10 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a74156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a74156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39490/0 (socket says 172.21.15.67:39490) 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 -- 172.21.15.67:0/938333934 learned_addr learned my addr 172.21.15.67:0/938333934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 -- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a741e8f10 msgr2=0x7f8a7414bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a741e8f10 0x7f8a7414bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 -- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74155250 msgr2=0x7f8a7414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74155250 0x7f8a7414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.391+0000 7f8a71e00640 1 -- 172.21.15.67:0/938333934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74157010 con 0x7f8a58003f80 2024-02-26T10:42:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.392+0000 7f8a71e00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a74156a10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8a640116d0 tx=0x7f8a64011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.392+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400f9c0 con 0x7f8a58003f80 2024-02-26T10:42:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.392+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6400fb60 con 0x7f8a58003f80 2024-02-26T10:42:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.392+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64015460 con 0x7f8a58003f80 2024-02-26T10:42:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.392+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a741e8430 con 0x7f8a58003f80 2024-02-26T10:42:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.392+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a741e89f0 con 0x7f8a58003f80 2024-02-26T10:42:41.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.393+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a741e8240 con 0x7f8a58003f80 2024-02-26T10:42:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.394+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a64015600 con 0x7f8a58003f80 2024-02-26T10:42:41.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.395+0000 7f8a6a000640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a5006b4d0 0x7f8a5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.395+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8a6408af30 con 0x7f8a58003f80 2024-02-26T10:42:41.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.396+0000 7f8a6be00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a5006b4d0 0x7f8a5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.397+0000 7f8a6be00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a5006b4d0 0x7f8a5006d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c036000 tx=0x7f8a5c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.400+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a64019030 con 0x7f8a58003f80 2024-02-26T10:42:41.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.607+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f8a74074a30 con 0x7f8a58003f80 2024-02-26T10:42:41.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.608+0000 7f8a6a000640 1 -- 172.21.15.67:0/938333934 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7f8a64059ae0 con 0x7f8a58003f80 2024-02-26T10:42:41.606 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a5006b4d0 msgr2=0x7f8a5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a5006b4d0 0x7f8a5006d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c036000 tx=0x7f8a5c02e000 comp rx=0 tx=0).stop 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 msgr2=0x7f8a74156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a74156a10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8a640116d0 tx=0x7f8a64011b90 comp rx=0 tx=0).stop 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 shutdown_connections 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a741e8f10 0x7f8a7414bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8a5006b4d0 0x7f8a5006d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a74155250 0x7f8a7414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 --2- 172.21.15.67:0/938333934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a58003f80 0x7f8a74156a10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 >> 172.21.15.67:0/938333934 conn(0x7f8a740c6da0 msgr2=0x7f8a74151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 shutdown_connections 2024-02-26T10:42:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.611+0000 7f8a78c00640 1 -- 172.21.15.67:0/938333934 wait complete. 2024-02-26T10:42:41.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:42:41.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.844+0000 7f44e7200640 1 Processor -- start 2024-02-26T10:42:41.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.844+0000 7f44e7200640 1 -- start start 2024-02-26T10:42:41.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.845+0000 7f44e7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e8152380 0x7f44e80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.845+0000 7f44e7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44e80c7020 0x7f44e8155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.845+0000 7f44e7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44e806d290 0x7f44e80c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.845+0000 7f44e7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e8152760 con 0x7f44e80c7020 2024-02-26T10:42:41.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.845+0000 7f44e7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e8155cd0 con 0x7f44e806d290 2024-02-26T10:42:41.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.845+0000 7f44e7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80c7550 con 0x7f44e8152380 2024-02-26T10:42:41.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44e80c7020 0x7f44e8155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44e80c7020 0x7f44e8155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39506/0 (socket says 172.21.15.67:39506) 2024-02-26T10:42:41.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 -- 172.21.15.67:0/3938309834 learned_addr learned my addr 172.21.15.67:0/3938309834 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e6800640 1 --2- 172.21.15.67:0/3938309834 >> v2:172.21.15.82:3300/0 conn(0x7f44e806d290 0x7f44e80c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5e00640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e8152380 0x7f44e80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 -- 172.21.15.67:0/3938309834 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e8152380 msgr2=0x7f44e80c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e8152380 0x7f44e80c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 -- 172.21.15.67:0/3938309834 >> v2:172.21.15.82:3300/0 conn(0x7f44e806d290 msgr2=0x7f44e80c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 --2- 172.21.15.67:0/3938309834 >> v2:172.21.15.82:3300/0 conn(0x7f44e806d290 0x7f44e80c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5400640 1 -- 172.21.15.67:0/3938309834 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80c7830 con 0x7f44e80c7020 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.846+0000 7f44e5e00640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e8152380 0x7f44e80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.847+0000 7f44e5400640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e80c7020 0x7f44e8155790 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f44e806dd70 tx=0x7f44d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b490fe50641874e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.848+0000 7f44e4a00640 1 -- 172.21.15.67:0/3938309834 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d402e070 con 0x7f44e80c7020 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.848+0000 7f44e4a00640 1 -- 172.21.15.67:0/3938309834 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4037070 con 0x7f44e80c7020 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.848+0000 7f44e4a00640 1 -- 172.21.15.67:0/3938309834 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4031a00 con 0x7f44e80c7020 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 -- 172.21.15.67:0/3938309834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e80c7020 msgr2=0x7f44e8155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e80c7020 0x7f44e8155790 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f44e806dd70 tx=0x7f44d402b880 comp rx=0 tx=0).stop 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 -- 172.21.15.67:0/3938309834 shutdown_connections 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 --2- 172.21.15.67:0/3938309834 >> v2:172.21.15.82:3300/0 conn(0x7f44e806d290 0x7f44e80c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e80c7020 0x7f44e8155790 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 --2- 172.21.15.67:0/3938309834 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e8152380 0x7f44e80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 -- 172.21.15.67:0/3938309834 >> 172.21.15.67:0/3938309834 conn(0x7f44e80ac1a0 msgr2=0x7f44e80ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 -- 172.21.15.67:0/3938309834 shutdown_connections 2024-02-26T10:42:41.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.849+0000 7f44e7200640 1 -- 172.21.15.67:0/3938309834 wait complete. 2024-02-26T10:42:41.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.850+0000 7f44e7200640 1 Processor -- start 2024-02-26T10:42:41.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 -- start start 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 0x7f44e8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80c7020 0x7f44e80bf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e8152380 0x7f44e81483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e806d720 con 0x7f44e806d290 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e806cd70 con 0x7f44e8152380 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.851+0000 7f44e7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e806d8a0 con 0x7f44e80c7020 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 0x7f44e8154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 0x7f44e8154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39518/0 (socket says 172.21.15.67:39518) 2024-02-26T10:42:41.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 -- 172.21.15.67:0/1576683730 learned_addr learned my addr 172.21.15.67:0/1576683730 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5400640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80c7020 0x7f44e80bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e6800640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e8152380 0x7f44e81483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 -- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80c7020 msgr2=0x7f44e80bf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80c7020 0x7f44e80bf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 -- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e8152380 msgr2=0x7f44e81483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e8152380 0x7f44e81483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5e00640 1 -- 172.21.15.67:0/1576683730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80c4540 con 0x7f44e806d290 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.852+0000 7f44e5400640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80c7020 0x7f44e80bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:41.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.853+0000 7f44e5e00640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 0x7f44e8154110 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c7f0 tx=0x7f44d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.853+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8015070 con 0x7f44e806d290 2024-02-26T10:42:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.853+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d8007a30 con 0x7f44e806d290 2024-02-26T10:42:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.853+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81eea20 con 0x7f44e806d290 2024-02-26T10:42:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.853+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8007e00 con 0x7f44e806d290 2024-02-26T10:42:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.853+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e8153910 con 0x7f44e806d290 2024-02-26T10:42:41.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.854+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e8074a30 con 0x7f44e806d290 2024-02-26T10:42:41.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.855+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44d8018070 con 0x7f44e806d290 2024-02-26T10:42:41.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.856+0000 7f44de000640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f44c806b4d0 0x7f44c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:41.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.856+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f44d808a4d0 con 0x7f44e806d290 2024-02-26T10:42:41.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.857+0000 7f44e5400640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f44c806b4d0 0x7f44c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:41.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.858+0000 7f44e5400640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f44c806b4d0 0x7f44c806d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f44d4036000 tx=0x7f44d4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:41.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:41.861+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44e80c4540 con 0x7f44e806d290 2024-02-26T10:42:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.074+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44e80c7c50 con 0x7f44e806d290 2024-02-26T10:42:42.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.075+0000 7f44de000640 1 -- 172.21.15.67:0/1576683730 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44d8058ff0 con 0x7f44e806d290 2024-02-26T10:42:42.073 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f44c806b4d0 msgr2=0x7f44c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f44c806b4d0 0x7f44c806d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f44d4036000 tx=0x7f44d4008030 comp rx=0 tx=0).stop 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 msgr2=0x7f44e8154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 0x7f44e8154110 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c7f0 tx=0x7f44d800cbf0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 shutdown_connections 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f44c806b4d0 0x7f44c806d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e8152380 0x7f44e81483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80c7020 0x7f44e80bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 --2- 172.21.15.67:0/1576683730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44e806d290 0x7f44e8154110 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 >> 172.21.15.67:0/1576683730 conn(0x7f44e80ac1a0 msgr2=0x7f44e80ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 shutdown_connections 2024-02-26T10:42:42.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.078+0000 7f44e7200640 1 -- 172.21.15.67:0/1576683730 wait complete. 2024-02-26T10:42:42.095 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-26T10:42:42.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:42.095 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:42.249 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:42:42.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:42.250 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:42:42.294 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:42.294 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:42.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:42:42.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-26T10:42:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.576+0000 7f2b5de00640 1 Processor -- start 2024-02-26T10:42:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.576+0000 7f2b5de00640 1 -- start start 2024-02-26T10:42:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 0x7f2b5806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b5de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b58071700 0x7f2b58071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b5de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5806ed10 con 0x7f2b580c2870 2024-02-26T10:42:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b5de00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5806dd80 con 0x7f2b58071700 2024-02-26T10:42:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b5de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58155c00 con 0x7f2b5814f3d0 2024-02-26T10:42:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b4e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2b580c2870 0x7f2b58155160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37842/0 (socket says 172.21.15.67:37842) 2024-02-26T10:42:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.578+0000 7f2b4e000640 1 -- 172.21.15.67:0/184961930 learned_addr learned my addr 172.21.15.67:0/184961930 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56a00640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 0x7f2b5806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b57400640 1 --2- 172.21.15.67:0/184961930 >> v2:172.21.15.82:3300/0 conn(0x7f2b58071700 0x7f2b58071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231217234 0 0) 0x7f2b5806ed10 con 0x7f2b580c2870 2024-02-26T10:42:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c003520 con 0x7f2b580c2870 2024-02-26T10:42:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3583798080 0 0) 0x7f2b3c003520 con 0x7f2b580c2870 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 msgr2=0x7f2b5806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 0x7f2b5806e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 >> v2:172.21.15.82:3300/0 conn(0x7f2b58071700 msgr2=0x7f2b58071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 --2- 172.21.15.67:0/184961930 >> v2:172.21.15.82:3300/0 conn(0x7f2b58071700 0x7f2b58071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58155ee0 con 0x7f2b580c2870 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f2b40001670 con 0x7f2b580c2870 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 >> v1:172.21.15.67:6789/0 conn(0x7f2b580c2870 legacy=0x7f2b58155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.579+0000 7f2b56000640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 0x7f2b3c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b56000640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c003f80 0x7f2b3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b56000640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b3c004d40 0x7f2b3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5806dd80 con 0x7f2b3c004d40 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58155c00 con 0x7f2b3c003f80 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00dc80 con 0x7f2b5814f3d0 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b57400640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b3c004d40 0x7f2b3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b56a00640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 0x7f2b3c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b4e000640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c003f80 0x7f2b3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b57400640 1 -- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 msgr2=0x7f2b3c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b57400640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b5814f3d0 0x7f2b3c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b57400640 1 -- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c003f80 msgr2=0x7f2b3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b57400640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c003f80 0x7f2b3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b57400640 1 -- 172.21.15.67:0/184961930 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c003520 con 0x7f2b3c004d40 2024-02-26T10:42:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.580+0000 7f2b4e000640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b3c003f80 0x7f2b3c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.582+0000 7f2b57400640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b3c004d40 0x7f2b3c0051c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009ac0 tx=0x7f2b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37706b268d5468cb server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.582+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50039070 con 0x7f2b3c004d40 2024-02-26T10:42:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.582+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5002c710 con 0x7f2b3c004d40 2024-02-26T10:42:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.582+0000 7f2b56000640 1 -- 172.21.15.67:0/184961930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50009690 con 0x7f2b3c004d40 2024-02-26T10:42:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.582+0000 7f2b5de00640 1 -- 172.21.15.67:0/184961930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b3c004d40 msgr2=0x7f2b3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.582+0000 7f2b5de00640 1 --2- 172.21.15.67:0/184961930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b3c004d40 0x7f2b3c0051c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009ac0 tx=0x7f2b3c005830 comp rx=0 tx=0).stop 2024-02-26T10:42:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.583+0000 7f2b57400640 1 -- 172.21.15.67:0/184961930 reap_dead start 2024-02-26T10:42:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.583+0000 7f2b5de00640 1 -- 172.21.15.67:0/184961930 shutdown_connections 2024-02-26T10:42:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.583+0000 7f2b5de00640 1 -- 172.21.15.67:0/184961930 >> 172.21.15.67:0/184961930 conn(0x7f2b580a84a0 msgr2=0x7f2b580a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.583+0000 7f2b5de00640 1 -- 172.21.15.67:0/184961930 shutdown_connections 2024-02-26T10:42:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.583+0000 7f2b5de00640 1 -- 172.21.15.67:0/184961930 wait complete. 2024-02-26T10:42:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.585+0000 7f2b5de00640 1 Processor -- start 2024-02-26T10:42:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.585+0000 7f2b5de00640 1 -- start start 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.586+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b3c004d40 0x7f2b5814e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.586+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b58145d20 0x7f2b580bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.586+0000 7f2b5de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 0x7f2b58143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.586+0000 7f2b5de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580c1810 con 0x7f2b58143350 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.586+0000 7f2b5de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5806e2b0 con 0x7f2b58145d20 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.586+0000 7f2b5de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58155870 con 0x7f2b3c004d40 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 0x7f2b58143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 0x7f2b58143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39544/0 (socket says 172.21.15.67:39544) 2024-02-26T10:42:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 -- 172.21.15.67:0/2052693420 learned_addr learned my addr 172.21.15.67:0/2052693420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b4e000640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b58145d20 0x7f2b580bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b56a00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b3c004d40 0x7f2b5814e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 -- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b3c004d40 msgr2=0x7f2b5814e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b3c004d40 0x7f2b5814e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 -- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b58145d20 msgr2=0x7f2b580bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b58145d20 0x7f2b580bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.587+0000 7f2b57400640 1 -- 172.21.15.67:0/2052693420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b581ea660 con 0x7f2b58143350 2024-02-26T10:42:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.588+0000 7f2b57400640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 0x7f2b58143800 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2b5002c380 tx=0x7f2b50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.588+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50039070 con 0x7f2b58143350 2024-02-26T10:42:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.588+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50043070 con 0x7f2b58143350 2024-02-26T10:42:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.588+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50009690 con 0x7f2b58143350 2024-02-26T10:42:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.588+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b58154f50 con 0x7f2b58143350 2024-02-26T10:42:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.588+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5814d9c0 con 0x7f2b58143350 2024-02-26T10:42:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.589+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b580c13f0 con 0x7f2b58143350 2024-02-26T10:42:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.590+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b50003a50 con 0x7f2b58143350 2024-02-26T10:42:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.591+0000 7f2b4fe00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2b3406b500 0x7f2b3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.591+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f2b500ab580 con 0x7f2b58143350 2024-02-26T10:42:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.592+0000 7f2b56a00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2b3406b500 0x7f2b3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:42.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.593+0000 7f2b56a00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2b3406b500 0x7f2b3406d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b48007520 tx=0x7f2b4800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:42.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.595+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b5004c060 con 0x7f2b58143350 2024-02-26T10:42:42.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.838+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b580742b0 con 0x7f2b58143350 2024-02-26T10:42:42.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.839+0000 7f2b4fe00640 1 -- 172.21.15.67:0/2052693420 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+149 (secure 0 0 0) 0x7f2b5007a090 con 0x7f2b58143350 2024-02-26T10:42:42.837 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-26T10:42:42.837 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2b3406b500 msgr2=0x7f2b3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2b3406b500 0x7f2b3406d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b48007520 tx=0x7f2b4800b040 comp rx=0 tx=0).stop 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 msgr2=0x7f2b58143800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 0x7f2b58143800 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2b5002c380 tx=0x7f2b50004730 comp rx=0 tx=0).stop 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 shutdown_connections 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b58143350 0x7f2b58143800 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2b3406b500 0x7f2b3406d9c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b58145d20 0x7f2b580bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 --2- 172.21.15.67:0/2052693420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2b3c004d40 0x7f2b5814e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 >> 172.21.15.67:0/2052693420 conn(0x7f2b580a84a0 msgr2=0x7f2b580c2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 shutdown_connections 2024-02-26T10:42:42.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:42.843+0000 7f2b5de00640 1 -- 172.21.15.67:0/2052693420 wait complete. 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-26T10:42:42.859 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.rk0n2Hqx1w 2024-02-26T10:42:42.859 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-02-26T10:42:42.918 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.05z58hzNOi 2024-02-26T10:42:42.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /tmp/tmp.rk0n2Hqx1w /tmp/tmp.05z58hzNOi 2024-02-26T10:42:43.015 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.05z58hzNOi 2024-02-26T10:42:43.166 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.05z58hzNOi is 226B 2024-02-26T10:42:43.226 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.05z58hzNOi 2024-02-26T10:42:43.274 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ== caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" caps mon = "allow r" caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-02-26T10:42:43.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:43.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.274 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-02-26T10:42:43.324 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-26T10:42:43.324 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-26T10:42:43.324 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-02-26T10:42:43.324 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-26T10:42:43.325 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42115sec preferred_lft 42115sec 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::6c35:fbff:fe3d:ca30/64 scope link 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 46:9d:5e:fe:41:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::449d:5eff:fefe:410c/64 scope link 2024-02-26T10:42:43.326 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-02-26T10:42:43.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.327 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-26T10:42:43.413 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-02-26T10:42:43.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.417 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-02-26T10:42:43.467 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-02-26T10:42:43.467 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-26T10:42:43.468 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-26T10:42:43.468 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-26T10:42:43.468 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-26T10:42:43.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.469 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-02-26T10:42:43.469 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-02-26T10:42:43.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-26T10:42:43.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-26T10:42:43.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-26T10:42:43.469 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-02-26T10:42:43.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.648 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:42:43.696 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:42:43.697 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:43.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.697 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-02-26T10:42:43.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:43.786 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:43.838 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-26T10:42:43.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-26T10:42:43.838 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:43.887 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:43.887 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/tmp/tmp.rk0n2Hqx1w of=/dev/stdout 2024-02-26T10:42:43.937 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs2=/ 2024-02-26T10:42:43.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:43.937 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,secret=AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,secret=AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:keyring.get_secret failed 2024-02-26T10:42:44.107 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,secret=AQAYa9xl4yTuChAASi9nPgwg5u/5Ko+4aybUcQ==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-02-26T10:42:44.107 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-02-26T10:42:44.107 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-02-26T10:42:44.107 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: testuser@f4a3a702-2ce8-46d3-b32d-e4f6872b8b58.cephfs2=/ 2024-02-26T10:42:44.107 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.67:3300/172.21.15.82:3300/172.21.15.82:3301" will pass to kernel 2024-02-26T10:42:44.109 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-26T10:42:44.109 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-02-26T10:42:44.109 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-26T10:42:44.109 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-02-26T10:42:44.109 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-26T10:42:44.113 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-26T10:42:44.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:44.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:44.190 INFO:teuthology.orchestra.run.smithi082.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.1': Permission denied 2024-02-26T10:42:44.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:42:44.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:44.194 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-26T10:42:44.252 INFO:teuthology.orchestra.run.smithi082.stdout:client5459 2024-02-26T10:42:44.252 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:44.252 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:44.303 INFO:tasks.cephfs.caps_helper:test read perm: read file /home/ubuntu/cephtest/mnt.0/testdir/testfile and expect data "self.path = /home/ubuntu/cephtest/mnt.0/testdir/testfile cephfs_name = cephfs cephfs_mntpt = / hostfs_mntpt = /home/ubuntu/cephtest/mnt.0" 2024-02-26T10:42:44.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:44.304 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec cat /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-02-26T10:42:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:cat: /home/ubuntu/cephtest/mnt.0/testdir/testfile: Permission denied 2024-02-26T10:42:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:42:44.367 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:44.367 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:44.425 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:44.425 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:44.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:44.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.711+0000 7fb80a400640 1 Processor -- start 2024-02-26T10:42:44.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.711+0000 7fb80a400640 1 -- start start 2024-02-26T10:42:44.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.713+0000 7fb80a400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb804072a30 0x7fb804155580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.713+0000 7fb80a400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040c0a40 con 0x7fb804072ea0 2024-02-26T10:42:44.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.713+0000 7fb80a400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb804155ac0 con 0x7fb80406ae90 2024-02-26T10:42:44.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.713+0000 7fb80a400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb804073330 con 0x7fb804072a30 2024-02-26T10:42:44.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.713+0000 7fb7faa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb804072ea0 0x7fb804154e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37858/0 (socket says 172.21.15.67:37858) 2024-02-26T10:42:44.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.713+0000 7fb7faa00640 1 -- 172.21.15.67:0/204957913 learned_addr learned my addr 172.21.15.67:0/204957913 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb803e00640 1 --2- 172.21.15.67:0/204957913 >> v2:172.21.15.82:3300/0 conn(0x7fb804072a30 0x7fb804155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356900443 0 0) 0x7fb8040c0a40 con 0x7fb804072ea0 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e8003520 con 0x7fb804072ea0 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274566404 0 0) 0x7fb7e8003520 con 0x7fb804072ea0 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 >> v2:172.21.15.82:3300/0 conn(0x7fb804072a30 msgr2=0x7fb804155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 --2- 172.21.15.67:0/204957913 >> v2:172.21.15.82:3300/0 conn(0x7fb804072a30 0x7fb804155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 >> v1:172.21.15.82:6789/0 conn(0x7fb80406ae90 legacy=0x7fb8041521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040c7010 con 0x7fb804072ea0 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fb7ec001690 con 0x7fb804072ea0 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 >> v1:172.21.15.67:6789/0 conn(0x7fb804072ea0 legacy=0x7fb804154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e8003a00 0x7fb7e8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.714+0000 7fb802a00640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8004800 0x7fb7e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb802a00640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8005230 0x7fb7e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804073330 con 0x7fb7e8003a00 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e800c190 con 0x7fb7e8005230 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e800c390 con 0x7fb7e8004800 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803400640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e8003a00 0x7fb7e8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803e00640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8005230 0x7fb7e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803400640 1 -- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8004800 msgr2=0x7fb7e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803400640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7e8004800 0x7fb7e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803400640 1 -- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8005230 msgr2=0x7fb7e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803400640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8005230 0x7fb7e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803400640 1 -- 172.21.15.67:0/204957913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8003520 con 0x7fb7e8003a00 2024-02-26T10:42:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.715+0000 7fb803e00640 1 -- 172.21.15.67:0/204957913 reap_dead start 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.716+0000 7fb803400640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e8003a00 0x7fb7e8003e40 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4001e50 tx=0x7fb7e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3b4bcfbef176f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4001a70 con 0x7fb7e8003a00 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4003b70 con 0x7fb7e8003a00 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb802a00640 1 -- 172.21.15.67:0/204957913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40050d0 con 0x7fb7e8003a00 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb80a400640 1 -- 172.21.15.67:0/204957913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e8003a00 msgr2=0x7fb7e8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb80a400640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e8003a00 0x7fb7e8003e40 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4001e50 tx=0x7fb7e8005830 comp rx=0 tx=0).stop 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb80a400640 1 -- 172.21.15.67:0/204957913 shutdown_connections 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.717+0000 7fb80a400640 1 --2- 172.21.15.67:0/204957913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e8003a00 0x7fb7e8003e40 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.718+0000 7fb80a400640 1 -- 172.21.15.67:0/204957913 >> 172.21.15.67:0/204957913 conn(0x7fb8040c7c40 msgr2=0x7fb80406bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.718+0000 7fb80a400640 1 -- 172.21.15.67:0/204957913 shutdown_connections 2024-02-26T10:42:44.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.718+0000 7fb80a400640 1 -- 172.21.15.67:0/204957913 wait complete. 2024-02-26T10:42:44.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.719+0000 7fb80a400640 1 Processor -- start 2024-02-26T10:42:44.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 -- start start 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003a00 0x7fb80414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 0x7fb8041ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb80413fcb0 0x7fb804140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80406b270 con 0x7fb8040aa6c0 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040c0d70 con 0x7fb7e8003a00 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.720+0000 7fb80a400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80406b3f0 con 0x7fb80413fcb0 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 0x7fb8041ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 0x7fb8041ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39570/0 (socket says 172.21.15.67:39570) 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 -- 172.21.15.67:0/2509978121 learned_addr learned my addr 172.21.15.67:0/2509978121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb803400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003a00 0x7fb80414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb803e00640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb80413fcb0 0x7fb804140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 -- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb80413fcb0 msgr2=0x7fb804140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb80413fcb0 0x7fb804140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 -- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003a00 msgr2=0x7fb80414e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003a00 0x7fb80414e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 -- 172.21.15.67:0/2509978121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804151300 con 0x7fb8040aa6c0 2024-02-26T10:42:44.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.721+0000 7fb7faa00640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 0x7fb8041ee170 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c700 tx=0x7fb7ec00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.722+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec018070 con 0x7fb8040aa6c0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.722+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec0072f0 con 0x7fb8040aa6c0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.722+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb804155190 con 0x7fb8040aa6c0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.722+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0047e0 con 0x7fb8040aa6c0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.722+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb804155430 con 0x7fb8040aa6c0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.723+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8040c6430 con 0x7fb8040aa6c0 2024-02-26T10:42:44.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.724+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7ec01b070 con 0x7fb8040aa6c0 2024-02-26T10:42:44.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.725+0000 7fb7fbe00640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7e006b500 0x7fb7e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:44.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.725+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb7ec0898e0 con 0x7fb8040aa6c0 2024-02-26T10:42:44.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.725+0000 7fb803400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7e006b500 0x7fb7e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:44.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.726+0000 7fb803400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7e006b500 0x7fb7e006d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4034000 tx=0x7fb7f4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:44.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.729+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8040c6430 con 0x7fb8040aa6c0 2024-02-26T10:42:44.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.929+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb804074a30 con 0x7fb8040aa6c0 2024-02-26T10:42:44.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.933+0000 7fb7fbe00640 1 -- 172.21.15.67:0/2509978121 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7fb7ec0583f0 con 0x7fb8040aa6c0 2024-02-26T10:42:44.931 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:44.931 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.935+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7e006b500 msgr2=0x7fb7e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.935+0000 7fb80a400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7e006b500 0x7fb7e006d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4034000 tx=0x7fb7f4009030 comp rx=0 tx=0).stop 2024-02-26T10:42:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.935+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 msgr2=0x7fb8041ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.935+0000 7fb80a400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 0x7fb8041ee170 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c700 tx=0x7fb7ec00cbc0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 shutdown_connections 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb80413fcb0 0x7fb804140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8040aa6c0 0x7fb8041ee170 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb7e006b500 0x7fb7e006d9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 --2- 172.21.15.67:0/2509978121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003a00 0x7fb80414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 >> 172.21.15.67:0/2509978121 conn(0x7fb8040c7c40 msgr2=0x7fb8040a88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 shutdown_connections 2024-02-26T10:42:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:44.936+0000 7fb80a400640 1 -- 172.21.15.67:0/2509978121 wait complete. 2024-02-26T10:42:44.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:44.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:45.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.184+0000 7f25f2600640 1 Processor -- start 2024-02-26T10:42:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.184+0000 7f25f2600640 1 -- start start 2024-02-26T10:42:45.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25f2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25ec154cf0 0x7f25ec0c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25f2600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec0c3840 con 0x7f25ec071fd0 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25f2600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ec06dc80 con 0x7f25ec1523b0 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25f2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec152790 con 0x7f25ec154cf0 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25ec154cf0 0x7f25ec0c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25f1c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f25ec071fd0 0x7f25ec0723e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37870/0 (socket says 172.21.15.67:37870) 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25ec154cf0 0x7f25ec0c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39574/0 (socket says 172.21.15.67:39574) 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25f1c00640 1 -- 172.21.15.67:0/2461598278 learned_addr learned my addr 172.21.15.67:0/2461598278 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25ebe00640 1 -- 172.21.15.67:0/2461598278 >> v1:172.21.15.82:6789/0 conn(0x7f25ec1523b0 legacy=0x7f25ec0c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25ebe00640 1 -- 172.21.15.67:0/2461598278 >> v1:172.21.15.67:6789/0 conn(0x7f25ec071fd0 legacy=0x7f25ec0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25ebe00640 1 -- 172.21.15.67:0/2461598278 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec06df60 con 0x7f25ec154cf0 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.186+0000 7f25eb400640 1 -- 172.21.15.67:0/2461598278 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988711522 0 0) 0x7f25ec0c3840 con 0x7f25ec071fd0 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.188+0000 7f25ebe00640 1 --2- 172.21.15.67:0/2461598278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec154cf0 0x7f25ec0c3300 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f25d80097a0 tx=0x7f25d802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96583b359bcfe3b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.188+0000 7f25eb400640 1 -- 172.21.15.67:0/2461598278 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8038070 con 0x7f25ec154cf0 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.188+0000 7f25eb400640 1 -- 172.21.15.67:0/2461598278 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8031700 con 0x7f25ec154cf0 2024-02-26T10:42:45.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.188+0000 7f25eb400640 1 -- 172.21.15.67:0/2461598278 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8031ad0 con 0x7f25ec154cf0 2024-02-26T10:42:45.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.188+0000 7f25f2600640 1 -- 172.21.15.67:0/2461598278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec154cf0 msgr2=0x7f25ec0c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.188+0000 7f25f2600640 1 --2- 172.21.15.67:0/2461598278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec154cf0 0x7f25ec0c3300 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f25d80097a0 tx=0x7f25d802b7a0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.189+0000 7f25f2600640 1 -- 172.21.15.67:0/2461598278 shutdown_connections 2024-02-26T10:42:45.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.189+0000 7f25f2600640 1 --2- 172.21.15.67:0/2461598278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec154cf0 0x7f25ec0c3300 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.189+0000 7f25f2600640 1 -- 172.21.15.67:0/2461598278 >> 172.21.15.67:0/2461598278 conn(0x7f25ec0c6380 msgr2=0x7f25ec0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:45.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.189+0000 7f25f2600640 1 -- 172.21.15.67:0/2461598278 shutdown_connections 2024-02-26T10:42:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.189+0000 7f25f2600640 1 -- 172.21.15.67:0/2461598278 wait complete. 2024-02-26T10:42:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.190+0000 7f25f2600640 1 Processor -- start 2024-02-26T10:42:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.190+0000 7f25f2600640 1 -- start start 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.191+0000 7f25f2600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec071fd0 0x7f25ec1eea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.191+0000 7f25f2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 0x7f25ec1ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.191+0000 7f25f2600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec154cf0 0x7f25ec14f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.191+0000 7f25f2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec06e520 con 0x7f25ec1523b0 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.191+0000 7f25f2600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec06da30 con 0x7f25ec071fd0 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.191+0000 7f25f2600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec06e800 con 0x7f25ec154cf0 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 0x7f25ec1ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 0x7f25ec1ebfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39590/0 (socket says 172.21.15.67:39590) 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 -- 172.21.15.67:0/921908373 learned_addr learned my addr 172.21.15.67:0/921908373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 -- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec154cf0 msgr2=0x7f25ec14f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec154cf0 0x7f25ec14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25f1200640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec071fd0 0x7f25ec1eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 -- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec071fd0 msgr2=0x7f25ec1eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec071fd0 0x7f25ec1eea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 -- 172.21.15.67:0/921908373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec0c5aa0 con 0x7f25ec1523b0 2024-02-26T10:42:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25f1200640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec071fd0 0x7f25ec1eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.192+0000 7f25ebe00640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 0x7f25ec1ebfd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f25d80097a0 tx=0x7f25d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.193+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8043070 con 0x7f25ec1523b0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.193+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8033a30 con 0x7f25ec1523b0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.193+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8038040 con 0x7f25ec1523b0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.193+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec0c5d80 con 0x7f25ec1523b0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.193+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec1eab60 con 0x7f25ec1523b0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.194+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec074a30 con 0x7f25ec1523b0 2024-02-26T10:42:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.195+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25d8046070 con 0x7f25ec1523b0 2024-02-26T10:42:45.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.196+0000 7f25e8c00640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25cc06b4d0 0x7f25cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.196+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f25d80ac200 con 0x7f25ec1523b0 2024-02-26T10:42:45.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.196+0000 7f25f1200640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25cc06b4d0 0x7f25cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.197+0000 7f25f1200640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25cc06b4d0 0x7f25cc06d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f25ec1e13c0 tx=0x7f25e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:45.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.200+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25d8030070 con 0x7f25ec1523b0 2024-02-26T10:42:45.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.400+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25ec0bf840 con 0x7f25ec1523b0 2024-02-26T10:42:45.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.400+0000 7f25e8c00640 1 -- 172.21.15.67:0/921908373 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25d807ad10 con 0x7f25ec1523b0 2024-02-26T10:42:45.398 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:45.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25cc06b4d0 msgr2=0x7f25cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25cc06b4d0 0x7f25cc06d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f25ec1e13c0 tx=0x7f25e0009040 comp rx=0 tx=0).stop 2024-02-26T10:42:45.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 msgr2=0x7f25ec1ebfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 0x7f25ec1ebfd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f25d80097a0 tx=0x7f25d80046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 shutdown_connections 2024-02-26T10:42:45.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec154cf0 0x7f25ec14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec1523b0 0x7f25ec1ebfd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f25cc06b4d0 0x7f25cc06d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 --2- 172.21.15.67:0/921908373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec071fd0 0x7f25ec1eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 >> 172.21.15.67:0/921908373 conn(0x7f25ec0c6380 msgr2=0x7f25ec0723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:45.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.403+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 shutdown_connections 2024-02-26T10:42:45.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.404+0000 7f25f2600640 1 -- 172.21.15.67:0/921908373 wait complete. 2024-02-26T10:42:45.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.652+0000 7f3ee9c00640 1 Processor -- start 2024-02-26T10:42:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.652+0000 7f3ee9c00640 1 -- start start 2024-02-26T10:42:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.653+0000 7f3ee9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 0x7f3ee40c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee9c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee4158d20 con 0x7f3ee4155430 2024-02-26T10:42:45.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee9c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee40a6450 con 0x7f3ee40a5cb0 2024-02-26T10:42:45.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4072bd0 con 0x7f3ee40bfed0 2024-02-26T10:42:45.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3ee4155430 0x7f3ee415f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37882/0 (socket says 172.21.15.67:37882) 2024-02-26T10:42:45.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee9200640 1 -- 172.21.15.67:0/2492307436 learned_addr learned my addr 172.21.15.67:0/2492307436 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee3e00640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 0x7f3ee40c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154476237 0 0) 0x7f3ee4158d20 con 0x7f3ee4155430 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.654+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec8003520 con 0x7f3ee4155430 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651748457 0 0) 0x7f3ec8003520 con 0x7f3ee4155430 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 msgr2=0x7f3ee40c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 0x7f3ee40c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 >> v1:172.21.15.82:6789/0 conn(0x7f3ee40a5cb0 legacy=0x7f3ee4154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4158fe0 con 0x7f3ee4155430 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f3edc001a10 con 0x7f3ee4155430 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 >> v1:172.21.15.67:6789/0 conn(0x7f3ee4155430 legacy=0x7f3ee415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 0x7f3ec80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ec8003fb0 0x7f3ec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec800bad0 con 0x7f3ec8004d40 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec800bc80 con 0x7f3ec8003fb0 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec800be80 con 0x7f3ee40bfed0 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.655+0000 7f3ee9200640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3ee3e00640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 0x7f3ec80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3ee9200640 1 -- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 msgr2=0x7f3ec80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3edbe00640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ec8003fb0 0x7f3ec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3ee9200640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee40bfed0 0x7f3ec80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3ee9200640 1 -- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ec8003fb0 msgr2=0x7f3ec8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3ee9200640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ec8003fb0 0x7f3ec8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3ee9200640 1 -- 172.21.15.67:0/2492307436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8003520 con 0x7f3ec8004d40 2024-02-26T10:42:45.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.656+0000 7f3edbe00640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ec8003fb0 0x7f3ec8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:45.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.657+0000 7f3ee9200640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3ee406fbd0 tx=0x7f3ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9006e69429fec20f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:45.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.657+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00a3b0 con 0x7f3ec8004d40 2024-02-26T10:42:45.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.657+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc034630 con 0x7f3ec8004d40 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.657+0000 7f3ee3400640 1 -- 172.21.15.67:0/2492307436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc034a60 con 0x7f3ec8004d40 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9c00640 1 -- 172.21.15.67:0/2492307436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ec8004d40 msgr2=0x7f3ec80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/2492307436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3ee406fbd0 tx=0x7f3ec8005830 comp rx=0 tx=0).stop 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9200640 1 -- 172.21.15.67:0/2492307436 reap_dead start 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9c00640 1 -- 172.21.15.67:0/2492307436 shutdown_connections 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9c00640 1 -- 172.21.15.67:0/2492307436 >> 172.21.15.67:0/2492307436 conn(0x7f3ee40a9360 msgr2=0x7f3ee40c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9c00640 1 -- 172.21.15.67:0/2492307436 shutdown_connections 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.658+0000 7f3ee9c00640 1 -- 172.21.15.67:0/2492307436 wait complete. 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.660+0000 7f3ee9c00640 1 Processor -- start 2024-02-26T10:42:45.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.660+0000 7f3ee9c00640 1 -- start start 2024-02-26T10:42:45.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ec8004d40 0x7f3ee41e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee41e1520 0x7f3ee41debe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 0x7f3ee41edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee40a6200 con 0x7f3ee41df120 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee406c730 con 0x7f3ee41e1520 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee4073040 con 0x7f3ec8004d40 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 0x7f3ee41edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 0x7f3ee41edc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39594/0 (socket says 172.21.15.67:39594) 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.661+0000 7f3ee9200640 1 -- 172.21.15.67:0/4077255197 learned_addr learned my addr 172.21.15.67:0/4077255197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee3e00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ec8004d40 0x7f3ee41e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3edbe00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee41e1520 0x7f3ee41debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee9200640 1 -- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ec8004d40 msgr2=0x7f3ee41e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee9200640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ec8004d40 0x7f3ee41e0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee9200640 1 -- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee41e1520 msgr2=0x7f3ee41debe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee9200640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee41e1520 0x7f3ee41debe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee9200640 1 -- 172.21.15.67:0/4077255197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee41de420 con 0x7f3ee41df120 2024-02-26T10:42:45.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.662+0000 7f3ee9200640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 0x7f3ee41edc30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00a820 tx=0x7f3ee40a6140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:45.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.663+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc001f00 con 0x7f3ee41df120 2024-02-26T10:42:45.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.663+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc044070 con 0x7f3ee41df120 2024-02-26T10:42:45.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.663+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0349e0 con 0x7f3ee41df120 2024-02-26T10:42:45.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.663+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee4152140 con 0x7f3ee41df120 2024-02-26T10:42:45.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.663+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee4066a90 con 0x7f3ee41df120 2024-02-26T10:42:45.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.664+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee40714a0 con 0x7f3ee41df120 2024-02-26T10:42:45.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.665+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3edc034b80 con 0x7f3ee41df120 2024-02-26T10:42:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.666+0000 7f3ee0c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec006b500 0x7f3ec006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:45.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.666+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3edc0aca30 con 0x7f3ee41df120 2024-02-26T10:42:45.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.666+0000 7f3ee3e00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec006b500 0x7f3ec006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:45.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.668+0000 7f3ee3e00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec006b500 0x7f3ec006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009a10 tx=0x7f3ed4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:45.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.671+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ee40714a0 con 0x7f3ee41df120 2024-02-26T10:42:45.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.872+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ee4074a30 con 0x7f3ee41df120 2024-02-26T10:42:45.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.873+0000 7f3ee0c00640 1 -- 172.21.15.67:0/4077255197 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3edc07b5d0 con 0x7f3ee41df120 2024-02-26T10:42:45.871 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:45.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec006b500 msgr2=0x7f3ec006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec006b500 0x7f3ec006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009a10 tx=0x7f3ed4008040 comp rx=0 tx=0).stop 2024-02-26T10:42:45.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 msgr2=0x7f3ee41edc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:45.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 0x7f3ee41edc30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00a820 tx=0x7f3ee40a6140 comp rx=0 tx=0).stop 2024-02-26T10:42:45.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 shutdown_connections 2024-02-26T10:42:45.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee41df120 0x7f3ee41edc30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec006b500 0x7f3ec006d9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee41e1520 0x7f3ee41debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 --2- 172.21.15.67:0/4077255197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ec8004d40 0x7f3ee41e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.876+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 >> 172.21.15.67:0/4077255197 conn(0x7f3ee40a9360 msgr2=0x7f3ee40c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.877+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 shutdown_connections 2024-02-26T10:42:45.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:45.877+0000 7f3ee9c00640 1 -- 172.21.15.67:0/4077255197 wait complete. 2024-02-26T10:42:45.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.124+0000 7f77db200640 1 Processor -- start 2024-02-26T10:42:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.124+0000 7f77db200640 1 -- start start 2024-02-26T10:42:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.126+0000 7f77db200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 0x7f77dc151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.126+0000 7f77db200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0c1be0 con 0x7f77dc0c25a0 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.126+0000 7f77db200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0c3660 con 0x7f77dc0c1eb0 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.126+0000 7f77db200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c1750 con 0x7f77dc151980 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.126+0000 7f77d9e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f77dc0c25a0 0x7f77dc0c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37892/0 (socket says 172.21.15.67:37892) 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d9e00640 1 -- 172.21.15.67:0/628290801 learned_addr learned my addr 172.21.15.67:0/628290801 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77da800640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 0x7f77dc151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270122284 0 0) 0x7f77dc0c1be0 con 0x7f77dc0c25a0 2024-02-26T10:42:46.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c0003520 con 0x7f77dc0c25a0 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2259309097 0 0) 0x7f77c0003520 con 0x7f77dc0c25a0 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 msgr2=0x7f77dc151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 0x7f77dc151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 >> v1:172.21.15.82:6789/0 conn(0x7f77dc0c1eb0 legacy=0x7f77dc14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0c0e20 con 0x7f77dc0c25a0 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.127+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f77cc001690 con 0x7f77dc0c25a0 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 >> v1:172.21.15.67:6789/0 conn(0x7f77dc0c25a0 legacy=0x7f77dc0c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 0x7f77c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003f60 0x7f77c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77c0004d40 0x7f77c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c1750 con 0x7f77c0003f60 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000dd60 con 0x7f77c0004d40 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000df60 con 0x7f77dc151980 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9400640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003f60 0x7f77c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77da800640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77c0004d40 0x7f77c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9e00640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 0x7f77c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9400640 1 -- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 msgr2=0x7f77c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9400640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77dc151980 0x7f77c00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9400640 1 -- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77c0004d40 msgr2=0x7f77c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9400640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77c0004d40 0x7f77c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77d9400640 1 -- 172.21.15.67:0/628290801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c0003520 con 0x7f77c0003f60 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.128+0000 7f77da800640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77c0004d40 0x7f77c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77d9400640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003f60 0x7f77c0004800 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001720 tx=0x7f77c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573884e32257f2ae server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8001370 con 0x7f77c0003f60 2024-02-26T10:42:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c80052d0 con 0x7f77c0003f60 2024-02-26T10:42:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77d8a00640 1 -- 172.21.15.67:0/628290801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8003830 con 0x7f77c0003f60 2024-02-26T10:42:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77db200640 1 -- 172.21.15.67:0/628290801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003f60 msgr2=0x7f77c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77db200640 1 --2- 172.21.15.67:0/628290801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c0003f60 0x7f77c0004800 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001720 tx=0x7f77c0005830 comp rx=0 tx=0).stop 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.130+0000 7f77da800640 1 -- 172.21.15.67:0/628290801 reap_dead start 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.131+0000 7f77db200640 1 -- 172.21.15.67:0/628290801 shutdown_connections 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.131+0000 7f77db200640 1 -- 172.21.15.67:0/628290801 >> 172.21.15.67:0/628290801 conn(0x7f77dc0c6380 msgr2=0x7f77dc0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.131+0000 7f77db200640 1 -- 172.21.15.67:0/628290801 shutdown_connections 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.131+0000 7f77db200640 1 -- 172.21.15.67:0/628290801 wait complete. 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.132+0000 7f77db200640 1 Processor -- start 2024-02-26T10:42:46.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.133+0000 7f77db200640 1 -- start start 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77db200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77c0003f60 0x7f77dc1eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77db200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 0x7f77dc0c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77db200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc146b40 0x7f77dc146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77db200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c2980 con 0x7f77dc14f370 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77db200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c13e0 con 0x7f77dc146b40 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77db200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c2b00 con 0x7f77c0003f60 2024-02-26T10:42:46.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 0x7f77dc0c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 0x7f77dc0c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39610/0 (socket says 172.21.15.67:39610) 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 -- 172.21.15.67:0/2837540168 learned_addr learned my addr 172.21.15.67:0/2837540168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9e00640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77c0003f60 0x7f77dc1eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77da800640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc146b40 0x7f77dc146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 -- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77c0003f60 msgr2=0x7f77dc1eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77c0003f60 0x7f77dc1eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 -- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc146b40 msgr2=0x7f77dc146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc146b40 0x7f77dc146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77d9400640 1 -- 172.21.15.67:0/2837540168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc1453c0 con 0x7f77dc14f370 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.134+0000 7f77da800640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc146b40 0x7f77dc146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.135+0000 7f77d9400640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 0x7f77dc0c5230 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001840 tx=0x7f77c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.135+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8041070 con 0x7f77dc14f370 2024-02-26T10:42:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.135+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8005480 con 0x7f77dc14f370 2024-02-26T10:42:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.135+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc150280 con 0x7f77dc14f370 2024-02-26T10:42:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.135+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8007a30 con 0x7f77dc14f370 2024-02-26T10:42:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.135+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc14f9e0 con 0x7f77dc14f370 2024-02-26T10:42:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.136+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc0678e0 con 0x7f77dc14f370 2024-02-26T10:42:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.137+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77c8044070 con 0x7f77dc14f370 2024-02-26T10:42:46.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.139+0000 7f77d2000640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77bc06b4d0 0x7f77bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.139+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f77c80ac100 con 0x7f77dc14f370 2024-02-26T10:42:46.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.139+0000 7f77d9e00640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77bc06b4d0 0x7f77bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.140+0000 7f77d9e00640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77bc06b4d0 0x7f77bc06d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c3b80 tx=0x7f77cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.143+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77c807ac10 con 0x7f77dc14f370 2024-02-26T10:42:46.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.347+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77dc14be00 con 0x7f77dc14f370 2024-02-26T10:42:46.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.348+0000 7f77d2000640 1 -- 172.21.15.67:0/2837540168 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77dc14be00 con 0x7f77dc14f370 2024-02-26T10:42:46.346 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:46.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77bc06b4d0 msgr2=0x7f77bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77bc06b4d0 0x7f77bc06d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c3b80 tx=0x7f77cc009040 comp rx=0 tx=0).stop 2024-02-26T10:42:46.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 msgr2=0x7f77dc0c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 0x7f77dc0c5230 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f77c8001840 tx=0x7f77c8001120 comp rx=0 tx=0).stop 2024-02-26T10:42:46.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 shutdown_connections 2024-02-26T10:42:46.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f77bc06b4d0 0x7f77bc06d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc146b40 0x7f77dc146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77dc14f370 0x7f77dc0c5230 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 --2- 172.21.15.67:0/2837540168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77c0003f60 0x7f77dc1eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 >> 172.21.15.67:0/2837540168 conn(0x7f77dc0c6380 msgr2=0x7f77dc151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:46.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.351+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 shutdown_connections 2024-02-26T10:42:46.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.352+0000 7f77db200640 1 -- 172.21.15.67:0/2837540168 wait complete. 2024-02-26T10:42:46.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:42:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.602+0000 7f9448a00640 1 Processor -- start 2024-02-26T10:42:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.602+0000 7f9448a00640 1 -- start start 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9448a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9444156320 0x7f9444156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9448a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9444156cd0 con 0x7f9444071710 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9448a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944415d0b0 con 0x7f94440c2540 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9448a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944415d390 con 0x7f9444156320 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9442800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9444071710 0x7f944406e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37894/0 (socket says 172.21.15.67:37894) 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9442800640 1 -- 172.21.15.67:0/1075193056 learned_addr learned my addr 172.21.15.67:0/1075193056 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.604+0000 7f9443200640 1 --2- 172.21.15.67:0/1075193056 >> v2:172.21.15.67:3300/0 conn(0x7f9444156320 0x7f9444156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.605+0000 7f9441400640 1 -- 172.21.15.67:0/1075193056 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254800554 0 0) 0x7f9444156cd0 con 0x7f9444071710 2024-02-26T10:42:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.605+0000 7f9441400640 1 -- 172.21.15.67:0/1075193056 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9428003520 con 0x7f9444071710 2024-02-26T10:42:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.605+0000 7f9443200640 1 -- 172.21.15.67:0/1075193056 >> v1:172.21.15.82:6789/0 conn(0x7f94440c2540 legacy=0x7f944415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.605+0000 7f9443200640 1 -- 172.21.15.67:0/1075193056 >> v1:172.21.15.67:6789/0 conn(0x7f9444071710 legacy=0x7f944406e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.605+0000 7f9443200640 1 -- 172.21.15.67:0/1075193056 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944406dbd0 con 0x7f9444156320 2024-02-26T10:42:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.606+0000 7f9443200640 1 --2- 172.21.15.67:0/1075193056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444156320 0x7f9444156790 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f943c009d90 tx=0x7f943c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bc7c19c9fd8056 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9441400640 1 -- 172.21.15.67:0/1075193056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c039070 con 0x7f9444156320 2024-02-26T10:42:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9441400640 1 -- 172.21.15.67:0/1075193056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c02cc60 con 0x7f9444156320 2024-02-26T10:42:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9441400640 1 -- 172.21.15.67:0/1075193056 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c034660 con 0x7f9444156320 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 -- 172.21.15.67:0/1075193056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444156320 msgr2=0x7f9444156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 --2- 172.21.15.67:0/1075193056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444156320 0x7f9444156790 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f943c009d90 tx=0x7f943c02beb0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 -- 172.21.15.67:0/1075193056 shutdown_connections 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 --2- 172.21.15.67:0/1075193056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444156320 0x7f9444156790 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 -- 172.21.15.67:0/1075193056 >> 172.21.15.67:0/1075193056 conn(0x7f94440c6380 msgr2=0x7f94440c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 -- 172.21.15.67:0/1075193056 shutdown_connections 2024-02-26T10:42:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.607+0000 7f9448a00640 1 -- 172.21.15.67:0/1075193056 wait complete. 2024-02-26T10:42:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.609+0000 7f9448a00640 1 Processor -- start 2024-02-26T10:42:46.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.609+0000 7f9448a00640 1 -- start start 2024-02-26T10:42:46.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9448a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 0x7f94440a9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9448a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94440c2540 0x7f94441ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9448a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444156320 0x7f94440ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9448a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944415cd70 con 0x7f9444071710 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9448a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440c29d0 con 0x7f9444156320 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9448a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9444159290 con 0x7f94440c2540 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9442800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 0x7f94440a9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9442800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 0x7f94440a9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39618/0 (socket says 172.21.15.67:39618) 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.610+0000 7f9442800640 1 -- 172.21.15.67:0/1503871004 learned_addr learned my addr 172.21.15.67:0/1503871004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9441e00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94440c2540 0x7f94441ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9443200640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444156320 0x7f94440ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9442800640 1 -- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94440c2540 msgr2=0x7f94441ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9442800640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94440c2540 0x7f94441ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9442800640 1 -- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444156320 msgr2=0x7f94440ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9442800640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444156320 0x7f94440ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9442800640 1 -- 172.21.15.67:0/1503871004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440a95b0 con 0x7f9444071710 2024-02-26T10:42:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9441e00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94440c2540 0x7f94441ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9443200640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444156320 0x7f94440ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f9442800640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 0x7f94440a9b60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f943400c440 tx=0x7f943400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434017070 con 0x7f9444071710 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434005ce0 con 0x7f9444071710 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.611+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340072b0 con 0x7f9444071710 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.612+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94440a9890 con 0x7f9444071710 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.612+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94441e3e30 con 0x7f9444071710 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.613+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94440678e0 con 0x7f9444071710 2024-02-26T10:42:46.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.614+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f943401a070 con 0x7f9444071710 2024-02-26T10:42:46.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.615+0000 7f943aa00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f942406b4d0 0x7f942406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:46.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.615+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f9434089880 con 0x7f9444071710 2024-02-26T10:42:46.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.616+0000 7f9441e00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f942406b4d0 0x7f942406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.617+0000 7f9441e00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f942406b4d0 0x7f942406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f94440c51e0 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:46.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.620+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9434058450 con 0x7f9444071710 2024-02-26T10:42:46.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.821+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94441e4210 con 0x7f9444071710 2024-02-26T10:42:46.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.822+0000 7f943aa00640 1 -- 172.21.15.67:0/1503871004 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7f94441e4210 con 0x7f9444071710 2024-02-26T10:42:46.820 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f942406b4d0 msgr2=0x7f942406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f942406b4d0 0x7f942406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f94440c51e0 tx=0x7f9430009040 comp rx=0 tx=0).stop 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 msgr2=0x7f94440a9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 0x7f94440a9b60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f943400c440 tx=0x7f943400c900 comp rx=0 tx=0).stop 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 shutdown_connections 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f942406b4d0 0x7f942406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444156320 0x7f94440ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94440c2540 0x7f94441ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 --2- 172.21.15.67:0/1503871004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9444071710 0x7f94440a9b60 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:46.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 >> 172.21.15.67:0/1503871004 conn(0x7f94440c6380 msgr2=0x7f94440c3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 shutdown_connections 2024-02-26T10:42:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:46.825+0000 7f9448a00640 1 -- 172.21.15.67:0/1503871004 wait complete. 2024-02-26T10:42:46.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:42:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.055+0000 7fa958e00640 1 Processor -- start 2024-02-26T10:42:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.055+0000 7fa958e00640 1 -- start start 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.056+0000 7fa958e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9541503a0 0x7fa9540ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.056+0000 7fa958e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9540a9430 0x7fa9541535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.056+0000 7fa958e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa954153ae0 con 0x7fa95406e2d0 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.056+0000 7fa958e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa954150780 con 0x7fa9541503a0 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.056+0000 7fa958e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540a98c0 con 0x7fa9540a9430 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9541503a0 0x7fa9540ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9541503a0 0x7fa9540ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39622/0 (socket says 172.21.15.67:39622) 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953400640 1 -- 172.21.15.67:0/3549945722 learned_addr learned my addr 172.21.15.67:0/3549945722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953e00640 1 --1- 172.21.15.67:0/3549945722 >> v1:172.21.15.67:6789/0 conn(0x7fa95406e2d0 0x7fa9540c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37908/0 (socket says 172.21.15.67:37908) 2024-02-26T10:42:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa952a00640 1 --2- 172.21.15.67:0/3549945722 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9540a9430 0x7fa9541535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953400640 1 -- 172.21.15.67:0/3549945722 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9540a9430 msgr2=0x7fa9541535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953400640 1 --2- 172.21.15.67:0/3549945722 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9540a9430 0x7fa9541535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.057+0000 7fa953400640 1 -- 172.21.15.67:0/3549945722 >> v1:172.21.15.67:6789/0 conn(0x7fa95406e2d0 legacy=0x7fa9540c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.058+0000 7fa953400640 1 -- 172.21.15.67:0/3549945722 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540a9ba0 con 0x7fa9541503a0 2024-02-26T10:42:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.059+0000 7fa953400640 1 --2- 172.21.15.67:0/3549945722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa9540ab480 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa944009e50 tx=0x7fa94402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c8c006144ae69a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.059+0000 7fa952000640 1 -- 172.21.15.67:0/3549945722 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94402cb00 con 0x7fa9541503a0 2024-02-26T10:42:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.059+0000 7fa952000640 1 -- 172.21.15.67:0/3549945722 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94402cca0 con 0x7fa9541503a0 2024-02-26T10:42:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.059+0000 7fa952000640 1 -- 172.21.15.67:0/3549945722 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944034610 con 0x7fa9541503a0 2024-02-26T10:42:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.059+0000 7fa958e00640 1 -- 172.21.15.67:0/3549945722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 msgr2=0x7fa9540ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.059+0000 7fa958e00640 1 --2- 172.21.15.67:0/3549945722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa9540ab480 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa944009e50 tx=0x7fa94402bf60 comp rx=0 tx=0).stop 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.060+0000 7fa958e00640 1 -- 172.21.15.67:0/3549945722 shutdown_connections 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.060+0000 7fa958e00640 1 --2- 172.21.15.67:0/3549945722 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9540a9430 0x7fa9541535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.060+0000 7fa958e00640 1 --2- 172.21.15.67:0/3549945722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa9540ab480 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.060+0000 7fa958e00640 1 -- 172.21.15.67:0/3549945722 >> 172.21.15.67:0/3549945722 conn(0x7fa9540c6380 msgr2=0x7fa9540c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.060+0000 7fa958e00640 1 -- 172.21.15.67:0/3549945722 shutdown_connections 2024-02-26T10:42:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.060+0000 7fa958e00640 1 -- 172.21.15.67:0/3549945722 wait complete. 2024-02-26T10:42:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.061+0000 7fa958e00640 1 Processor -- start 2024-02-26T10:42:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 -- start start 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa95406e2d0 0x7fa954151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa954143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9541482b0 0x7fa9541440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95406ddb0 con 0x7fa9541503a0 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95406dad0 con 0x7fa9541482b0 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.062+0000 7fa958e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540c0f80 con 0x7fa95406e2d0 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa954143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa954143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39636/0 (socket says 172.21.15.67:39636) 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 -- 172.21.15.67:0/886156689 learned_addr learned my addr 172.21.15.67:0/886156689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa953400640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa95406e2d0 0x7fa954151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa953e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9541482b0 0x7fa9541440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 -- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa95406e2d0 msgr2=0x7fa954151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa95406e2d0 0x7fa954151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 -- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9541482b0 msgr2=0x7fa9541440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9541482b0 0x7fa9541440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.063+0000 7fa952a00640 1 -- 172.21.15.67:0/886156689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954154120 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.064+0000 7fa952a00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa954143b90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00d800 tx=0x7fa93c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.064+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00b660 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.064+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c019070 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.064+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00ba20 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.064+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9541ec940 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.064+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9541522c0 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.065+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9540678e0 con 0x7fa9541503a0 2024-02-26T10:42:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.066+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa93c00bbc0 con 0x7fa9541503a0 2024-02-26T10:42:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.067+0000 7fa94b400640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa93006b500 0x7fa93006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.068+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa93c08a9b0 con 0x7fa9541503a0 2024-02-26T10:42:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.068+0000 7fa953400640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa93006b500 0x7fa93006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.069+0000 7fa953400640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa93006b500 0x7fa93006d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa944035000 tx=0x7fa944030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.072+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa93c01c360 con 0x7fa9541503a0 2024-02-26T10:42:47.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.270+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fa9540aaa30 con 0x7fa9541503a0 2024-02-26T10:42:47.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.272+0000 7fa94b400640 1 -- 172.21.15.67:0/886156689 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7fa9540aaa30 con 0x7fa9541503a0 2024-02-26T10:42:47.269 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa93006b500 msgr2=0x7fa93006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa93006b500 0x7fa93006d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa944035000 tx=0x7fa944030000 comp rx=0 tx=0).stop 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 msgr2=0x7fa954143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa954143b90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00d800 tx=0x7fa93c00dcc0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 shutdown_connections 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa93006b500 0x7fa93006d9c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9541482b0 0x7fa9541440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9541503a0 0x7fa954143b90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 --2- 172.21.15.67:0/886156689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa95406e2d0 0x7fa954151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 >> 172.21.15.67:0/886156689 conn(0x7fa9540c6380 msgr2=0x7fa95414d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 shutdown_connections 2024-02-26T10:42:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.274+0000 7fa958e00640 1 -- 172.21.15.67:0/886156689 wait complete. 2024-02-26T10:42:47.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:42:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.515+0000 7f6896800640 1 Processor -- start 2024-02-26T10:42:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.515+0000 7f6896800640 1 -- start start 2024-02-26T10:42:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.516+0000 7f6896800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6890153630 0x7f689006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.516+0000 7f6896800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890155f10 con 0x7f689006ac20 2024-02-26T10:42:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.516+0000 7f6896800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890153a60 con 0x7f68900c2310 2024-02-26T10:42:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.516+0000 7f6896800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900c28d0 con 0x7f6890153630 2024-02-26T10:42:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.517+0000 7f6895400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f689006ac20 0x7f6890152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37912/0 (socket says 172.21.15.67:37912) 2024-02-26T10:42:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.517+0000 7f6895400640 1 -- 172.21.15.67:0/1268755652 learned_addr learned my addr 172.21.15.67:0/1268755652 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.517+0000 7f6895e00640 1 --2- 172.21.15.67:0/1268755652 >> v2:172.21.15.82:3300/0 conn(0x7f6890153630 0x7f689006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.517+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387146076 0 0) 0x7f6890155f10 con 0x7f689006ac20 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.517+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6878003520 con 0x7f689006ac20 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865251714 0 0) 0x7f6878003520 con 0x7f689006ac20 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 >> v2:172.21.15.82:3300/0 conn(0x7f6890153630 msgr2=0x7f689006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 --2- 172.21.15.67:0/1268755652 >> v2:172.21.15.82:3300/0 conn(0x7f6890153630 0x7f689006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 >> v1:172.21.15.82:6789/0 conn(0x7f68900c2310 legacy=0x7f689015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890155150 con 0x7f689006ac20 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f6880001670 con 0x7f689006ac20 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 >> v1:172.21.15.67:6789/0 conn(0x7f689006ac20 legacy=0x7f6890152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780039b0 0x7f6878003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6878004800 0x7f6878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780052e0 0x7f687800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900c28d0 con 0x7f68780052e0 2024-02-26T10:42:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687800c1b0 con 0x7f68780039b0 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687800c3b0 con 0x7f6878004800 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f6895e00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780052e0 0x7f687800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.518+0000 7f6895400640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780039b0 0x7f6878003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6894a00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6878004800 0x7f6878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6895e00640 1 -- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6878004800 msgr2=0x7f6878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6895e00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6878004800 0x7f6878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6895e00640 1 -- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780039b0 msgr2=0x7f6878003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6895e00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780039b0 0x7f6878003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6895e00640 1 -- 172.21.15.67:0/1268755652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878003520 con 0x7f68780052e0 2024-02-26T10:42:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.519+0000 7f6894a00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6878004800 0x7f6878004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.520+0000 7f6895e00640 1 -- 172.21.15.67:0/1268755652 reap_dead start 2024-02-26T10:42:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.520+0000 7f6895e00640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780052e0 0x7f687800b9c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6888009a90 tx=0x7f6878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adfaef4db31a7b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.520+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888009690 con 0x7f68780052e0 2024-02-26T10:42:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.520+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888004020 con 0x7f68780052e0 2024-02-26T10:42:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.520+0000 7f688fe00640 1 -- 172.21.15.67:0/1268755652 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880043d0 con 0x7f68780052e0 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 -- 172.21.15.67:0/1268755652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780052e0 msgr2=0x7f687800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780052e0 0x7f687800b9c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6888009a90 tx=0x7f6878005830 comp rx=0 tx=0).stop 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 -- 172.21.15.67:0/1268755652 shutdown_connections 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 --2- 172.21.15.67:0/1268755652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780052e0 0x7f687800b9c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 -- 172.21.15.67:0/1268755652 >> 172.21.15.67:0/1268755652 conn(0x7f68900c63f0 msgr2=0x7f68900c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 -- 172.21.15.67:0/1268755652 shutdown_connections 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.521+0000 7f6896800640 1 -- 172.21.15.67:0/1268755652 wait complete. 2024-02-26T10:42:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.522+0000 7f6896800640 1 Processor -- start 2024-02-26T10:42:47.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.523+0000 7f6896800640 1 -- start start 2024-02-26T10:42:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.523+0000 7f6896800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780052e0 0x7f68901ede70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.523+0000 7f6896800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 0x7f6890161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6896800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68900abd50 0x7f6890071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6896800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901559f0 con 0x7f6890160150 2024-02-26T10:42:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6896800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890155710 con 0x7f68780052e0 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6896800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6890148850 con 0x7f68900abd50 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 0x7f6890161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 0x7f6890161550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39646/0 (socket says 172.21.15.67:39646) 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 -- 172.21.15.67:0/4020541258 learned_addr learned my addr 172.21.15.67:0/4020541258 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6895e00640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68900abd50 0x7f6890071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6895400640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780052e0 0x7f68901ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 -- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68900abd50 msgr2=0x7f6890071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68900abd50 0x7f6890071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 -- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780052e0 msgr2=0x7f68901ede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780052e0 0x7f68901ede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.524+0000 7f6894a00640 1 -- 172.21.15.67:0/4020541258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890161a90 con 0x7f6890160150 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.525+0000 7f6894a00640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 0x7f6890161550 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f688400d8e0 tx=0x7f688400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.525+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884003da0 con 0x7f6890160150 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.525+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884017070 con 0x7f6890160150 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.525+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400b8d0 con 0x7f6890160150 2024-02-26T10:42:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.525+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689015fb90 con 0x7f6890160150 2024-02-26T10:42:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.525+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68900c3440 con 0x7f6890160150 2024-02-26T10:42:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.526+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6890074a30 con 0x7f6890160150 2024-02-26T10:42:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.527+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f688400ba70 con 0x7f6890160150 2024-02-26T10:42:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.528+0000 7f688d600640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f687406b500 0x7f687406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.528+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f688408a850 con 0x7f6890160150 2024-02-26T10:42:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.529+0000 7f6895400640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f687406b500 0x7f687406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.530+0000 7f6895400640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f687406b500 0x7f687406d9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6880001eb0 tx=0x7f6880000fe0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.533+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68840592d0 con 0x7f6890160150 2024-02-26T10:42:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.746+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68900c20f0 con 0x7f6890160150 2024-02-26T10:42:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.747+0000 7f688d600640 1 -- 172.21.15.67:0/4020541258 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6884058bd0 con 0x7f6890160150 2024-02-26T10:42:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.749+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f687406b500 msgr2=0x7f687406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f687406b500 0x7f687406d9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6880001eb0 tx=0x7f6880000fe0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 msgr2=0x7f6890161550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 0x7f6890161550 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f688400d8e0 tx=0x7f688400dda0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 shutdown_connections 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68900abd50 0x7f6890071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6890160150 0x7f6890161550 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f687406b500 0x7f687406d9c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 --2- 172.21.15.67:0/4020541258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68780052e0 0x7f68901ede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 >> 172.21.15.67:0/4020541258 conn(0x7f68900c63f0 msgr2=0x7f6890152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 shutdown_connections 2024-02-26T10:42:47.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:47.750+0000 7f6896800640 1 -- 172.21.15.67:0/4020541258 wait complete. 2024-02-26T10:42:47.767 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.testuser... 2024-02-26T10:42:47.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:47.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-26T10:42:47.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:42:47.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:47.877 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-26T10:42:47.930 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:47.930 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:47.987 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:47.987 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:48.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.269+0000 7f44b7200640 1 Processor -- start 2024-02-26T10:42:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.270+0000 7f44b7200640 1 -- start start 2024-02-26T10:42:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44b814b950 0x7f44b8155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44b80c6350 0x7f44b80c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b8158580 con 0x7f44b80c58b0 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b8155e60 con 0x7f44b80c6350 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b80c2f80 con 0x7f44b814b950 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44b80c6350 0x7f44b80c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f44b80c58b0 0x7f44b8158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37928/0 (socket says 172.21.15.67:37928) 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b6800640 1 -- 172.21.15.67:0/2369488300 learned_addr learned my addr 172.21.15.67:0/2369488300 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.271+0000 7f44b5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44b80c6350 0x7f44b80c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39654/0 (socket says 172.21.15.67:39654) 2024-02-26T10:42:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.272+0000 7f44b5e00640 1 --2- 172.21.15.67:0/2369488300 >> v2:172.21.15.82:3300/0 conn(0x7f44b814b950 0x7f44b8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.272+0000 7f44b5400640 1 -- 172.21.15.67:0/2369488300 >> v2:172.21.15.82:3300/0 conn(0x7f44b814b950 msgr2=0x7f44b8155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.272+0000 7f44b5400640 1 --2- 172.21.15.67:0/2369488300 >> v2:172.21.15.82:3300/0 conn(0x7f44b814b950 0x7f44b8155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.272+0000 7f44b5400640 1 -- 172.21.15.67:0/2369488300 >> v1:172.21.15.67:6789/0 conn(0x7f44b80c58b0 legacy=0x7f44b8158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.272+0000 7f44b5400640 1 -- 172.21.15.67:0/2369488300 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b80c3260 con 0x7f44b80c6350 2024-02-26T10:42:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.272+0000 7f44b4a00640 1 -- 172.21.15.67:0/2369488300 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617406345 0 0) 0x7f44b8158580 con 0x7f44b80c58b0 2024-02-26T10:42:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.273+0000 7f44b5400640 1 --2- 172.21.15.67:0/2369488300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b80c6350 0x7f44b80c5370 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f44b80c1e20 tx=0x7f44a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d35007130b192c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.273+0000 7f44b4a00640 1 -- 172.21.15.67:0/2369488300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a40315a0 con 0x7f44b80c6350 2024-02-26T10:42:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.273+0000 7f44b4a00640 1 -- 172.21.15.67:0/2369488300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a4037070 con 0x7f44b80c6350 2024-02-26T10:42:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.273+0000 7f44b4a00640 1 -- 172.21.15.67:0/2369488300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4031960 con 0x7f44b80c6350 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 -- 172.21.15.67:0/2369488300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b80c6350 msgr2=0x7f44b80c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 --2- 172.21.15.67:0/2369488300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b80c6350 0x7f44b80c5370 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f44b80c1e20 tx=0x7f44a402b840 comp rx=0 tx=0).stop 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 -- 172.21.15.67:0/2369488300 shutdown_connections 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 --2- 172.21.15.67:0/2369488300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b80c6350 0x7f44b80c5370 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 --2- 172.21.15.67:0/2369488300 >> v2:172.21.15.82:3300/0 conn(0x7f44b814b950 0x7f44b8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 -- 172.21.15.67:0/2369488300 >> 172.21.15.67:0/2369488300 conn(0x7f44b806b040 msgr2=0x7f44b806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 -- 172.21.15.67:0/2369488300 shutdown_connections 2024-02-26T10:42:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.274+0000 7f44b7200640 1 -- 172.21.15.67:0/2369488300 wait complete. 2024-02-26T10:42:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.276+0000 7f44b7200640 1 Processor -- start 2024-02-26T10:42:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.276+0000 7f44b7200640 1 -- start start 2024-02-26T10:42:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b80c58b0 0x7f44b81eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b80c6350 0x7f44b80a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 0x7f44b806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80c3820 con 0x7f44b814b950 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b814bd30 con 0x7f44b80c58b0 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80c39a0 con 0x7f44b80c6350 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 0x7f44b806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 0x7f44b806d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39664/0 (socket says 172.21.15.67:39664) 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b6800640 1 -- 172.21.15.67:0/101367697 learned_addr learned my addr 172.21.15.67:0/101367697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b5e00640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b80c58b0 0x7f44b81eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.277+0000 7f44b5400640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b80c6350 0x7f44b80a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b6800640 1 -- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b80c6350 msgr2=0x7f44b80a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b6800640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b80c6350 0x7f44b80a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b6800640 1 -- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b80c58b0 msgr2=0x7f44b81eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b6800640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b80c58b0 0x7f44b81eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b6800640 1 -- 172.21.15.67:0/101367697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8152900 con 0x7f44b814b950 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b6800640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 0x7f44b806d250 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c810 tx=0x7f44b000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0018070 con 0x7f44b814b950 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b000a640 con 0x7f44b814b950 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b000aa30 con 0x7f44b814b950 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.278+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b80a9f10 con 0x7f44b814b950 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.279+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b8144550 con 0x7f44b814b950 2024-02-26T10:42:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.280+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b8152ad0 con 0x7f44b814b950 2024-02-26T10:42:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.281+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44b001b070 con 0x7f44b814b950 2024-02-26T10:42:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.282+0000 7f44ae000640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b500 0x7f449806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.282+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f44b00899d0 con 0x7f44b814b950 2024-02-26T10:42:48.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.283+0000 7f44b5e00640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b500 0x7f449806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.284+0000 7f44b5e00640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b500 0x7f449806d9c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f44a80079a0 tx=0x7f44a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.287+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44b0057f30 con 0x7f44b814b950 2024-02-26T10:42:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.488+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44b8144790 con 0x7f44b814b950 2024-02-26T10:42:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.493+0000 7f44ae000640 1 -- 172.21.15.67:0/101367697 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7f44b005bfd0 con 0x7f44b814b950 2024-02-26T10:42:48.491 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:48.491 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.495+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b500 msgr2=0x7f449806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.495+0000 7f44b7200640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b500 0x7f449806d9c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f44a80079a0 tx=0x7f44a8008040 comp rx=0 tx=0).stop 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.495+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 msgr2=0x7f44b806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.495+0000 7f44b7200640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 0x7f44b806d250 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c810 tx=0x7f44b000ccd0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 shutdown_connections 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b814b950 0x7f44b806d250 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b80c6350 0x7f44b80a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f449806b500 0x7f449806d9c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 --2- 172.21.15.67:0/101367697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b80c58b0 0x7f44b81eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 >> 172.21.15.67:0/101367697 conn(0x7f44b806b040 msgr2=0x7f44b806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 shutdown_connections 2024-02-26T10:42:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.496+0000 7f44b7200640 1 -- 172.21.15.67:0/101367697 wait complete. 2024-02-26T10:42:48.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:48.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:48.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.742+0000 7f525ea00640 1 Processor -- start 2024-02-26T10:42:48.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.742+0000 7f525ea00640 1 -- start start 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f52581523b0 0x7f52580c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5258071fd0 0x7f5258072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52580c3840 con 0x7f5258154cf0 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5258152790 con 0x7f5258071fd0 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525806da30 con 0x7f52581523b0 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5258071fd0 0x7f5258072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525cc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5258154cf0 0x7f52580c3130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37930/0 (socket says 172.21.15.67:37930) 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5258071fd0 0x7f5258072440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39672/0 (socket says 172.21.15.67:39672) 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.744+0000 7f525cc00640 1 -- 172.21.15.67:0/2833704171 learned_addr learned my addr 172.21.15.67:0/2833704171 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.745+0000 7f525d600640 1 --2- 172.21.15.67:0/2833704171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f52581523b0 0x7f52580c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.745+0000 7f525e000640 1 -- 172.21.15.67:0/2833704171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f52581523b0 msgr2=0x7f52580c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:48.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.745+0000 7f525e000640 1 --2- 172.21.15.67:0/2833704171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f52581523b0 0x7f52580c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.745+0000 7f525e000640 1 -- 172.21.15.67:0/2833704171 >> v1:172.21.15.67:6789/0 conn(0x7f5258154cf0 legacy=0x7f52580c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.745+0000 7f525e000640 1 -- 172.21.15.67:0/2833704171 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525806dc80 con 0x7f5258071fd0 2024-02-26T10:42:48.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.746+0000 7f525e000640 1 --2- 172.21.15.67:0/2833704171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258071fd0 0x7f5258072440 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5250009d90 tx=0x7f525002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ba1f688fe4639 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.747+0000 7f5257e00640 1 -- 172.21.15.67:0/2833704171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250039070 con 0x7f5258071fd0 2024-02-26T10:42:48.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.747+0000 7f5257e00640 1 -- 172.21.15.67:0/2833704171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525002cbd0 con 0x7f5258071fd0 2024-02-26T10:42:48.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.747+0000 7f5257e00640 1 -- 172.21.15.67:0/2833704171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250034440 con 0x7f5258071fd0 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 -- 172.21.15.67:0/2833704171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258071fd0 msgr2=0x7f5258072440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 --2- 172.21.15.67:0/2833704171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258071fd0 0x7f5258072440 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5250009d90 tx=0x7f525002be20 comp rx=0 tx=0).stop 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 -- 172.21.15.67:0/2833704171 shutdown_connections 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 --2- 172.21.15.67:0/2833704171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258071fd0 0x7f5258072440 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 --2- 172.21.15.67:0/2833704171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f52581523b0 0x7f52580c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 -- 172.21.15.67:0/2833704171 >> 172.21.15.67:0/2833704171 conn(0x7f52580c6380 msgr2=0x7f52580c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 -- 172.21.15.67:0/2833704171 shutdown_connections 2024-02-26T10:42:48.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.748+0000 7f525ea00640 1 -- 172.21.15.67:0/2833704171 wait complete. 2024-02-26T10:42:48.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.750+0000 7f525ea00640 1 Processor -- start 2024-02-26T10:42:48.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.750+0000 7f525ea00640 1 -- start start 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5258071fd0 0x7f52580a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52581523b0 0x7f52580a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 0x7f52581edd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525806e520 con 0x7f5258154cf0 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525806e240 con 0x7f52581523b0 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525806e800 con 0x7f5258071fd0 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 0x7f52581edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 0x7f52581edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39682/0 (socket says 172.21.15.67:39682) 2024-02-26T10:42:48.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.751+0000 7f525e000640 1 -- 172.21.15.67:0/3484622703 learned_addr learned my addr 172.21.15.67:0/3484622703 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:48.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525cc00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52581523b0 0x7f52580a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525e000640 1 -- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5258071fd0 msgr2=0x7f52580a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525e000640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5258071fd0 0x7f52580a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525e000640 1 -- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52581523b0 msgr2=0x7f52580a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525e000640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52581523b0 0x7f52580a9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525e000640 1 -- 172.21.15.67:0/3484622703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258157ea0 con 0x7f5258154cf0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f525e000640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 0x7f52581edd20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5250009eb0 tx=0x7f5250004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.752+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250039050 con 0x7f5258154cf0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.753+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250045070 con 0x7f5258154cf0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.753+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525003e830 con 0x7f5258154cf0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.753+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52581e1cf0 con 0x7f5258154cf0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.753+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258158bf0 con 0x7f5258154cf0 2024-02-26T10:42:48.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.754+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52580678e0 con 0x7f5258154cf0 2024-02-26T10:42:48.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.755+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f525003e9d0 con 0x7f5258154cf0 2024-02-26T10:42:48.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.756+0000 7f5255600640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f523806b4d0 0x7f523806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:48.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.756+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f52500ad280 con 0x7f5258154cf0 2024-02-26T10:42:48.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.756+0000 7f525d600640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f523806b4d0 0x7f523806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.757+0000 7f525d600640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f523806b4d0 0x7f523806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5258154770 tx=0x7f524c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:48.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.760+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5250032e20 con 0x7f5258154cf0 2024-02-26T10:42:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.958+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5258074a30 con 0x7f5258154cf0 2024-02-26T10:42:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.959+0000 7f5255600640 1 -- 172.21.15.67:0/3484622703 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f525007be30 con 0x7f5258154cf0 2024-02-26T10:42:48.957 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f523806b4d0 msgr2=0x7f523806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f523806b4d0 0x7f523806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5258154770 tx=0x7f524c008040 comp rx=0 tx=0).stop 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 msgr2=0x7f52581edd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 0x7f52581edd20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5250009eb0 tx=0x7f5250004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 shutdown_connections 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5258154cf0 0x7f52581edd20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f523806b4d0 0x7f523806d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52581523b0 0x7f52580a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 --2- 172.21.15.67:0/3484622703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5258071fd0 0x7f52580a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 >> 172.21.15.67:0/3484622703 conn(0x7f52580c6380 msgr2=0x7f52580723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 shutdown_connections 2024-02-26T10:42:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:48.962+0000 7f525ea00640 1 -- 172.21.15.67:0/3484622703 wait complete. 2024-02-26T10:42:48.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:49.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.205+0000 7fefa9600640 1 Processor -- start 2024-02-26T10:42:49.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.206+0000 7fefa9600640 1 -- start start 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.207+0000 7fefa9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fefa40c5290 0x7fefa41562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.207+0000 7fefa9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.207+0000 7fefa9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa406c110 con 0x7fefa4155880 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.207+0000 7fefa9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa406d000 con 0x7fefa40c5290 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.207+0000 7fefa9600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40c72c0 con 0x7fefa40af8a0 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fefa4155880 0x7fefa406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37938/0 (socket says 172.21.15.67:37938) 2024-02-26T10:42:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3400640 1 -- 172.21.15.67:0/3662621137 learned_addr learned my addr 172.21.15.67:0/3662621137 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:49.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3e00640 1 --2- 172.21.15.67:0/3662621137 >> v2:172.21.15.67:3300/0 conn(0x7fefa40c5290 0x7fefa41562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa8c00640 1 --2- 172.21.15.67:0/3662621137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa2a00640 1 -- 172.21.15.67:0/3662621137 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244820103 0 0) 0x7fefa406c110 con 0x7fefa4155880 2024-02-26T10:42:49.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa2a00640 1 -- 172.21.15.67:0/3662621137 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef88003520 con 0x7fefa4155880 2024-02-26T10:42:49.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3e00640 1 -- 172.21.15.67:0/3662621137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 msgr2=0x7fefa40afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:49.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3e00640 1 --2- 172.21.15.67:0/3662621137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa40afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3e00640 1 -- 172.21.15.67:0/3662621137 >> v1:172.21.15.67:6789/0 conn(0x7fefa4155880 legacy=0x7fefa406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.208+0000 7fefa3e00640 1 -- 172.21.15.67:0/3662621137 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa40c75a0 con 0x7fefa40c5290 2024-02-26T10:42:49.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.209+0000 7fefa3e00640 1 --2- 172.21.15.67:0/3662621137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa41562f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fef94009d90 tx=0x7fef9402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c3b55f72645c22 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa2a00640 1 -- 172.21.15.67:0/3662621137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94039070 con 0x7fefa40c5290 2024-02-26T10:42:49.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa2a00640 1 -- 172.21.15.67:0/3662621137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9402cbc0 con 0x7fefa40c5290 2024-02-26T10:42:49.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa2a00640 1 -- 172.21.15.67:0/3662621137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94034440 con 0x7fefa40c5290 2024-02-26T10:42:49.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 -- 172.21.15.67:0/3662621137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 msgr2=0x7fefa41562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 --2- 172.21.15.67:0/3662621137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa41562f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fef94009d90 tx=0x7fef9402be10 comp rx=0 tx=0).stop 2024-02-26T10:42:49.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 -- 172.21.15.67:0/3662621137 shutdown_connections 2024-02-26T10:42:49.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 --2- 172.21.15.67:0/3662621137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa40afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 --2- 172.21.15.67:0/3662621137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa41562f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 -- 172.21.15.67:0/3662621137 >> 172.21.15.67:0/3662621137 conn(0x7fefa40ac230 msgr2=0x7fefa40ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.210+0000 7fefa9600640 1 -- 172.21.15.67:0/3662621137 shutdown_connections 2024-02-26T10:42:49.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.211+0000 7fefa9600640 1 -- 172.21.15.67:0/3662621137 wait complete. 2024-02-26T10:42:49.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.212+0000 7fefa9600640 1 Processor -- start 2024-02-26T10:42:49.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.212+0000 7fefa9600640 1 -- start start 2024-02-26T10:42:49.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa41549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa4156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa4155880 0x7fefa4153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4155cf0 con 0x7fefa40c5290 2024-02-26T10:42:49.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa9600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa406c3d0 con 0x7fefa4155880 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa9600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40c5670 con 0x7fefa40af8a0 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.213+0000 7fefa3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa4156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa4156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39698/0 (socket says 172.21.15.67:39698) 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 -- 172.21.15.67:0/73890553 learned_addr learned my addr 172.21.15.67:0/73890553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3e00640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa41549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 -- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 msgr2=0x7fefa41549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa41549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 -- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa4155880 msgr2=0x7fefa4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa8c00640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa4155880 0x7fefa4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa4155880 0x7fefa4153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 -- 172.21.15.67:0/73890553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa41550f0 con 0x7fefa40c5290 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3e00640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa41549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.214+0000 7fefa3400640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa4156460 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c390 tx=0x7fef8c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.215+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c017070 con 0x7fefa40c5290 2024-02-26T10:42:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.215+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c005ce0 con 0x7fefa40c5290 2024-02-26T10:42:49.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.215+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0072f0 con 0x7fefa40c5290 2024-02-26T10:42:49.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.215+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa41edae0 con 0x7fefa40c5290 2024-02-26T10:42:49.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.215+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa41ee0a0 con 0x7fefa40c5290 2024-02-26T10:42:49.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.216+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa4074a30 con 0x7fefa40c5290 2024-02-26T10:42:49.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.217+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef8c01a070 con 0x7fefa40c5290 2024-02-26T10:42:49.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.218+0000 7fef9be00640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fef8006b500 0x7fef8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.218+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fef8c089940 con 0x7fefa40c5290 2024-02-26T10:42:49.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.219+0000 7fefa3e00640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fef8006b500 0x7fef8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.220+0000 7fefa3e00640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fef8006b500 0x7fef8006d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fef94036000 tx=0x7fef94030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.223+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef8c058450 con 0x7fefa40c5290 2024-02-26T10:42:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.419+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fefa406d950 con 0x7fefa40c5290 2024-02-26T10:42:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.420+0000 7fef9be00640 1 -- 172.21.15.67:0/73890553 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef8c057d50 con 0x7fefa40c5290 2024-02-26T10:42:49.417 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:49.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.422+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fef8006b500 msgr2=0x7fef8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.422+0000 7fefa9600640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fef8006b500 0x7fef8006d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fef94036000 tx=0x7fef94030000 comp rx=0 tx=0).stop 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 msgr2=0x7fefa4156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa4156460 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c390 tx=0x7fef8c00c850 comp rx=0 tx=0).stop 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 shutdown_connections 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fef8006b500 0x7fef8006d9c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa4155880 0x7fefa4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fefa40c5290 0x7fefa4156460 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 --2- 172.21.15.67:0/73890553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fefa40af8a0 0x7fefa41549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 >> 172.21.15.67:0/73890553 conn(0x7fefa40ac230 msgr2=0x7fefa40aaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 shutdown_connections 2024-02-26T10:42:49.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.423+0000 7fefa9600640 1 -- 172.21.15.67:0/73890553 wait complete. 2024-02-26T10:42:49.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.664+0000 7f1aabe00640 1 Processor -- start 2024-02-26T10:42:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.664+0000 7f1aabe00640 1 -- start start 2024-02-26T10:42:49.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.666+0000 7f1aabe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1aac0c1eb0 0x7f1aac14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.666+0000 7f1aabe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac0c1be0 con 0x7f1aac14f260 2024-02-26T10:42:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.666+0000 7f1aabe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac0c3660 con 0x7f1aac0c25a0 2024-02-26T10:42:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.666+0000 7f1aabe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac0c1750 con 0x7f1aac0c1eb0 2024-02-26T10:42:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.666+0000 7f1aab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1aac14f260 0x7f1aac151980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49540/0 (socket says 172.21.15.67:49540) 2024-02-26T10:42:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.666+0000 7f1aab400640 1 -- 172.21.15.67:0/656547876 learned_addr learned my addr 172.21.15.67:0/656547876 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051737956 0 0) 0x7f1aac0c1be0 con 0x7f1aac14f260 2024-02-26T10:42:49.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aa3e00640 1 --2- 172.21.15.67:0/656547876 >> v2:172.21.15.82:3300/0 conn(0x7f1aac0c1eb0 0x7f1aac14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a90005730 con 0x7f1aac14f260 2024-02-26T10:42:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1471436957 0 0) 0x7f1a90005730 con 0x7f1aac14f260 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 >> v2:172.21.15.82:3300/0 conn(0x7f1aac0c1eb0 msgr2=0x7f1aac14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 --2- 172.21.15.67:0/656547876 >> v2:172.21.15.82:3300/0 conn(0x7f1aac0c1eb0 0x7f1aac14ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 >> v1:172.21.15.82:6789/0 conn(0x7f1aac0c25a0 legacy=0x7f1aac0c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0c0e20 con 0x7f1aac14f260 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1aa40019f0 con 0x7f1aac14f260 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.667+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 >> v1:172.21.15.67:6789/0 conn(0x7f1aac14f260 legacy=0x7f1aac151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaa000640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90005c10 0x7f1a90006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaa000640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a90006a10 0x7f1a90006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaa000640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a900074f0 0x7f1a9000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0c1750 con 0x7f1a900074f0 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9000e410 con 0x7f1a90005c10 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9000e610 con 0x7f1a90006a10 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aab400640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a900074f0 0x7f1a9000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aaaa00640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90005c10 0x7f1a90006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aa3e00640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a90006a10 0x7f1a90006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aab400640 1 -- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a90006a10 msgr2=0x7f1a90006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aab400640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a90006a10 0x7f1a90006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aab400640 1 -- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90005c10 msgr2=0x7f1a90006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aab400640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90005c10 0x7f1a90006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.668+0000 7f1aab400640 1 -- 172.21.15.67:0/656547876 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90005730 con 0x7f1a900074f0 2024-02-26T10:42:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.669+0000 7f1aa3e00640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a90006a10 0x7f1a90006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.669+0000 7f1aab400640 1 -- 172.21.15.67:0/656547876 reap_dead start 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.670+0000 7f1aab400640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a900074f0 0x7f1a9000dbd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400a790 tx=0x7f1a90007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ab376e74deff27 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.670+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400a3b0 con 0x7f1a900074f0 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.670+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4034670 con 0x7f1a900074f0 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.670+0000 7f1aaa000640 1 -- 172.21.15.67:0/656547876 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4034a60 con 0x7f1a900074f0 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 -- 172.21.15.67:0/656547876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a900074f0 msgr2=0x7f1a9000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a900074f0 0x7f1a9000dbd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400a790 tx=0x7f1a90007a40 comp rx=0 tx=0).stop 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 -- 172.21.15.67:0/656547876 shutdown_connections 2024-02-26T10:42:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 --2- 172.21.15.67:0/656547876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a900074f0 0x7f1a9000dbd0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 -- 172.21.15.67:0/656547876 >> 172.21.15.67:0/656547876 conn(0x7f1aac0c6380 msgr2=0x7f1aac0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 -- 172.21.15.67:0/656547876 shutdown_connections 2024-02-26T10:42:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.671+0000 7f1aabe00640 1 -- 172.21.15.67:0/656547876 wait complete. 2024-02-26T10:42:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.673+0000 7f1aabe00640 1 Processor -- start 2024-02-26T10:42:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.673+0000 7f1aabe00640 1 -- start start 2024-02-26T10:42:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aabe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a900074f0 0x7f1aac1e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aabe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aac0abc30 0x7f1aac142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aabe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 0x7f1aac0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aabe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0c13e0 con 0x7f1aac143410 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aabe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac14f640 con 0x7f1aac0abc30 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aabe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac0c2980 con 0x7f1a900074f0 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 0x7f1aac0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 0x7f1aac0c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34632/0 (socket says 172.21.15.67:34632) 2024-02-26T10:42:49.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 -- 172.21.15.67:0/3443314187 learned_addr learned my addr 172.21.15.67:0/3443314187 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:49.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aa3e00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aac0abc30 0x7f1aac142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aaaa00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a900074f0 0x7f1aac1e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 -- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a900074f0 msgr2=0x7f1aac1e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a900074f0 0x7f1aac1e9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 -- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aac0abc30 msgr2=0x7f1aac142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aac0abc30 0x7f1aac142ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.674+0000 7f1aab400640 1 -- 172.21.15.67:0/3443314187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac149900 con 0x7f1aac143410 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aa3e00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aac0abc30 0x7f1aac142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aab400640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 0x7f1aac0c4d80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400a8b0 tx=0x7f1aac0c3830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4043070 con 0x7f1aac143410 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4001250 con 0x7f1aac143410 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac149be0 con 0x7f1aac143410 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4039040 con 0x7f1aac143410 2024-02-26T10:42:49.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.675+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac0c4580 con 0x7f1aac143410 2024-02-26T10:42:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.676+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac0678e0 con 0x7f1aac143410 2024-02-26T10:42:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.677+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aa4046070 con 0x7f1aac143410 2024-02-26T10:42:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.678+0000 7f1aa2a00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a8806b500 0x7f1a8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.678+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1aa40ab9e0 con 0x7f1aac143410 2024-02-26T10:42:49.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.679+0000 7f1aaaa00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a8806b500 0x7f1a8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:49.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.680+0000 7f1aaaa00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a8806b500 0x7f1a8806d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0014d0 tx=0x7f1a9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.683+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1aa4079d80 con 0x7f1aac143410 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.887+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aac074a30 con 0x7f1aac143410 2024-02-26T10:42:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.888+0000 7f1aa2a00640 1 -- 172.21.15.67:0/3443314187 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1aa407de20 con 0x7f1aac143410 2024-02-26T10:42:49.886 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a8806b500 msgr2=0x7f1a8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a8806b500 0x7f1a8806d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0014d0 tx=0x7f1a9c009040 comp rx=0 tx=0).stop 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 msgr2=0x7f1aac0c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 0x7f1aac0c4d80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400a8b0 tx=0x7f1aac0c3830 comp rx=0 tx=0).stop 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 shutdown_connections 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac143410 0x7f1aac0c4d80 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1a8806b500 0x7f1a8806d9c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aac0abc30 0x7f1aac142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 --2- 172.21.15.67:0/3443314187 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1a900074f0 0x7f1aac1e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 >> 172.21.15.67:0/3443314187 conn(0x7f1aac0c6380 msgr2=0x7f1aac14c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 shutdown_connections 2024-02-26T10:42:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:49.891+0000 7f1aabe00640 1 -- 172.21.15.67:0/3443314187 wait complete. 2024-02-26T10:42:49.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:42:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.131+0000 7f596be00640 1 Processor -- start 2024-02-26T10:42:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.132+0000 7f596be00640 1 -- start start 2024-02-26T10:42:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.133+0000 7f596be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c0c5290 0x7f596c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.133+0000 7f596be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f596c0af8a0 0x7f596c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.133+0000 7f596be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c06c110 con 0x7f596c155880 2024-02-26T10:42:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.133+0000 7f596be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c06d000 con 0x7f596c0af8a0 2024-02-26T10:42:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.133+0000 7f596be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0c72c0 con 0x7f596c0c5290 2024-02-26T10:42:50.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f596c0af8a0 0x7f596c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f596c0af8a0 0x7f596c0afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34642/0 (socket says 172.21.15.67:34642) 2024-02-26T10:42:50.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 -- 172.21.15.67:0/3662467884 learned_addr learned my addr 172.21.15.67:0/3662467884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:50.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596aa00640 1 --2- 172.21.15.67:0/3662467884 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c0c5290 0x7f596c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 -- 172.21.15.67:0/3662467884 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c0c5290 msgr2=0x7f596c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 --2- 172.21.15.67:0/3662467884 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c0c5290 0x7f596c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 -- 172.21.15.67:0/3662467884 >> v1:172.21.15.82:6789/0 conn(0x7f596c155880 legacy=0x7f596c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.134+0000 7f596b400640 1 -- 172.21.15.67:0/3662467884 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0c75a0 con 0x7f596c0af8a0 2024-02-26T10:42:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.135+0000 7f596b400640 1 --2- 172.21.15.67:0/3662467884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0af8a0 0x7f596c0afd30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5964009e50 tx=0x7f596402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff2bec213cb2bbf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.136+0000 7f5969600640 1 -- 172.21.15.67:0/3662467884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596402ca70 con 0x7f596c0af8a0 2024-02-26T10:42:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.136+0000 7f5969600640 1 -- 172.21.15.67:0/3662467884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596402cc10 con 0x7f596c0af8a0 2024-02-26T10:42:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.136+0000 7f5969600640 1 -- 172.21.15.67:0/3662467884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640343f0 con 0x7f596c0af8a0 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 -- 172.21.15.67:0/3662467884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0af8a0 msgr2=0x7f596c0afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 --2- 172.21.15.67:0/3662467884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0af8a0 0x7f596c0afd30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5964009e50 tx=0x7f596402bed0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 -- 172.21.15.67:0/3662467884 shutdown_connections 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 --2- 172.21.15.67:0/3662467884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0af8a0 0x7f596c0afd30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 --2- 172.21.15.67:0/3662467884 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c0c5290 0x7f596c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 -- 172.21.15.67:0/3662467884 >> 172.21.15.67:0/3662467884 conn(0x7f596c0ac230 msgr2=0x7f596c0ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 -- 172.21.15.67:0/3662467884 shutdown_connections 2024-02-26T10:42:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.137+0000 7f596be00640 1 -- 172.21.15.67:0/3662467884 wait complete. 2024-02-26T10:42:50.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.138+0000 7f596be00640 1 Processor -- start 2024-02-26T10:42:50.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 -- start start 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f596c0af8a0 0x7f596c1eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 0x7f596c1eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c155880 0x7f596c0c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c155cc0 con 0x7f596c0c5290 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c06c3d0 con 0x7f596c0af8a0 2024-02-26T10:42:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.139+0000 7f596be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0c5670 con 0x7f596c155880 2024-02-26T10:42:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 0x7f596c1eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 0x7f596c1eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34658/0 (socket says 172.21.15.67:34658) 2024-02-26T10:42:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 -- 172.21.15.67:0/3068961208 learned_addr learned my addr 172.21.15.67:0/3068961208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596b400640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c155880 0x7f596c0c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596aa00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f596c0af8a0 0x7f596c1eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 -- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c155880 msgr2=0x7f596c0c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c155880 0x7f596c0c06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 -- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f596c0af8a0 msgr2=0x7f596c1eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f596c0af8a0 0x7f596c1eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596a000640 1 -- 172.21.15.67:0/3068961208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c1eb010 con 0x7f596c0c5290 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596b400640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c155880 0x7f596c0c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.140+0000 7f596aa00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f596c0af8a0 0x7f596c1eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.141+0000 7f596a000640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 0x7f596c1eb530 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f595400c3c0 tx=0x7f595400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.141+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540052d0 con 0x7f596c0c5290 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.141+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954005490 con 0x7f596c0c5290 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.141+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954007260 con 0x7f596c0c5290 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.141+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c1eba70 con 0x7f596c0c5290 2024-02-26T10:42:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.141+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c0bec30 con 0x7f596c0c5290 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.142+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c074a30 con 0x7f596c0c5290 2024-02-26T10:42:50.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.143+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5954007400 con 0x7f596c0c5290 2024-02-26T10:42:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.145+0000 7f5963400640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f594806b4d0 0x7f594806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.145+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f59540899b0 con 0x7f596c0c5290 2024-02-26T10:42:50.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.145+0000 7f596aa00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f594806b4d0 0x7f594806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.146+0000 7f596aa00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f594806b4d0 0x7f594806d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f595c006fd0 tx=0x7f595c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.150+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5954058560 con 0x7f596c0c5290 2024-02-26T10:42:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.349+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f596c06d950 con 0x7f596c0c5290 2024-02-26T10:42:50.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.350+0000 7f5963400640 1 -- 172.21.15.67:0/3068961208 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7f5954057e60 con 0x7f596c0c5290 2024-02-26T10:42:50.348 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:50.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.352+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f594806b4d0 msgr2=0x7f594806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.352+0000 7f596be00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f594806b4d0 0x7f594806d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f595c006fd0 tx=0x7f595c008040 comp rx=0 tx=0).stop 2024-02-26T10:42:50.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.352+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 msgr2=0x7f596c1eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.352+0000 7f596be00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 0x7f596c1eb530 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f595400c3c0 tx=0x7f595400c880 comp rx=0 tx=0).stop 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 shutdown_connections 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f596c155880 0x7f596c0c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c0c5290 0x7f596c1eb530 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f594806b4d0 0x7f594806d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 --2- 172.21.15.67:0/3068961208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f596c0af8a0 0x7f596c1eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 >> 172.21.15.67:0/3068961208 conn(0x7f596c0ac230 msgr2=0x7f596c1561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 shutdown_connections 2024-02-26T10:42:50.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.353+0000 7f596be00640 1 -- 172.21.15.67:0/3068961208 wait complete. 2024-02-26T10:42:50.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:42:50.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.599+0000 7f3928200640 1 Processor -- start 2024-02-26T10:42:50.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.599+0000 7f3928200640 1 -- start start 2024-02-26T10:42:50.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.600+0000 7f3928200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f392006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.600+0000 7f3928200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f392006e390 0x7f39201533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.600+0000 7f3928200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39200a6eb0 con 0x7f39200c2950 2024-02-26T10:42:50.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.600+0000 7f3928200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200a7190 con 0x7f392006e390 2024-02-26T10:42:50.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.600+0000 7f3928200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200a7470 con 0x7f39201503d0 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.601+0000 7f3926200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f39200c2950 0x7f3920156320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49542/0 (socket says 172.21.15.67:49542) 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.601+0000 7f3926200640 1 -- 172.21.15.67:0/3664404207 learned_addr learned my addr 172.21.15.67:0/3664404207 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.601+0000 7f3924e00640 1 --2- 172.21.15.67:0/3664404207 >> v2:172.21.15.82:3300/0 conn(0x7f392006e390 0x7f39201533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.601+0000 7f3925800640 1 --2- 172.21.15.67:0/3664404207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f392006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.601+0000 7f391fe00640 1 -- 172.21.15.67:0/3664404207 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330246501 0 0) 0x7f39200a6eb0 con 0x7f39200c2950 2024-02-26T10:42:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.601+0000 7f391fe00640 1 -- 172.21.15.67:0/3664404207 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3904003520 con 0x7f39200c2950 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.602+0000 7f3925800640 1 -- 172.21.15.67:0/3664404207 >> v2:172.21.15.82:3300/0 conn(0x7f392006e390 msgr2=0x7f39201533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.602+0000 7f3925800640 1 --2- 172.21.15.67:0/3664404207 >> v2:172.21.15.82:3300/0 conn(0x7f392006e390 0x7f39201533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.602+0000 7f3925800640 1 -- 172.21.15.67:0/3664404207 >> v1:172.21.15.67:6789/0 conn(0x7f39200c2950 legacy=0x7f3920156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.602+0000 7f3925800640 1 -- 172.21.15.67:0/3664404207 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200a7750 con 0x7f39201503d0 2024-02-26T10:42:50.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.603+0000 7f3925800640 1 --2- 172.21.15.67:0/3664404207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f392006de50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3910009d90 tx=0x7f391002bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a2978102892e31 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f391fe00640 1 -- 172.21.15.67:0/3664404207 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910039070 con 0x7f39201503d0 2024-02-26T10:42:50.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f391fe00640 1 -- 172.21.15.67:0/3664404207 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391002cb30 con 0x7f39201503d0 2024-02-26T10:42:50.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f391fe00640 1 -- 172.21.15.67:0/3664404207 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910034440 con 0x7f39201503d0 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 -- 172.21.15.67:0/3664404207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 msgr2=0x7f392006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 --2- 172.21.15.67:0/3664404207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f392006de50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3910009d90 tx=0x7f391002bd80 comp rx=0 tx=0).stop 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 -- 172.21.15.67:0/3664404207 shutdown_connections 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 --2- 172.21.15.67:0/3664404207 >> v2:172.21.15.82:3300/0 conn(0x7f392006e390 0x7f39201533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 --2- 172.21.15.67:0/3664404207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f392006de50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 -- 172.21.15.67:0/3664404207 >> 172.21.15.67:0/3664404207 conn(0x7f39200c6da0 msgr2=0x7f39200c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 -- 172.21.15.67:0/3664404207 shutdown_connections 2024-02-26T10:42:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.604+0000 7f3928200640 1 -- 172.21.15.67:0/3664404207 wait complete. 2024-02-26T10:42:50.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.606+0000 7f3928200640 1 Processor -- start 2024-02-26T10:42:50.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.606+0000 7f3928200640 1 -- start start 2024-02-26T10:42:50.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3928200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 0x7f3920148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3928200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39200c2950 0x7f3920156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3928200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f3920156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3928200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39201507b0 con 0x7f392006e390 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3928200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392006ef00 con 0x7f39200c2950 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3928200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200c0ef0 con 0x7f39201503d0 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.607+0000 7f3925800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 0x7f3920148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 0x7f3920148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34674/0 (socket says 172.21.15.67:34674) 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 -- 172.21.15.67:0/4234536235 learned_addr learned my addr 172.21.15.67:0/4234536235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3926200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f3920156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 -- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 msgr2=0x7f3920156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f3920156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 -- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39200c2950 msgr2=0x7f3920156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3924e00640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39200c2950 0x7f3920156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39200c2950 0x7f3920156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 -- 172.21.15.67:0/4234536235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3920153aa0 con 0x7f392006e390 2024-02-26T10:42:50.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3926200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f3920156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:50.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.608+0000 7f3925800640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 0x7f3920148b10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3910009eb0 tx=0x7f3910004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.609+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910039050 con 0x7f392006e390 2024-02-26T10:42:50.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.609+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910045070 con 0x7f392006e390 2024-02-26T10:42:50.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.609+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910009450 con 0x7f392006e390 2024-02-26T10:42:50.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.609+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392014d220 con 0x7f392006e390 2024-02-26T10:42:50.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.609+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3920149050 con 0x7f392006e390 2024-02-26T10:42:50.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.610+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3920074a30 con 0x7f392006e390 2024-02-26T10:42:50.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.611+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f391003e850 con 0x7f392006e390 2024-02-26T10:42:50.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.612+0000 7f391d600640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:50.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.612+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f39100ad210 con 0x7f392006e390 2024-02-26T10:42:50.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.613+0000 7f3924e00640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:50.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.613+0000 7f3924e00640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f390006b4d0 0x7f390006d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f392014dcc0 tx=0x7f390c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:50.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.617+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f391007bc90 con 0x7f392006e390 2024-02-26T10:42:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.818+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f392006ec50 con 0x7f392006e390 2024-02-26T10:42:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.819+0000 7f391d600640 1 -- 172.21.15.67:0/4234536235 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7f391007b590 con 0x7f392006e390 2024-02-26T10:42:50.817 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:50.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.821+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f390006b4d0 msgr2=0x7f390006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f390006b4d0 0x7f390006d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f392014dcc0 tx=0x7f390c008040 comp rx=0 tx=0).stop 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 msgr2=0x7f3920148b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 0x7f3920148b10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3910009eb0 tx=0x7f3910004a40 comp rx=0 tx=0).stop 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 shutdown_connections 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39201503d0 0x7f3920156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39200c2950 0x7f3920156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 --2- 172.21.15.67:0/4234536235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f392006e390 0x7f3920148b10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 >> 172.21.15.67:0/4234536235 conn(0x7f39200c6da0 msgr2=0x7f39200a5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 shutdown_connections 2024-02-26T10:42:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:50.822+0000 7f3928200640 1 -- 172.21.15.67:0/4234536235 wait complete. 2024-02-26T10:42:50.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-26T10:42:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.058+0000 7f8ea3200640 1 Processor -- start 2024-02-26T10:42:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.058+0000 7f8ea3200640 1 -- start start 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e9c06ac20 0x7f8e9c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 0x7f8e9c1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9c0c1e20 con 0x7f8e9c0c12e0 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9c15a090 con 0x7f8e9c06ac20 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c159ad0 con 0x7f8e9c0c19b0 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea1400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8e9c0c12e0 0x7f8e9c06e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49546/0 (socket says 172.21.15.67:49546) 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.060+0000 7f8ea1400640 1 -- 172.21.15.67:0/1610872408 learned_addr learned my addr 172.21.15.67:0/1610872408 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea1e00640 1 --2- 172.21.15.67:0/1610872408 >> v2:172.21.15.67:3300/0 conn(0x7f8e9c06ac20 0x7f8e9c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea0a00640 1 -- 172.21.15.67:0/1610872408 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734511184 0 0) 0x7f8e9c0c1e20 con 0x7f8e9c0c12e0 2024-02-26T10:42:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea0a00640 1 -- 172.21.15.67:0/1610872408 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e84003520 con 0x7f8e9c0c12e0 2024-02-26T10:42:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea1e00640 1 -- 172.21.15.67:0/1610872408 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 msgr2=0x7f8e9c1590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea1e00640 1 --2- 172.21.15.67:0/1610872408 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 0x7f8e9c1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea1e00640 1 -- 172.21.15.67:0/1610872408 >> v1:172.21.15.67:6789/0 conn(0x7f8e9c0c12e0 legacy=0x7f8e9c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.061+0000 7f8ea1e00640 1 -- 172.21.15.67:0/1610872408 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c159db0 con 0x7f8e9c06ac20 2024-02-26T10:42:51.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.062+0000 7f8ea1e00640 1 --2- 172.21.15.67:0/1610872408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c06b000 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8e900097a0 tx=0x7f8e9002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ccd3ba9f90fbb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:51.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea0a00640 1 -- 172.21.15.67:0/1610872408 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90038070 con 0x7f8e9c06ac20 2024-02-26T10:42:51.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea0a00640 1 -- 172.21.15.67:0/1610872408 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90031700 con 0x7f8e9c06ac20 2024-02-26T10:42:51.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea0a00640 1 -- 172.21.15.67:0/1610872408 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90031ad0 con 0x7f8e9c06ac20 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 -- 172.21.15.67:0/1610872408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 msgr2=0x7f8e9c06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 --2- 172.21.15.67:0/1610872408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c06b000 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8e900097a0 tx=0x7f8e9002b7a0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 -- 172.21.15.67:0/1610872408 shutdown_connections 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 --2- 172.21.15.67:0/1610872408 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 0x7f8e9c1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 --2- 172.21.15.67:0/1610872408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c06b000 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 -- 172.21.15.67:0/1610872408 >> 172.21.15.67:0/1610872408 conn(0x7f8e9c0c63f0 msgr2=0x7f8e9c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:51.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 -- 172.21.15.67:0/1610872408 shutdown_connections 2024-02-26T10:42:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.063+0000 7f8ea3200640 1 -- 172.21.15.67:0/1610872408 wait complete. 2024-02-26T10:42:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.065+0000 7f8ea3200640 1 Processor -- start 2024-02-26T10:42:51.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.065+0000 7f8ea3200640 1 -- start start 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e9c0c12e0 0x7f8e9c1e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 0x7f8e9c1e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1534d0 con 0x7f8e9c06ac20 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c06df10 con 0x7f8e9c0c12e0 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c159630 con 0x7f8e9c0c19b0 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34690/0 (socket says 172.21.15.67:34690) 2024-02-26T10:42:51.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.066+0000 7f8ea1e00640 1 -- 172.21.15.67:0/705012637 learned_addr learned my addr 172.21.15.67:0/705012637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:51.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea1e00640 1 -- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 msgr2=0x7f8e9c1e3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:51.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea1e00640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 0x7f8e9c1e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea1e00640 1 -- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e9c0c12e0 msgr2=0x7f8e9c1e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:51.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea1e00640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e9c0c12e0 0x7f8e9c1e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea1e00640 1 -- 172.21.15.67:0/705012637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c1e4020 con 0x7f8e9c06ac20 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea1e00640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c149ac0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8e900098c0 tx=0x7f8e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90043070 con 0x7f8e9c06ac20 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9003f890 con 0x7f8e9c06ac20 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90038040 con 0x7f8e9c06ac20 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.067+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c15d4e0 con 0x7f8e9c06ac20 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.068+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c1e50d0 con 0x7f8e9c06ac20 2024-02-26T10:42:51.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.069+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c074a30 con 0x7f8e9c06ac20 2024-02-26T10:42:51.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.069+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e90046070 con 0x7f8e9c06ac20 2024-02-26T10:42:51.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.070+0000 7f8e96000640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e8006b4d0 0x7f8e8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:51.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.070+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8e900ac360 con 0x7f8e9c06ac20 2024-02-26T10:42:51.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.071+0000 7f8ea1400640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e8006b4d0 0x7f8e8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:51.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.072+0000 7f8ea1400640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e8006b4d0 0x7f8e8006d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000f60 tx=0x7f8e8c0078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:51.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.075+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e9c1e4020 con 0x7f8e9c06ac20 2024-02-26T10:42:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.290+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8e9c0c0460 con 0x7f8e9c06ac20 2024-02-26T10:42:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.290+0000 7f8e96000640 1 -- 172.21.15.67:0/705012637 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8e90030070 con 0x7f8e9c06ac20 2024-02-26T10:42:51.288 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e8006b4d0 msgr2=0x7f8e8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e8006b4d0 0x7f8e8006d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000f60 tx=0x7f8e8c0078f0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 msgr2=0x7f8e9c149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c149ac0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8e900098c0 tx=0x7f8e900046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 shutdown_connections 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e9c0c19b0 0x7f8e9c1e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8e8006b4d0 0x7f8e8006d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e9c0c12e0 0x7f8e9c1e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 --2- 172.21.15.67:0/705012637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9c06ac20 0x7f8e9c149ac0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 >> 172.21.15.67:0/705012637 conn(0x7f8e9c0c63f0 msgr2=0x7f8e9c14b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 shutdown_connections 2024-02-26T10:42:51.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:51.293+0000 7f8ea3200640 1 -- 172.21.15.67:0/705012637 wait complete. 2024-02-26T10:42:51.310 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.testuser... 2024-02-26T10:42:51.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:51.311 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-26T10:42:51.423 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:42:51.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:42:51.423 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-26T10:42:51.476 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-26T10:42:51.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:42:51.603 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-26T10:42:51.726 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-26T10:42:51.726 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:42:51.726 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:51.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:42:51.787 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:42:51.787 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:42:51.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:42:51.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:52.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.075+0000 7f5f37e00640 1 Processor -- start 2024-02-26T10:42:52.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.075+0000 7f5f37e00640 1 -- start start 2024-02-26T10:42:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f37e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f3806b000 0x7f5f3806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f37e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380a9a80 con 0x7f5f3806ac20 2024-02-26T10:42:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f37e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380a9d60 con 0x7f5f380a94a0 2024-02-26T10:42:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f37e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3806b470 con 0x7f5f3806b000 2024-02-26T10:42:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f36a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5f3806ac20 0x7f5f38153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49560/0 (socket says 172.21.15.67:49560) 2024-02-26T10:42:52.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f36a00640 1 -- 172.21.15.67:0/2593060747 learned_addr learned my addr 172.21.15.67:0/2593060747 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215227576 0 0) 0x7f5f380a9a80 con 0x7f5f3806ac20 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f1c003520 con 0x7f5f3806ac20 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f37400640 1 --2- 172.21.15.67:0/2593060747 >> v2:172.21.15.82:3300/0 conn(0x7f5f3806b000 0x7f5f3806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747603504 0 0) 0x7f5f1c003520 con 0x7f5f3806ac20 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 >> v2:172.21.15.82:3300/0 conn(0x7f5f3806b000 msgr2=0x7f5f3806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 --2- 172.21.15.67:0/2593060747 >> v2:172.21.15.82:3300/0 conn(0x7f5f3806b000 0x7f5f3806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 >> v1:172.21.15.82:6789/0 conn(0x7f5f380a94a0 legacy=0x7f5f380abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380ab750 con 0x7f5f3806ac20 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f5f28001670 con 0x7f5f3806ac20 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 >> v1:172.21.15.67:6789/0 conn(0x7f5f3806ac20 legacy=0x7f5f38153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c004800 0x7f5f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f1c005210 0x7f5f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3806b470 con 0x7f5f1c003a00 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00c160 con 0x7f5f1c004800 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00c360 con 0x7f5f1c005210 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.077+0000 7f5f36a00640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36000640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c004800 0x7f5f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f37400640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f1c005210 0x7f5f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36a00640 1 -- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f1c005210 msgr2=0x7f5f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36a00640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f1c005210 0x7f5f1c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36a00640 1 -- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c004800 msgr2=0x7f5f1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36a00640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c004800 0x7f5f1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36a00640 1 -- 172.21.15.67:0/2593060747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c003520 con 0x7f5f1c003a00 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f36000640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c004800 0x7f5f1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.078+0000 7f5f37400640 1 -- 172.21.15.67:0/2593060747 reap_dead start 2024-02-26T10:42:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.079+0000 7f5f36a00640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f1c003e20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800c110 tx=0x7f5f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b400e1227b78814d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.079+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28007c40 con 0x7f5f1c003a00 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.079+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28030070 con 0x7f5f1c003a00 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.079+0000 7f5f35600640 1 -- 172.21.15.67:0/2593060747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28005d70 con 0x7f5f1c003a00 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 -- 172.21.15.67:0/2593060747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 msgr2=0x7f5f1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f1c003e20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800c110 tx=0x7f5f1c005830 comp rx=0 tx=0).stop 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 -- 172.21.15.67:0/2593060747 shutdown_connections 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 --2- 172.21.15.67:0/2593060747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f1c003e20 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 -- 172.21.15.67:0/2593060747 >> 172.21.15.67:0/2593060747 conn(0x7f5f380c63f0 msgr2=0x7f5f380c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 -- 172.21.15.67:0/2593060747 shutdown_connections 2024-02-26T10:42:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.080+0000 7f5f37e00640 1 -- 172.21.15.67:0/2593060747 wait complete. 2024-02-26T10:42:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.082+0000 7f5f37e00640 1 Processor -- start 2024-02-26T10:42:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.082+0000 7f5f37e00640 1 -- start start 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f3814abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f37e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f380c56e0 0x7f5f381444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f37e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f38071ae0 0x7f5f381476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f37e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38150590 con 0x7f5f1c003a00 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f37e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38150410 con 0x7f5f38071ae0 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f37e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3806db00 con 0x7f5f380c56e0 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f3814abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f3814abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34702/0 (socket says 172.21.15.67:34702) 2024-02-26T10:42:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.083+0000 7f5f36a00640 1 -- 172.21.15.67:0/3548019488 learned_addr learned my addr 172.21.15.67:0/3548019488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f37400640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f38071ae0 0x7f5f381476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36000640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f380c56e0 0x7f5f381444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36a00640 1 -- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f380c56e0 msgr2=0x7f5f381444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36a00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f380c56e0 0x7f5f381444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36a00640 1 -- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f38071ae0 msgr2=0x7f5f381476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36a00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f38071ae0 0x7f5f381476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36a00640 1 -- 172.21.15.67:0/3548019488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38072cb0 con 0x7f5f1c003a00 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f37400640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f38071ae0 0x7f5f381476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36000640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f380c56e0 0x7f5f381444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f36a00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f3814abd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5f2802c450 tx=0x7f5f2802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2802cbf0 con 0x7f5f1c003a00 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28000ed0 con 0x7f5f1c003a00 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28007070 con 0x7f5f1c003a00 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.084+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f38145940 con 0x7f5f1c003a00 2024-02-26T10:42:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.085+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f380729d0 con 0x7f5f1c003a00 2024-02-26T10:42:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.086+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38074a30 con 0x7f5f1c003a00 2024-02-26T10:42:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.086+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f28048020 con 0x7f5f1c003a00 2024-02-26T10:42:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.087+0000 7f5f2ea00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f1406b500 0x7f5f1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.087+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f5f280ab950 con 0x7f5f1c003a00 2024-02-26T10:42:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.088+0000 7f5f36000640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f1406b500 0x7f5f1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.089+0000 7f5f36000640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f1406b500 0x7f5f1406d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5f38151820 tx=0x7f5f20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:52.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.093+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f2807a500 con 0x7f5f1c003a00 2024-02-26T10:42:52.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.332+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f38074a30 con 0x7f5f1c003a00 2024-02-26T10:42:52.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.334+0000 7f5f2ea00640 1 -- 172.21.15.67:0/3548019488 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6185 (secure 0 0 0) 0x7f5f28034e80 con 0x7f5f1c003a00 2024-02-26T10:42:52.332 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:52.332 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:42:52.332 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 53 2024-02-26T10:42:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f1406b500 msgr2=0x7f5f1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f1406b500 0x7f5f1406d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5f38151820 tx=0x7f5f20009040 comp rx=0 tx=0).stop 2024-02-26T10:42:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 msgr2=0x7f5f3814abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f3814abd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5f2802c450 tx=0x7f5f2802e070 comp rx=0 tx=0).stop 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 shutdown_connections 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5f1406b500 0x7f5f1406d9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f38071ae0 0x7f5f381476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f380c56e0 0x7f5f381444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 --2- 172.21.15.67:0/3548019488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f1c003a00 0x7f5f3814abd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 >> 172.21.15.67:0/3548019488 conn(0x7f5f380c63f0 msgr2=0x7f5f380abda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:52.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 shutdown_connections 2024-02-26T10:42:52.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.337+0000 7f5f37e00640 1 -- 172.21.15.67:0/3548019488 wait complete. 2024-02-26T10:42:52.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:52.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.570+0000 7fb0e6000640 1 Processor -- start 2024-02-26T10:42:52.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.570+0000 7fb0e6000640 1 -- start start 2024-02-26T10:42:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0e0158cd0 0x7fb0e006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e006fb80 0x7fb0e015d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e6000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00c2ce0 con 0x7fb0e006f250 2024-02-26T10:42:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00c3d60 con 0x7fb0e0158cd0 2024-02-26T10:42:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015d540 con 0x7fb0e006fb80 2024-02-26T10:42:52.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0e0158cd0 0x7fb0e006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0e0158cd0 0x7fb0e006ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34704/0 (socket says 172.21.15.67:34704) 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3116761973 learned_addr learned my addr 172.21.15.67:0/3116761973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.572+0000 7fb0dfe00640 1 --1- 172.21.15.67:0/3116761973 >> v1:172.21.15.67:6789/0 conn(0x7fb0e006f250 0x7fb0e015c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49572/0 (socket says 172.21.15.67:49572) 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.573+0000 7fb0e5600640 1 --2- 172.21.15.67:0/3116761973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e006fb80 0x7fb0e015d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.573+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3116761973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e006fb80 msgr2=0x7fb0e015d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.573+0000 7fb0e4c00640 1 --2- 172.21.15.67:0/3116761973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e006fb80 0x7fb0e015d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.573+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3116761973 >> v1:172.21.15.67:6789/0 conn(0x7fb0e006f250 legacy=0x7fb0e015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.573+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3116761973 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00c37a0 con 0x7fb0e0158cd0 2024-02-26T10:42:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.574+0000 7fb0e4c00640 1 --2- 172.21.15.67:0/3116761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e0158cd0 0x7fb0e006ed10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009870 tx=0x7fb0d002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44775728dc4404c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.574+0000 7fb0df400640 1 -- 172.21.15.67:0/3116761973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00317c0 con 0x7fb0e0158cd0 2024-02-26T10:42:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.574+0000 7fb0df400640 1 -- 172.21.15.67:0/3116761973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0037070 con 0x7fb0e0158cd0 2024-02-26T10:42:52.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.574+0000 7fb0df400640 1 -- 172.21.15.67:0/3116761973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0031b80 con 0x7fb0e0158cd0 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 -- 172.21.15.67:0/3116761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e0158cd0 msgr2=0x7fb0e006ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3116761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e0158cd0 0x7fb0e006ed10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009870 tx=0x7fb0d002b860 comp rx=0 tx=0).stop 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 -- 172.21.15.67:0/3116761973 shutdown_connections 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3116761973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e006fb80 0x7fb0e015d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3116761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e0158cd0 0x7fb0e006ed10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 -- 172.21.15.67:0/3116761973 >> 172.21.15.67:0/3116761973 conn(0x7fb0e006e5d0 msgr2=0x7fb0e00c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 -- 172.21.15.67:0/3116761973 shutdown_connections 2024-02-26T10:42:52.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.575+0000 7fb0e6000640 1 -- 172.21.15.67:0/3116761973 wait complete. 2024-02-26T10:42:52.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.577+0000 7fb0e6000640 1 Processor -- start 2024-02-26T10:42:52.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.577+0000 7fb0e6000640 1 -- start start 2024-02-26T10:42:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 0x7fb0e00a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e6000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0e006fb80 0x7fb0e0147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e0158cd0 0x7fb0e01479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e006b030 con 0x7fb0e006f250 2024-02-26T10:42:52.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e6000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e006ad50 con 0x7fb0e006fb80 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e01590b0 con 0x7fb0e0158cd0 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 0x7fb0e00a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 0x7fb0e00a9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34720/0 (socket says 172.21.15.67:34720) 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.578+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3177873402 learned_addr learned my addr 172.21.15.67:0/3177873402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e0158cd0 msgr2=0x7fb0e01479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e5600640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e0158cd0 0x7fb0e01479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0dfe00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0e006fb80 0x7fb0e0147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e4c00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e0158cd0 0x7fb0e01479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0e006fb80 msgr2=0x7fb0e0147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e4c00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0e006fb80 0x7fb0e0147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e4c00640 1 -- 172.21.15.67:0/3177873402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e01e5b30 con 0x7fb0e006f250 2024-02-26T10:42:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e4c00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 0x7fb0e00a9970 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb0d002b8a0 tx=0x7fb0d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0032650 con 0x7fb0e006f250 2024-02-26T10:42:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0032810 con 0x7fb0e006f250 2024-02-26T10:42:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0032c00 con 0x7fb0e006f250 2024-02-26T10:42:52.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.579+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e01e6750 con 0x7fb0e006f250 2024-02-26T10:42:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.580+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e01f0120 con 0x7fb0e006f250 2024-02-26T10:42:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.581+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0074a30 con 0x7fb0e006f250 2024-02-26T10:42:52.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.582+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0d004c060 con 0x7fb0e006f250 2024-02-26T10:42:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.582+0000 7fb0dcc00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0bc06b500 0x7fb0bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.583+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb0d00abff0 con 0x7fb0e006f250 2024-02-26T10:42:52.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.583+0000 7fb0dfe00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0bc06b500 0x7fb0bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:52.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.584+0000 7fb0dfe00640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0bc06b500 0x7fb0bc06d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0156390 tx=0x7fb0c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:52.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.587+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0d0040070 con 0x7fb0e006f250 2024-02-26T10:42:52.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.796+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0e00c1f00 con 0x7fb0e006f250 2024-02-26T10:42:52.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.799+0000 7fb0dcc00640 1 -- 172.21.15.67:0/3177873402 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7fb0d007ab40 con 0x7fb0e006f250 2024-02-26T10:42:52.796 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:52.796 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:52.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0bc06b500 msgr2=0x7fb0bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0bc06b500 0x7fb0bc06d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0156390 tx=0x7fb0c8009040 comp rx=0 tx=0).stop 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 msgr2=0x7fb0e00a9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 0x7fb0e00a9970 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb0d002b8a0 tx=0x7fb0d00046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 shutdown_connections 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0e0158cd0 0x7fb0e01479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0bc06b500 0x7fb0bc06d9c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0e006fb80 0x7fb0e0147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 --2- 172.21.15.67:0/3177873402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0e006f250 0x7fb0e00a9970 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 >> 172.21.15.67:0/3177873402 conn(0x7fb0e006e5d0 msgr2=0x7fb0e0156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 shutdown_connections 2024-02-26T10:42:52.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:52.802+0000 7fb0e6000640 1 -- 172.21.15.67:0/3177873402 wait complete. 2024-02-26T10:42:52.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-26T10:41:57.123912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-02-26T10:41:57.744647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:42:52.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:42:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.042+0000 7f921c600640 1 Processor -- start 2024-02-26T10:42:53.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.043+0000 7f921c600640 1 -- start start 2024-02-26T10:42:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.044+0000 7f921c600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92141503d0 0x7f921406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.044+0000 7f921c600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92140a6eb0 con 0x7f9214153af0 2024-02-26T10:42:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.044+0000 7f921c600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92140a7190 con 0x7f921406e390 2024-02-26T10:42:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.044+0000 7f921c600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92140a7470 con 0x7f92141503d0 2024-02-26T10:42:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.045+0000 7f921a600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9214153af0 0x7f921415d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49574/0 (socket says 172.21.15.67:49574) 2024-02-26T10:42:53.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.045+0000 7f921a600640 1 -- 172.21.15.67:0/2085714969 learned_addr learned my addr 172.21.15.67:0/2085714969 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.045+0000 7f9219c00640 1 --2- 172.21.15.67:0/2085714969 >> v2:172.21.15.82:3300/0 conn(0x7f92141503d0 0x7f921406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.045+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492632175 0 0) 0x7f92140a6eb0 con 0x7f9214153af0 2024-02-26T10:42:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.045+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f8003520 con 0x7f9214153af0 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855332599 0 0) 0x7f91f8003520 con 0x7f9214153af0 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 >> v2:172.21.15.82:3300/0 conn(0x7f92141503d0 msgr2=0x7f921406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 --2- 172.21.15.67:0/2085714969 >> v2:172.21.15.82:3300/0 conn(0x7f92141503d0 0x7f921406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 >> v1:172.21.15.82:6789/0 conn(0x7f921406e390 legacy=0x7f92141533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140a7750 con 0x7f9214153af0 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f920c001670 con 0x7f9214153af0 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 >> v1:172.21.15.67:6789/0 conn(0x7f9214153af0 legacy=0x7f921415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8003a00 0x7f91f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91f8004800 0x7f91f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f80052e0 0x7f91f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92140a7470 con 0x7f91f8003a00 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f800c1e0 con 0x7f91f80052e0 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f800c3e0 con 0x7f91f8004800 2024-02-26T10:42:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9219c00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8003a00 0x7f91f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f921a600640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f80052e0 0x7f91f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9219c00640 1 -- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91f8004800 msgr2=0x7f91f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9219c00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91f8004800 0x7f91f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9219c00640 1 -- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f80052e0 msgr2=0x7f91f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9219c00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f80052e0 0x7f91f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f9219c00640 1 -- 172.21.15.67:0/2085714969 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8003520 con 0x7f91f8003a00 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f921a600640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f80052e0 0x7f91f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:53.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.046+0000 7f921a600640 1 -- 172.21.15.67:0/2085714969 reap_dead start 2024-02-26T10:42:53.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.047+0000 7f9219c00640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8003a00 0x7f91f8003e20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fc0 tx=0x7f91f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d7089ded94c58f server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204039070 con 0x7f91f8003a00 2024-02-26T10:42:53.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920402cc20 con 0x7f91f8003a00 2024-02-26T10:42:53.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f9213e00640 1 -- 172.21.15.67:0/2085714969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204009c10 con 0x7f91f8003a00 2024-02-26T10:42:53.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f921c600640 1 -- 172.21.15.67:0/2085714969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8003a00 msgr2=0x7f91f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f921c600640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8003a00 0x7f91f8003e20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9204009fc0 tx=0x7f91f8005830 comp rx=0 tx=0).stop 2024-02-26T10:42:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f921c600640 1 -- 172.21.15.67:0/2085714969 shutdown_connections 2024-02-26T10:42:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f921c600640 1 --2- 172.21.15.67:0/2085714969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91f8003a00 0x7f91f8003e20 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.048+0000 7f921c600640 1 -- 172.21.15.67:0/2085714969 >> 172.21.15.67:0/2085714969 conn(0x7f92140c6da0 msgr2=0x7f92140c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.049+0000 7f921c600640 1 -- 172.21.15.67:0/2085714969 shutdown_connections 2024-02-26T10:42:53.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.049+0000 7f921c600640 1 -- 172.21.15.67:0/2085714969 wait complete. 2024-02-26T10:42:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.050+0000 7f921c600640 1 Processor -- start 2024-02-26T10:42:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.050+0000 7f921c600640 1 -- start start 2024-02-26T10:42:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.051+0000 7f921c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8003a00 0x7f92141ee060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.051+0000 7f921c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9214070ed0 0x7f9214145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.051+0000 7f921c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 0x7f921413f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.051+0000 7f921c600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92141507b0 con 0x7f92140c5b40 2024-02-26T10:42:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.051+0000 7f921c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921406ef00 con 0x7f91f8003a00 2024-02-26T10:42:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.051+0000 7f921c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92140c0ef0 con 0x7f9214070ed0 2024-02-26T10:42:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 0x7f921413f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 0x7f921413f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34738/0 (socket says 172.21.15.67:34738) 2024-02-26T10:42:53.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 -- 172.21.15.67:0/1341354390 learned_addr learned my addr 172.21.15.67:0/1341354390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f9219c00640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8003a00 0x7f92141ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f9219200640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9214070ed0 0x7f9214145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 -- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9214070ed0 msgr2=0x7f9214145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9214070ed0 0x7f9214145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 -- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8003a00 msgr2=0x7f92141ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8003a00 0x7f92141ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f921a600640 1 -- 172.21.15.67:0/1341354390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140c4480 con 0x7f92140c5b40 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.052+0000 7f9219c00640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8003a00 0x7f92141ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.053+0000 7f921a600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 0x7f921413f4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f92140a5fc0 tx=0x7f920c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.053+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c007500 con 0x7f92140c5b40 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.053+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c0047e0 con 0x7f92140c5b40 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.053+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c004b90 con 0x7f92140c5b40 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.053+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92141eb570 con 0x7f92140c5b40 2024-02-26T10:42:53.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.053+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92140c06a0 con 0x7f92140c5b40 2024-02-26T10:42:53.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.054+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92140a6880 con 0x7f92140c5b40 2024-02-26T10:42:53.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.055+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f920c0133d0 con 0x7f92140c5b40 2024-02-26T10:42:53.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.056+0000 7f9211600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f91f406b4d0 0x7f91f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.056+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f920c089c30 con 0x7f92140c5b40 2024-02-26T10:42:53.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.057+0000 7f9219c00640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f91f406b4d0 0x7f91f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.058+0000 7f9219c00640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f91f406b4d0 0x7f91f406d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9204036000 tx=0x7f920402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.061+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92140a6880 con 0x7f92140c5b40 2024-02-26T10:42:53.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.257+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9214074a30 con 0x7f92140c5b40 2024-02-26T10:42:53.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.258+0000 7f9211600640 1 -- 172.21.15.67:0/1341354390 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f920c058740 con 0x7f92140c5b40 2024-02-26T10:42:53.255 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:42:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.261+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f91f406b4d0 msgr2=0x7f91f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.261+0000 7f921c600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f91f406b4d0 0x7f91f406d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9204036000 tx=0x7f920402e000 comp rx=0 tx=0).stop 2024-02-26T10:42:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.261+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 msgr2=0x7f921413f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.261+0000 7f921c600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 0x7f921413f4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f92140a5fc0 tx=0x7f920c00d890 comp rx=0 tx=0).stop 2024-02-26T10:42:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 shutdown_connections 2024-02-26T10:42:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92140c5b40 0x7f921413f4a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9214070ed0 0x7f9214145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f91f406b4d0 0x7f91f406d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 --2- 172.21.15.67:0/1341354390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91f8003a00 0x7f92141ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 >> 172.21.15.67:0/1341354390 conn(0x7f92140c6da0 msgr2=0x7f92140c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 shutdown_connections 2024-02-26T10:42:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.262+0000 7f921c600640 1 -- 172.21.15.67:0/1341354390 wait complete. 2024-02-26T10:42:53.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-26T10:42:53.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.504+0000 7f94c2200640 1 Processor -- start 2024-02-26T10:42:53.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.505+0000 7f94c2200640 1 -- start start 2024-02-26T10:42:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c2200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94bc0c5290 0x7f94bc1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94bc0af8a0 0x7f94bc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c2200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc06c110 con 0x7f94bc155880 2024-02-26T10:42:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc06d000 con 0x7f94bc0af8a0 2024-02-26T10:42:53.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c2200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc0c72c0 con 0x7f94bc0c5290 2024-02-26T10:42:53.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94bc0af8a0 0x7f94bc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94bc0af8a0 0x7f94bc0afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34746/0 (socket says 172.21.15.67:34746) 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.506+0000 7f94c1800640 1 -- 172.21.15.67:0/1215685416 learned_addr learned my addr 172.21.15.67:0/1215685416 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.507+0000 7f94c0e00640 1 --2- 172.21.15.67:0/1215685416 >> v2:172.21.15.82:3300/0 conn(0x7f94bc0c5290 0x7f94bc1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.507+0000 7f94c1800640 1 -- 172.21.15.67:0/1215685416 >> v2:172.21.15.82:3300/0 conn(0x7f94bc0c5290 msgr2=0x7f94bc1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.507+0000 7f94c1800640 1 --2- 172.21.15.67:0/1215685416 >> v2:172.21.15.82:3300/0 conn(0x7f94bc0c5290 0x7f94bc1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.507+0000 7f94c1800640 1 -- 172.21.15.67:0/1215685416 >> v1:172.21.15.82:6789/0 conn(0x7f94bc155880 legacy=0x7f94bc06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.507+0000 7f94c1800640 1 -- 172.21.15.67:0/1215685416 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0c75a0 con 0x7f94bc0af8a0 2024-02-26T10:42:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.508+0000 7f94c1800640 1 --2- 172.21.15.67:0/1215685416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0afd30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0c7080 tx=0x7f94b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47dbcf6640e3d9e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.508+0000 7f94bb400640 1 -- 172.21.15.67:0/1215685416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b402e070 con 0x7f94bc0af8a0 2024-02-26T10:42:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.508+0000 7f94bb400640 1 -- 172.21.15.67:0/1215685416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4037070 con 0x7f94bc0af8a0 2024-02-26T10:42:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.508+0000 7f94bb400640 1 -- 172.21.15.67:0/1215685416 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4031a00 con 0x7f94bc0af8a0 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 -- 172.21.15.67:0/1215685416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0af8a0 msgr2=0x7f94bc0afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 --2- 172.21.15.67:0/1215685416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0afd30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0c7080 tx=0x7f94b402b880 comp rx=0 tx=0).stop 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 -- 172.21.15.67:0/1215685416 shutdown_connections 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 --2- 172.21.15.67:0/1215685416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0afd30 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 --2- 172.21.15.67:0/1215685416 >> v2:172.21.15.82:3300/0 conn(0x7f94bc0c5290 0x7f94bc1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 -- 172.21.15.67:0/1215685416 >> 172.21.15.67:0/1215685416 conn(0x7f94bc0ac230 msgr2=0x7f94bc0ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 -- 172.21.15.67:0/1215685416 shutdown_connections 2024-02-26T10:42:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.509+0000 7f94c2200640 1 -- 172.21.15.67:0/1215685416 wait complete. 2024-02-26T10:42:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.511+0000 7f94c2200640 1 Processor -- start 2024-02-26T10:42:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.511+0000 7f94c2200640 1 -- start start 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94c2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94c2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 0x7f94bc0be640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94c2200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94bc155880 0x7f94bc0bd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94c2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc155cf0 con 0x7f94bc0c5290 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94c2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc06c3d0 con 0x7f94bc0af8a0 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94c2200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0c5670 con 0x7f94bc155880 2024-02-26T10:42:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 0x7f94bc0be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94bbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 0x7f94bc0be640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34758/0 (socket says 172.21.15.67:34758) 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.512+0000 7f94bbe00640 1 -- 172.21.15.67:0/4007222723 learned_addr learned my addr 172.21.15.67:0/4007222723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94c0e00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94c1800640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94bc155880 0x7f94bc0bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94bbe00640 1 -- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94bc155880 msgr2=0x7f94bc0bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94bbe00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94bc155880 0x7f94bc0bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94bbe00640 1 -- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94bc0af8a0 msgr2=0x7f94bc0aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94bbe00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94bbe00640 1 -- 172.21.15.67:0/4007222723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc1eef50 con 0x7f94bc0c5290 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94bbe00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 0x7f94bc0be640 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f94a400ba80 tx=0x7f94a400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a400cc00 con 0x7f94bc0c5290 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a400cda0 con 0x7f94bc0c5290 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4003bb0 con 0x7f94bc0c5290 2024-02-26T10:42:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.513+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc1ef210 con 0x7f94bc0c5290 2024-02-26T10:42:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.514+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc1ef7d0 con 0x7f94bc0c5290 2024-02-26T10:42:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.515+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc0c6970 con 0x7f94bc0c5290 2024-02-26T10:42:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.516+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94a4014440 con 0x7f94bc0c5290 2024-02-26T10:42:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.516+0000 7f94b8c00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949806b4d0 0x7f949806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.517+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f94a408a3f0 con 0x7f94bc0c5290 2024-02-26T10:42:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.517+0000 7f94c0e00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949806b4d0 0x7f949806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.518+0000 7f94c0e00640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949806b4d0 0x7f949806d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f94ac006fd0 tx=0x7f94ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.521+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94bc0c6970 con 0x7f94bc0c5290 2024-02-26T10:42:53.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.720+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94bc074a30 con 0x7f94bc0c5290 2024-02-26T10:42:53.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.721+0000 7f94b8c00640 1 -- 172.21.15.67:0/4007222723 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94a4058fa0 con 0x7f94bc0c5290 2024-02-26T10:42:53.717 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:53.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.723+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949806b4d0 msgr2=0x7f949806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.723+0000 7f94c2200640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949806b4d0 0x7f949806d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f94ac006fd0 tx=0x7f94ac008040 comp rx=0 tx=0).stop 2024-02-26T10:42:53.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.723+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 msgr2=0x7f94bc0be640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.723+0000 7f94c2200640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 0x7f94bc0be640 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f94a400ba80 tx=0x7f94a400bf40 comp rx=0 tx=0).stop 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 shutdown_connections 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94bc155880 0x7f94bc0bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94bc0c5290 0x7f94bc0be640 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f949806b4d0 0x7f949806d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 --2- 172.21.15.67:0/4007222723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94bc0af8a0 0x7f94bc0aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 >> 172.21.15.67:0/4007222723 conn(0x7f94bc0ac230 msgr2=0x7f94bc1547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 shutdown_connections 2024-02-26T10:42:53.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.724+0000 7f94c2200640 1 -- 172.21.15.67:0/4007222723 wait complete. 2024-02-26T10:42:53.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-26T10:42:53.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.965+0000 7f9c0d400640 1 Processor -- start 2024-02-26T10:42:53.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.966+0000 7f9c0d400640 1 -- start start 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.967+0000 7f9c0d400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c081503a0 0x7f9c080ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.967+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c080a9430 0x7f9c081535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.967+0000 7f9c0d400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c08153ae0 con 0x7f9c0806e2d0 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.967+0000 7f9c0d400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c08150780 con 0x7f9c081503a0 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.967+0000 7f9c0d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080a98c0 con 0x7f9c080a9430 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.968+0000 7f9c0ca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9c0806e2d0 0x7f9c080c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49582/0 (socket says 172.21.15.67:49582) 2024-02-26T10:42:53.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.968+0000 7f9c0ca00640 1 -- 172.21.15.67:0/3041526945 learned_addr learned my addr 172.21.15.67:0/3041526945 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.968+0000 7f9bffe00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c080a9430 0x7f9c081535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.968+0000 7f9c07e00640 1 --2- 172.21.15.67:0/3041526945 >> v2:172.21.15.82:3300/0 conn(0x7f9c081503a0 0x7f9c080ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.968+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424728855 0 0) 0x7f9c08153ae0 con 0x7f9c0806e2d0 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.968+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bec003520 con 0x7f9c0806e2d0 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384706285 0 0) 0x7f9bec003520 con 0x7f9c0806e2d0 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c080a9430 msgr2=0x7f9c081535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c080a9430 0x7f9c081535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 >> v2:172.21.15.82:3300/0 conn(0x7f9c081503a0 msgr2=0x7f9c080ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 --2- 172.21.15.67:0/3041526945 >> v2:172.21.15.82:3300/0 conn(0x7f9c081503a0 0x7f9c080ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080a9ba0 con 0x7f9c0806e2d0 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f9c00001a10 con 0x7f9c0806e2d0 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 >> v1:172.21.15.67:6789/0 conn(0x7f9c0806e2d0 legacy=0x7f9c080c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bec0039b0 0x7f9bec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bec004800 0x7f9bec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bec005180 0x7f9bec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08150780 con 0x7f9bec005180 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080a98c0 con 0x7f9bec0039b0 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec00bf00 con 0x7f9bec004800 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c0ca00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bec005180 0x7f9bec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9c07e00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bec0039b0 0x7f9bec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.969+0000 7f9bffe00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bec004800 0x7f9bec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.970+0000 7f9c0ca00640 1 -- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bec004800 msgr2=0x7f9bec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.970+0000 7f9c0ca00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bec004800 0x7f9bec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.970+0000 7f9c0ca00640 1 -- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bec0039b0 msgr2=0x7f9bec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:53.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.970+0000 7f9c0ca00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bec0039b0 0x7f9bec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.970+0000 7f9c0ca00640 1 -- 172.21.15.67:0/3041526945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec003520 con 0x7f9bec005180 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c0ca00640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bec005180 0x7f9bec00b9c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9c08153cb0 tx=0x7f9bec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d83cb927946365e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000a410 con 0x7f9bec005180 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00034650 con 0x7f9bec005180 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c07400640 1 -- 172.21.15.67:0/3041526945 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00034a40 con 0x7f9bec005180 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c0d400640 1 -- 172.21.15.67:0/3041526945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bec005180 msgr2=0x7f9bec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c0d400640 1 --2- 172.21.15.67:0/3041526945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9bec005180 0x7f9bec00b9c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9c08153cb0 tx=0x7f9bec005830 comp rx=0 tx=0).stop 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.971+0000 7f9c0ca00640 1 -- 172.21.15.67:0/3041526945 reap_dead start 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.972+0000 7f9c0d400640 1 -- 172.21.15.67:0/3041526945 shutdown_connections 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.972+0000 7f9c0d400640 1 -- 172.21.15.67:0/3041526945 >> 172.21.15.67:0/3041526945 conn(0x7f9c080c6380 msgr2=0x7f9c080c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.972+0000 7f9c0d400640 1 -- 172.21.15.67:0/3041526945 shutdown_connections 2024-02-26T10:42:53.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.972+0000 7f9c0d400640 1 -- 172.21.15.67:0/3041526945 wait complete. 2024-02-26T10:42:53.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.973+0000 7f9c0d400640 1 Processor -- start 2024-02-26T10:42:53.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 -- start start 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 0x7f9c081561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c08156dd0 0x7f9c081eb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c08153ed0 0x7f9c08144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0806ddb0 con 0x7f9c08146de0 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0806dad0 con 0x7f9c08153ed0 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.974+0000 7f9c0d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080c0f80 con 0x7f9c08156dd0 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 0x7f9c081561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 0x7f9c081561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34770/0 (socket says 172.21.15.67:34770) 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 -- 172.21.15.67:0/1520350087 learned_addr learned my addr 172.21.15.67:0/1520350087 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c0ca00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c08153ed0 0x7f9c08144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9bffe00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c08156dd0 0x7f9c081eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 -- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c08156dd0 msgr2=0x7f9c081eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c08156dd0 0x7f9c081eb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 -- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c08153ed0 msgr2=0x7f9c08144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c08153ed0 0x7f9c08144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 -- 172.21.15.67:0/1520350087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c080c5830 con 0x7f9c08146de0 2024-02-26T10:42:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c0ca00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c08153ed0 0x7f9c08144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.975+0000 7f9c07e00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 0x7f9c081561b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8011730 tx=0x7f9bf8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.976+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800f9c0 con 0x7f9c08146de0 2024-02-26T10:42:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.976+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800fb60 con 0x7f9c08146de0 2024-02-26T10:42:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.976+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8015460 con 0x7f9c08146de0 2024-02-26T10:42:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.976+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c080c5b10 con 0x7f9c08146de0 2024-02-26T10:42:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.976+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08155c90 con 0x7f9c08146de0 2024-02-26T10:42:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.977+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c080678e0 con 0x7f9c08146de0 2024-02-26T10:42:53.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.978+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bf8015600 con 0x7f9c08146de0 2024-02-26T10:42:53.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.979+0000 7f9c04c00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9be406b500 0x7f9be406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:53.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.979+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f9bf808af80 con 0x7f9c08146de0 2024-02-26T10:42:53.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.980+0000 7f9bffe00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9be406b500 0x7f9be406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:53.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.981+0000 7f9bffe00640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9be406b500 0x7f9be406d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9c080c0940 tx=0x7f9bf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:53.983+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bf8019030 con 0x7f9c08146de0 2024-02-26T10:42:54.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.183+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c08074a30 con 0x7f9c08146de0 2024-02-26T10:42:54.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.183+0000 7f9c04c00640 1 -- 172.21.15.67:0/1520350087 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bf8059a90 con 0x7f9c08146de0 2024-02-26T10:42:54.180 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9be406b500 msgr2=0x7f9be406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9be406b500 0x7f9be406d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9c080c0940 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2024-02-26T10:42:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 msgr2=0x7f9c081561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 0x7f9c081561b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8011730 tx=0x7f9bf8011bf0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 shutdown_connections 2024-02-26T10:42:54.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9be406b500 0x7f9be406d9c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c08153ed0 0x7f9c08144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9c08156dd0 0x7f9c081eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 --2- 172.21.15.67:0/1520350087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c08146de0 0x7f9c081561b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.186+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 >> 172.21.15.67:0/1520350087 conn(0x7f9c080c6380 msgr2=0x7f9c080c2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:54.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.187+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 shutdown_connections 2024-02-26T10:42:54.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.187+0000 7f9c0d400640 1 -- 172.21.15.67:0/1520350087 wait complete. 2024-02-26T10:42:54.202 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:42:54.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.430+0000 7f6d55c00640 1 Processor -- start 2024-02-26T10:42:54.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.431+0000 7f6d55c00640 1 -- start start 2024-02-26T10:42:54.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.432+0000 7f6d55c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d500c0ef0 0x7f6d50158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.432+0000 7f6d55c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d50072ce0 con 0x7f6d500c1400 2024-02-26T10:42:54.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.432+0000 7f6d55c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d50155bf0 con 0x7f6d501591b0 2024-02-26T10:42:54.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.432+0000 7f6d55c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d50073010 con 0x7f6d500c0ef0 2024-02-26T10:42:54.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.433+0000 7f6d4f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6d500c1400 0x7f6d5006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49588/0 (socket says 172.21.15.67:49588) 2024-02-26T10:42:54.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.433+0000 7f6d4f400640 1 -- 172.21.15.67:0/1511692750 learned_addr learned my addr 172.21.15.67:0/1511692750 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:54.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.433+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224736976 0 0) 0x7f6d50072ce0 con 0x7f6d500c1400 2024-02-26T10:42:54.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.433+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d34003520 con 0x7f6d500c1400 2024-02-26T10:42:54.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.433+0000 7f6d4e000640 1 --2- 172.21.15.67:0/1511692750 >> v2:172.21.15.82:3300/0 conn(0x7f6d500c0ef0 0x7f6d50158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 929293378 0 0) 0x7f6d34003520 con 0x7f6d500c1400 2024-02-26T10:42:54.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 >> v2:172.21.15.82:3300/0 conn(0x7f6d500c0ef0 msgr2=0x7f6d50158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 --2- 172.21.15.67:0/1511692750 >> v2:172.21.15.82:3300/0 conn(0x7f6d500c0ef0 0x7f6d50158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 >> v1:172.21.15.82:6789/0 conn(0x7f6d501591b0 legacy=0x7f6d50070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50155ed0 con 0x7f6d500c1400 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f6d48001a10 con 0x7f6d500c1400 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 >> v1:172.21.15.67:6789/0 conn(0x7f6d500c1400 legacy=0x7f6d5006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d340039b0 0x7f6d34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d34004800 0x7f6d34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d340052e0 0x7f6d3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50073010 con 0x7f6d34004800 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3400c1b0 con 0x7f6d340039b0 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3400c3b0 con 0x7f6d340052e0 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4e000640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d34004800 0x7f6d34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.434+0000 7f6d4ea00640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d340039b0 0x7f6d34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4e000640 1 -- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d340052e0 msgr2=0x7f6d3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:54.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4f400640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d340052e0 0x7f6d3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4e000640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d340052e0 0x7f6d3400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4e000640 1 -- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d340039b0 msgr2=0x7f6d34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:54.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4e000640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d340039b0 0x7f6d34003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4e000640 1 -- 172.21.15.67:0/1511692750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34003520 con 0x7f6d34004800 2024-02-26T10:42:54.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4ea00640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d340039b0 0x7f6d34003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:54.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.435+0000 7f6d4f400640 1 -- 172.21.15.67:0/1511692750 reap_dead start 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.436+0000 7f6d4e000640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d34004800 0x7f6d34004c80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6d500c1350 tx=0x7f6d34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d790bedd61a9975 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.436+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800b600 con 0x7f6d34004800 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.436+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38038070 con 0x7f6d34004800 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.436+0000 7f6d4d600640 1 -- 172.21.15.67:0/1511692750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38033890 con 0x7f6d34004800 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 -- 172.21.15.67:0/1511692750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d34004800 msgr2=0x7f6d34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d34004800 0x7f6d34004c80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6d500c1350 tx=0x7f6d34005830 comp rx=0 tx=0).stop 2024-02-26T10:42:54.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 -- 172.21.15.67:0/1511692750 shutdown_connections 2024-02-26T10:42:54.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1511692750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d34004800 0x7f6d34004c80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 -- 172.21.15.67:0/1511692750 >> 172.21.15.67:0/1511692750 conn(0x7f6d500c6da0 msgr2=0x7f6d500c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:54.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 -- 172.21.15.67:0/1511692750 shutdown_connections 2024-02-26T10:42:54.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.437+0000 7f6d55c00640 1 -- 172.21.15.67:0/1511692750 wait complete. 2024-02-26T10:42:54.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.439+0000 7f6d55c00640 1 Processor -- start 2024-02-26T10:42:54.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.439+0000 7f6d55c00640 1 -- start start 2024-02-26T10:42:54.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d55c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d34004800 0x7f6d500ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d55c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d5014ced0 0x7f6d500a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d55c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 0x7f6d500aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d55c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50158580 con 0x7f6d5014c980 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d55c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5014f490 con 0x7f6d5014ced0 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d55c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50158700 con 0x7f6d34004800 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 0x7f6d500aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 0x7f6d500aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34788/0 (socket says 172.21.15.67:34788) 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d4f400640 1 -- 172.21.15.67:0/1814991557 learned_addr learned my addr 172.21.15.67:0/1814991557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d4ea00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d34004800 0x7f6d500ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.440+0000 7f6d4e000640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d5014ced0 0x7f6d500a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d4f400640 1 -- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d34004800 msgr2=0x7f6d500ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d4f400640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d34004800 0x7f6d500ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d4f400640 1 -- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d5014ced0 msgr2=0x7f6d500a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d4f400640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d5014ced0 0x7f6d500a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d4f400640 1 -- 172.21.15.67:0/1814991557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500a9430 con 0x7f6d5014c980 2024-02-26T10:42:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d4f400640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 0x7f6d500aa410 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6d48001350 tx=0x7f6d4800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:54.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800bb80 con 0x7f6d5014c980 2024-02-26T10:42:54.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4800bd20 con 0x7f6d5014c980 2024-02-26T10:42:54.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d500aab60 con 0x7f6d5014c980 2024-02-26T10:42:54.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48007350 con 0x7f6d5014c980 2024-02-26T10:42:54.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.441+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d500a96d0 con 0x7f6d5014c980 2024-02-26T10:42:54.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.443+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d50074a30 con 0x7f6d5014c980 2024-02-26T10:42:54.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.443+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d4801b020 con 0x7f6d5014c980 2024-02-26T10:42:54.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.444+0000 7f6d46a00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.444+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f6d4808a7a0 con 0x7f6d5014c980 2024-02-26T10:42:54.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.445+0000 7f6d4ea00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.446+0000 7f6d4ea00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.449+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d48059350 con 0x7f6d5014c980 2024-02-26T10:42:54.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.647+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d50153480 con 0x7f6d5014c980 2024-02-26T10:42:54.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.648+0000 7f6d46a00640 1 -- 172.21.15.67:0/1814991557 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7f6d48058c50 con 0x7f6d5014c980 2024-02-26T10:42:54.644 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6d2c06b4d0 msgr2=0x7f6d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d40009040 comp rx=0 tx=0).stop 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 msgr2=0x7f6d500aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 0x7f6d500aa410 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6d48001350 tx=0x7f6d4800b4a0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 shutdown_connections 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d5014c980 0x7f6d500aa410 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d5014ced0 0x7f6d500a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 --2- 172.21.15.67:0/1814991557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d34004800 0x7f6d500ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 >> 172.21.15.67:0/1814991557 conn(0x7f6d500c6da0 msgr2=0x7f6d50153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 shutdown_connections 2024-02-26T10:42:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.651+0000 7f6d55c00640 1 -- 172.21.15.67:0/1814991557 wait complete. 2024-02-26T10:42:54.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:42:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.901+0000 7f3eeee00640 1 Processor -- start 2024-02-26T10:42:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.902+0000 7f3eeee00640 1 -- start start 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.903+0000 7f3eeee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3ee80c5290 0x7f3ee81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.903+0000 7f3eeee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee806c110 con 0x7f3ee8155880 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.903+0000 7f3eeee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee806d000 con 0x7f3ee80af8a0 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.903+0000 7f3eeee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee80c72c0 con 0x7f3ee80c5290 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.903+0000 7f3eed000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3ee8155880 0x7f3ee806c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49600/0 (socket says 172.21.15.67:49600) 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.903+0000 7f3eed000640 1 -- 172.21.15.67:0/2144353667 learned_addr learned my addr 172.21.15.67:0/2144353667 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.904+0000 7f3eeda00640 1 --2- 172.21.15.67:0/2144353667 >> v2:172.21.15.67:3300/0 conn(0x7f3ee80c5290 0x7f3ee81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.904+0000 7f3ee7e00640 1 -- 172.21.15.67:0/2144353667 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312958368 0 0) 0x7f3ee806c110 con 0x7f3ee8155880 2024-02-26T10:42:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.904+0000 7f3ee7e00640 1 -- 172.21.15.67:0/2144353667 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed0003520 con 0x7f3ee8155880 2024-02-26T10:42:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.904+0000 7f3eeda00640 1 -- 172.21.15.67:0/2144353667 >> v1:172.21.15.82:6789/0 conn(0x7f3ee80af8a0 legacy=0x7f3ee814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.904+0000 7f3eeda00640 1 -- 172.21.15.67:0/2144353667 >> v1:172.21.15.67:6789/0 conn(0x7f3ee8155880 legacy=0x7f3ee806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.904+0000 7f3eeda00640 1 -- 172.21.15.67:0/2144353667 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80c75a0 con 0x7f3ee80c5290 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.905+0000 7f3eeda00640 1 --2- 172.21.15.67:0/2144353667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c5290 0x7f3ee81562f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009d90 tx=0x7f3edc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3302183050fc8724 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3ee7e00640 1 -- 172.21.15.67:0/2144353667 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc039070 con 0x7f3ee80c5290 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3ee7e00640 1 -- 172.21.15.67:0/2144353667 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc02cbd0 con 0x7f3ee80c5290 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3ee7e00640 1 -- 172.21.15.67:0/2144353667 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc034440 con 0x7f3ee80c5290 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3eeee00640 1 -- 172.21.15.67:0/2144353667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c5290 msgr2=0x7f3ee81562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2144353667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c5290 0x7f3ee81562f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009d90 tx=0x7f3edc02be20 comp rx=0 tx=0).stop 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3eeee00640 1 -- 172.21.15.67:0/2144353667 shutdown_connections 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2144353667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee80c5290 0x7f3ee81562f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3eeee00640 1 -- 172.21.15.67:0/2144353667 >> 172.21.15.67:0/2144353667 conn(0x7f3ee80ac230 msgr2=0x7f3ee80ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.906+0000 7f3eeee00640 1 -- 172.21.15.67:0/2144353667 shutdown_connections 2024-02-26T10:42:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.907+0000 7f3eeee00640 1 -- 172.21.15.67:0/2144353667 wait complete. 2024-02-26T10:42:54.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.908+0000 7f3eeee00640 1 Processor -- start 2024-02-26T10:42:54.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.908+0000 7f3eeee00640 1 -- start start 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eeee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee80af8a0 0x7f3ee80bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eeee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee80c5290 0x7f3ee80ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eeee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 0x7f3ee81ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eeee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee8155cf0 con 0x7f3ee8155880 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eeee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee806c3d0 con 0x7f3ee80c5290 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eeee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80c5670 con 0x7f3ee80af8a0 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.909+0000 7f3eee400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 0x7f3ee81ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 0x7f3ee81ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34804/0 (socket says 172.21.15.67:34804) 2024-02-26T10:42:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 -- 172.21.15.67:0/2500527251 learned_addr learned my addr 172.21.15.67:0/2500527251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eeda00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee80af8a0 0x7f3ee80bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eed000640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee80c5290 0x7f3ee80ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 -- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee80af8a0 msgr2=0x7f3ee80bccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee80af8a0 0x7f3ee80bccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 -- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee80c5290 msgr2=0x7f3ee80ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee80c5290 0x7f3ee80ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 -- 172.21.15.67:0/2500527251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee81ee5d0 con 0x7f3ee8155880 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eeda00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee80af8a0 0x7f3ee80bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eed000640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee80c5290 0x7f3ee80ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.910+0000 7f3eee400640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 0x7f3ee81ea530 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c410 tx=0x7f3ee000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.911+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0011790 con 0x7f3ee8155880 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.911+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0017070 con 0x7f3ee8155880 2024-02-26T10:42:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.911+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0011b70 con 0x7f3ee8155880 2024-02-26T10:42:54.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.911+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee81ee870 con 0x7f3ee8155880 2024-02-26T10:42:54.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.911+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee80c0260 con 0x7f3ee8155880 2024-02-26T10:42:54.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.912+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee80bfbc0 con 0x7f3ee8155880 2024-02-26T10:42:54.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.913+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ee0011d10 con 0x7f3ee8155880 2024-02-26T10:42:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.914+0000 7f3ee5600640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.914+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3ee008a310 con 0x7f3ee8155880 2024-02-26T10:42:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.915+0000 7f3eeda00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.916+0000 7f3eeda00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3edc036000 tx=0x7f3edc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:54.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:54.919+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ee0058e20 con 0x7f3ee8155880 2024-02-26T10:42:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.117+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f3ee8074a30 con 0x7f3ee8155880 2024-02-26T10:42:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.118+0000 7f3ee5600640 1 -- 172.21.15.67:0/2500527251 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7f3ee0058720 con 0x7f3ee8155880 2024-02-26T10:42:55.114 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:42:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec806b4d0 msgr2=0x7f3ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3edc036000 tx=0x7f3edc030000 comp rx=0 tx=0).stop 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 msgr2=0x7f3ee81ea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 0x7f3ee81ea530 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c410 tx=0x7f3ee000c8d0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 shutdown_connections 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ee8155880 0x7f3ee81ea530 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ee80c5290 0x7f3ee80ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 --2- 172.21.15.67:0/2500527251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ee80af8a0 0x7f3ee80bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 >> 172.21.15.67:0/2500527251 conn(0x7f3ee80ac230 msgr2=0x7f3ee8158380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 shutdown_connections 2024-02-26T10:42:55.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.121+0000 7f3eeee00640 1 -- 172.21.15.67:0/2500527251 wait complete. 2024-02-26T10:42:55.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:55.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.370+0000 7fd101e00640 1 Processor -- start 2024-02-26T10:42:55.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.370+0000 7fd101e00640 1 -- start start 2024-02-26T10:42:55.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.371+0000 7fd101e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0fc06ac20 0x7fd0fc0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.371+0000 7fd101e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd0fc06b000 0x7fd0fc06db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.371+0000 7fd101e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.371+0000 7fd101e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc06b460 con 0x7fd0fc06ac20 2024-02-26T10:42:55.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.371+0000 7fd101e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc0a9a80 con 0x7fd0fc06b000 2024-02-26T10:42:55.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.371+0000 7fd101e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0a9d60 con 0x7fd0fc153610 2024-02-26T10:42:55.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0fc06ac20 0x7fd0fc0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0fc06ac20 0x7fd0fc0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34808/0 (socket says 172.21.15.67:34808) 2024-02-26T10:42:55.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 -- 172.21.15.67:0/4020109109 learned_addr learned my addr 172.21.15.67:0/4020109109 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 -- 172.21.15.67:0/4020109109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 msgr2=0x7fd0fc0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd101400640 1 --2- 172.21.15.67:0/4020109109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd0fbe00640 1 --2- 172.21.15.67:0/4020109109 >> v2:172.21.15.82:3300/0 conn(0x7fd0fc06b000 0x7fd0fc06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 --2- 172.21.15.67:0/4020109109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc0abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 -- 172.21.15.67:0/4020109109 >> v2:172.21.15.82:3300/0 conn(0x7fd0fc06b000 msgr2=0x7fd0fc06db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 --2- 172.21.15.67:0/4020109109 >> v2:172.21.15.82:3300/0 conn(0x7fd0fc06b000 0x7fd0fc06db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd100a00640 1 -- 172.21.15.67:0/4020109109 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc0ab750 con 0x7fd0fc06ac20 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.372+0000 7fd0fbe00640 1 --2- 172.21.15.67:0/4020109109 >> v2:172.21.15.82:3300/0 conn(0x7fd0fc06b000 0x7fd0fc06db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:55.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.373+0000 7fd100a00640 1 --2- 172.21.15.67:0/4020109109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc0a94a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc06ed00 tx=0x7fd0f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce83a9f092089557 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:55.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd0fb400640 1 -- 172.21.15.67:0/4020109109 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0035020 con 0x7fd0fc06ac20 2024-02-26T10:42:55.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd0fb400640 1 -- 172.21.15.67:0/4020109109 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0031870 con 0x7fd0fc06ac20 2024-02-26T10:42:55.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd0fb400640 1 -- 172.21.15.67:0/4020109109 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0031c40 con 0x7fd0fc06ac20 2024-02-26T10:42:55.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 -- 172.21.15.67:0/4020109109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 msgr2=0x7fd0fc0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 --2- 172.21.15.67:0/4020109109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc0a94a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc06ed00 tx=0x7fd0f002b910 comp rx=0 tx=0).stop 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 -- 172.21.15.67:0/4020109109 shutdown_connections 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 --2- 172.21.15.67:0/4020109109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 --2- 172.21.15.67:0/4020109109 >> v2:172.21.15.82:3300/0 conn(0x7fd0fc06b000 0x7fd0fc06db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 --2- 172.21.15.67:0/4020109109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc0a94a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.374+0000 7fd101e00640 1 -- 172.21.15.67:0/4020109109 >> 172.21.15.67:0/4020109109 conn(0x7fd0fc0c63f0 msgr2=0x7fd0fc0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.375+0000 7fd101e00640 1 -- 172.21.15.67:0/4020109109 shutdown_connections 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.375+0000 7fd101e00640 1 -- 172.21.15.67:0/4020109109 wait complete. 2024-02-26T10:42:55.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.376+0000 7fd101e00640 1 Processor -- start 2024-02-26T10:42:55.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 -- start start 2024-02-26T10:42:55.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc1467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0fc06b000 0x7fd0fc0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0ac490 con 0x7fd0fc06ac20 2024-02-26T10:42:55.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc153aa0 con 0x7fd0fc06b000 2024-02-26T10:42:55.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.377+0000 7fd101e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc150410 con 0x7fd0fc153610 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc1467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc1467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34812/0 (socket says 172.21.15.67:34812) 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 -- 172.21.15.67:0/1069165728 learned_addr learned my addr 172.21.15.67:0/1069165728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 -- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 msgr2=0x7fd0fc070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 -- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0fc06b000 msgr2=0x7fd0fc0c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0fc06b000 0x7fd0fc0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 -- 172.21.15.67:0/1069165728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc1ee6d0 con 0x7fd0fc06ac20 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.378+0000 7fd100a00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc1467e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd0f002b950 tx=0x7fd0f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.379+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0040070 con 0x7fd0fc06ac20 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.379+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f003a420 con 0x7fd0fc06ac20 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.379+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0035040 con 0x7fd0fc06ac20 2024-02-26T10:42:55.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.379+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc1ee9b0 con 0x7fd0fc06ac20 2024-02-26T10:42:55.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.379+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc0c5d90 con 0x7fd0fc06ac20 2024-02-26T10:42:55.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.380+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc074a30 con 0x7fd0fc06ac20 2024-02-26T10:42:55.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.381+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0f004b060 con 0x7fd0fc06ac20 2024-02-26T10:42:55.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.382+0000 7fd0f8c00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.382+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd0f00ac540 con 0x7fd0fc06ac20 2024-02-26T10:42:55.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.382+0000 7fd0fbe00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.383+0000 7fd0fbe00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc1526c0 tx=0x7fd0e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:55.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.387+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0f007b050 con 0x7fd0fc06ac20 2024-02-26T10:42:55.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.622+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0fc06b9b0 con 0x7fd0fc06ac20 2024-02-26T10:42:55.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.624+0000 7fd0f8c00640 1 -- 172.21.15.67:0/1069165728 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6185 (secure 0 0 0) 0x7fd0f007a950 con 0x7fd0fc06ac20 2024-02-26T10:42:55.624 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:55.624 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 53 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.626+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd0dc06b4d0 msgr2=0x7fd0dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc1526c0 tx=0x7fd0e8008040 comp rx=0 tx=0).stop 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 msgr2=0x7fd0fc1467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc1467e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd0f002b950 tx=0x7fd0f00046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 shutdown_connections 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd0fc153610 0x7fd0fc070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0fc06b000 0x7fd0fc0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 --2- 172.21.15.67:0/1069165728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0fc06ac20 0x7fd0fc1467e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 >> 172.21.15.67:0/1069165728 conn(0x7fd0fc0c63f0 msgr2=0x7fd0fc0c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 shutdown_connections 2024-02-26T10:42:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.627+0000 7fd101e00640 1 -- 172.21.15.67:0/1069165728 wait complete. 2024-02-26T10:42:55.643 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.870+0000 7f1951600640 1 Processor -- start 2024-02-26T10:42:55.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.870+0000 7f1951600640 1 -- start start 2024-02-26T10:42:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f1951600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 0x7f194c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f1951600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f194c06db00 0x7f194c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f1951600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c06b460 con 0x7f194c06b000 2024-02-26T10:42:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f1951600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c06dfb0 con 0x7f194c06db00 2024-02-26T10:42:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f1951600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0a9a80 con 0x7f194c06ac20 2024-02-26T10:42:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f194be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 0x7f194c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f194be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 0x7f194c0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:46216/0 (socket says 172.21.15.67:46216) 2024-02-26T10:42:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.872+0000 7f194be00640 1 -- 172.21.15.67:0/530179724 learned_addr learned my addr 172.21.15.67:0/530179724 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.873+0000 7f1950c00640 1 --2- 172.21.15.67:0/530179724 >> v2:172.21.15.82:3300/0 conn(0x7f194c06db00 0x7f194c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.873+0000 7f194be00640 1 -- 172.21.15.67:0/530179724 >> v2:172.21.15.82:3300/0 conn(0x7f194c06db00 msgr2=0x7f194c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.873+0000 7f194be00640 1 --2- 172.21.15.67:0/530179724 >> v2:172.21.15.82:3300/0 conn(0x7f194c06db00 0x7f194c0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.873+0000 7f194be00640 1 -- 172.21.15.67:0/530179724 >> v1:172.21.15.82:6789/0 conn(0x7f194c06b000 legacy=0x7f194c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.873+0000 7f194be00640 1 -- 172.21.15.67:0/530179724 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0a9d60 con 0x7f194c06ac20 2024-02-26T10:42:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.875+0000 7f194be00640 1 --2- 172.21.15.67:0/530179724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 0x7f194c0a94a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f193c009870 tx=0x7f193c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9705bd8197cdfd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.875+0000 7f194aa00640 1 -- 172.21.15.67:0/530179724 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c034070 con 0x7f194c06ac20 2024-02-26T10:42:55.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.875+0000 7f194aa00640 1 -- 172.21.15.67:0/530179724 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c030840 con 0x7f194c06ac20 2024-02-26T10:42:55.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.875+0000 7f194aa00640 1 -- 172.21.15.67:0/530179724 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c030c10 con 0x7f194c06ac20 2024-02-26T10:42:55.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 -- 172.21.15.67:0/530179724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 msgr2=0x7f194c0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 --2- 172.21.15.67:0/530179724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 0x7f194c0a94a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f193c009870 tx=0x7f193c02b980 comp rx=0 tx=0).stop 2024-02-26T10:42:55.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 -- 172.21.15.67:0/530179724 shutdown_connections 2024-02-26T10:42:55.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 --2- 172.21.15.67:0/530179724 >> v2:172.21.15.82:3300/0 conn(0x7f194c06db00 0x7f194c0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 --2- 172.21.15.67:0/530179724 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06ac20 0x7f194c0a94a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 -- 172.21.15.67:0/530179724 >> 172.21.15.67:0/530179724 conn(0x7f194c0c63f0 msgr2=0x7f194c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:55.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 -- 172.21.15.67:0/530179724 shutdown_connections 2024-02-26T10:42:55.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.876+0000 7f1951600640 1 -- 172.21.15.67:0/530179724 wait complete. 2024-02-26T10:42:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.878+0000 7f1951600640 1 Processor -- start 2024-02-26T10:42:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.878+0000 7f1951600640 1 -- start start 2024-02-26T10:42:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.879+0000 7f1951600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 0x7f194c145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.879+0000 7f1951600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06b000 0x7f194c1eeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f1951600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06db00 0x7f194c0bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f1951600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0ac490 con 0x7f194c06ac20 2024-02-26T10:42:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f1951600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0aba30 con 0x7f194c06db00 2024-02-26T10:42:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f1951600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c150410 con 0x7f194c06b000 2024-02-26T10:42:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 0x7f194c145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 0x7f194c145eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34824/0 (socket says 172.21.15.67:34824) 2024-02-26T10:42:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 -- 172.21.15.67:0/1327661525 learned_addr learned my addr 172.21.15.67:0/1327661525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194b400640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06b000 0x7f194c1eeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f1950c00640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06db00 0x7f194c0bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 -- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06b000 msgr2=0x7f194c1eeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06b000 0x7f194c1eeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 -- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06db00 msgr2=0x7f194c0bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06db00 0x7f194c0bfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.880+0000 7f194be00640 1 -- 172.21.15.67:0/1327661525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c1ee0c0 con 0x7f194c06ac20 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.881+0000 7f194be00640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 0x7f194c145eb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f193c000c00 tx=0x7f193c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.881+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c048070 con 0x7f194c06ac20 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.881+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c030640 con 0x7f194c06ac20 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.881+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c034040 con 0x7f194c06ac20 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.881+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c1480a0 con 0x7f194c06ac20 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.881+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0bffd0 con 0x7f194c06ac20 2024-02-26T10:42:55.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.883+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c074a30 con 0x7f194c06ac20 2024-02-26T10:42:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.883+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f193c04b070 con 0x7f194c06ac20 2024-02-26T10:42:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.884+0000 7f1943e00640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f192c06b4d0 0x7f192c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.884+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f193c0ac750 con 0x7f194c06ac20 2024-02-26T10:42:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.885+0000 7f194b400640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f192c06b4d0 0x7f192c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:55.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.886+0000 7f194b400640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f192c06b4d0 0x7f192c06d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f194c0c6350 tx=0x7f1938009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:55.889+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f193c07b380 con 0x7f194c06ac20 2024-02-26T10:42:56.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.089+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f194c06b9b0 con 0x7f194c06ac20 2024-02-26T10:42:56.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.092+0000 7f1943e00640 1 -- 172.21.15.67:0/1327661525 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7f193c07ac80 con 0x7f194c06ac20 2024-02-26T10:42:56.089 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:56.090 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.094+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f192c06b4d0 msgr2=0x7f192c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.094+0000 7f1951600640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f192c06b4d0 0x7f192c06d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f194c0c6350 tx=0x7f1938009040 comp rx=0 tx=0).stop 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.094+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 msgr2=0x7f194c145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.094+0000 7f1951600640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 0x7f194c145eb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f193c000c00 tx=0x7f193c004730 comp rx=0 tx=0).stop 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 shutdown_connections 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f192c06b4d0 0x7f192c06d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06db00 0x7f194c0bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c06b000 0x7f194c1eeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 --2- 172.21.15.67:0/1327661525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f194c06ac20 0x7f194c145eb0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 >> 172.21.15.67:0/1327661525 conn(0x7f194c0c63f0 msgr2=0x7f194c14b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 shutdown_connections 2024-02-26T10:42:56.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.095+0000 7f1951600640 1 -- 172.21.15.67:0/1327661525 wait complete. 2024-02-26T10:42:56.110 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T10:42:56.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:56.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.336+0000 7f1e27200640 1 Processor -- start 2024-02-26T10:42:56.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.336+0000 7f1e27200640 1 -- start start 2024-02-26T10:42:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e27200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e280a94a0 0x7f1e2806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e27200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e2806db00 0x7f1e280abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e27200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2806df70 con 0x7f1e2806ac20 2024-02-26T10:42:56.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e27200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280a9a80 con 0x7f1e280a94a0 2024-02-26T10:42:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e27200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280a9d60 con 0x7f1e2806db00 2024-02-26T10:42:56.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e25400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e280a94a0 0x7f1e2806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e25400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e280a94a0 0x7f1e2806b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34832/0 (socket says 172.21.15.67:34832) 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e25400640 1 -- 172.21.15.67:0/1221934422 learned_addr learned my addr 172.21.15.67:0/1221934422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.338+0000 7f1e26800640 1 --2- 172.21.15.67:0/1221934422 >> v2:172.21.15.82:3300/0 conn(0x7f1e2806db00 0x7f1e280abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.339+0000 7f1e25400640 1 -- 172.21.15.67:0/1221934422 >> v2:172.21.15.82:3300/0 conn(0x7f1e2806db00 msgr2=0x7f1e280abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.339+0000 7f1e25400640 1 --2- 172.21.15.67:0/1221934422 >> v2:172.21.15.82:3300/0 conn(0x7f1e2806db00 0x7f1e280abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.339+0000 7f1e25400640 1 -- 172.21.15.67:0/1221934422 >> v1:172.21.15.82:6789/0 conn(0x7f1e2806ac20 legacy=0x7f1e28153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.339+0000 7f1e25400640 1 -- 172.21.15.67:0/1221934422 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280ab750 con 0x7f1e280a94a0 2024-02-26T10:42:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.340+0000 7f1e25400640 1 --2- 172.21.15.67:0/1221934422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e2806b000 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1e28157220 tx=0x7f1e1402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d1d16cc0d2b130 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.340+0000 7f1e24a00640 1 -- 172.21.15.67:0/1221934422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14034070 con 0x7f1e280a94a0 2024-02-26T10:42:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e24a00640 1 -- 172.21.15.67:0/1221934422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e14030870 con 0x7f1e280a94a0 2024-02-26T10:42:56.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e24a00640 1 -- 172.21.15.67:0/1221934422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14030c40 con 0x7f1e280a94a0 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 -- 172.21.15.67:0/1221934422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 msgr2=0x7f1e2806b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 --2- 172.21.15.67:0/1221934422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e2806b000 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1e28157220 tx=0x7f1e1402b9b0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 -- 172.21.15.67:0/1221934422 shutdown_connections 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 --2- 172.21.15.67:0/1221934422 >> v2:172.21.15.82:3300/0 conn(0x7f1e2806db00 0x7f1e280abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 --2- 172.21.15.67:0/1221934422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e2806b000 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 -- 172.21.15.67:0/1221934422 >> 172.21.15.67:0/1221934422 conn(0x7f1e280c63f0 msgr2=0x7f1e280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 -- 172.21.15.67:0/1221934422 shutdown_connections 2024-02-26T10:42:56.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.341+0000 7f1e27200640 1 -- 172.21.15.67:0/1221934422 wait complete. 2024-02-26T10:42:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.343+0000 7f1e27200640 1 Processor -- start 2024-02-26T10:42:56.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.343+0000 7f1e27200640 1 -- start start 2024-02-26T10:42:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e27200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e2806ac20 0x7f1e281ec770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e27200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2806db00 0x7f1e2814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e27200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e28141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e27200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28150410 con 0x7f1e280a94a0 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e27200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280ac490 con 0x7f1e2806db00 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e27200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28150590 con 0x7f1e2806ac20 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e26800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e28141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e26800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e28141790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34836/0 (socket says 172.21.15.67:34836) 2024-02-26T10:42:56.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.344+0000 7f1e26800640 1 -- 172.21.15.67:0/745766193 learned_addr learned my addr 172.21.15.67:0/745766193 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:56.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e25e00640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e2806ac20 0x7f1e281ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e25400640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2806db00 0x7f1e2814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e26800640 1 -- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e2806ac20 msgr2=0x7f1e281ec770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e26800640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e2806ac20 0x7f1e281ec770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e26800640 1 -- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2806db00 msgr2=0x7f1e2814b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e26800640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2806db00 0x7f1e2814b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e26800640 1 -- 172.21.15.67:0/745766193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e281e8a00 con 0x7f1e280a94a0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e25400640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2806db00 0x7f1e2814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e26800640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e28141790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1e20007e40 tx=0x7f1e2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20014620 con 0x7f1e280a94a0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20019070 con 0x7f1e280a94a0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20014a20 con 0x7f1e280a94a0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.345+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e281e8ce0 con 0x7f1e280a94a0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.346+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2814ba30 con 0x7f1e280a94a0 2024-02-26T10:42:56.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.347+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e28074a30 con 0x7f1e280a94a0 2024-02-26T10:42:56.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.348+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e20014bc0 con 0x7f1e280a94a0 2024-02-26T10:42:56.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.348+0000 7f1e1e000640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1e0806b4d0 0x7f1e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.349+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1e2008af00 con 0x7f1e280a94a0 2024-02-26T10:42:56.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.349+0000 7f1e25e00640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1e0806b4d0 0x7f1e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.350+0000 7f1e25e00640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1e0806b4d0 0x7f1e0806d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1e180015a0 tx=0x7f1e18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:56.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.353+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e20059a10 con 0x7f1e280a94a0 2024-02-26T10:42:56.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.589+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e28148650 con 0x7f1e280a94a0 2024-02-26T10:42:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.590+0000 7f1e1e000640 1 -- 172.21.15.67:0/745766193 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6185 (secure 0 0 0) 0x7f1e20059310 con 0x7f1e280a94a0 2024-02-26T10:42:56.588 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:56.588 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:42:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 53 2024-02-26T10:42:56.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1e0806b4d0 msgr2=0x7f1e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1e0806b4d0 0x7f1e0806d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1e180015a0 tx=0x7f1e18009040 comp rx=0 tx=0).stop 2024-02-26T10:42:56.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 msgr2=0x7f1e28141790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e28141790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1e20007e40 tx=0x7f1e2000e4b0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 shutdown_connections 2024-02-26T10:42:56.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e280a94a0 0x7f1e28141790 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1e0806b4d0 0x7f1e0806d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e2806db00 0x7f1e2814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 --2- 172.21.15.67:0/745766193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e2806ac20 0x7f1e281ec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.593+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 >> 172.21.15.67:0/745766193 conn(0x7f1e280c63f0 msgr2=0x7f1e2814ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.594+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 shutdown_connections 2024-02-26T10:42:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.594+0000 7f1e27200640 1 -- 172.21.15.67:0/745766193 wait complete. 2024-02-26T10:42:56.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:42:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.838+0000 7f4968000640 1 Processor -- start 2024-02-26T10:42:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.838+0000 7f4968000640 1 -- start start 2024-02-26T10:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4968000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49600c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4968000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f49600bfa30 0x7f4960158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4968000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49600c15e0 con 0x7f49600c35d0 2024-02-26T10:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4968000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496006e660 con 0x7f49600bfa30 2024-02-26T10:42:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4968000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496006e920 con 0x7f4960152420 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4964c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f49600bfa30 0x7f4960158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4964c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f49600bfa30 0x7f4960158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34838/0 (socket says 172.21.15.67:34838) 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.840+0000 7f4964c00640 1 -- 172.21.15.67:0/4245361599 learned_addr learned my addr 172.21.15.67:0/4245361599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.841+0000 7f4965600640 1 --2- 172.21.15.67:0/4245361599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49600c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.841+0000 7f4964c00640 1 -- 172.21.15.67:0/4245361599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 msgr2=0x7f49600c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.841+0000 7f4964c00640 1 --2- 172.21.15.67:0/4245361599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49600c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.841+0000 7f4964c00640 1 -- 172.21.15.67:0/4245361599 >> v1:172.21.15.82:6789/0 conn(0x7f49600c35d0 legacy=0x7f496006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.841+0000 7f4964c00640 1 -- 172.21.15.67:0/4245361599 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600c1880 con 0x7f49600bfa30 2024-02-26T10:42:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.842+0000 7f4964c00640 1 --2- 172.21.15.67:0/4245361599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600bfa30 0x7f4960158920 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f494c009870 tx=0x7f494c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a01ee80e1c0f29 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.842+0000 7f495fe00640 1 -- 172.21.15.67:0/4245361599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0317c0 con 0x7f49600bfa30 2024-02-26T10:42:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.842+0000 7f495fe00640 1 -- 172.21.15.67:0/4245361599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c037070 con 0x7f49600bfa30 2024-02-26T10:42:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.842+0000 7f495fe00640 1 -- 172.21.15.67:0/4245361599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c031b80 con 0x7f49600bfa30 2024-02-26T10:42:56.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 -- 172.21.15.67:0/4245361599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600bfa30 msgr2=0x7f4960158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 --2- 172.21.15.67:0/4245361599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600bfa30 0x7f4960158920 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f494c009870 tx=0x7f494c02b860 comp rx=0 tx=0).stop 2024-02-26T10:42:56.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 -- 172.21.15.67:0/4245361599 shutdown_connections 2024-02-26T10:42:56.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 --2- 172.21.15.67:0/4245361599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600bfa30 0x7f4960158920 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 --2- 172.21.15.67:0/4245361599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49600c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 -- 172.21.15.67:0/4245361599 >> 172.21.15.67:0/4245361599 conn(0x7f49600a84a0 msgr2=0x7f49600a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:56.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 -- 172.21.15.67:0/4245361599 shutdown_connections 2024-02-26T10:42:56.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.843+0000 7f4968000640 1 -- 172.21.15.67:0/4245361599 wait complete. 2024-02-26T10:42:56.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.845+0000 7f4968000640 1 Processor -- start 2024-02-26T10:42:56.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.845+0000 7f4968000640 1 -- start start 2024-02-26T10:42:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4968000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600bfa30 0x7f496014aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4968000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 0x7f49600a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4968000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49601560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4968000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960152800 con 0x7f49600c35d0 2024-02-26T10:42:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4968000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960158400 con 0x7f49600bfa30 2024-02-26T10:42:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4968000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4960154e10 con 0x7f4960152420 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4964c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 0x7f49600a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4964c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 0x7f49600a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34846/0 (socket says 172.21.15.67:34846) 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.846+0000 7f4964c00640 1 -- 172.21.15.67:0/2827457796 learned_addr learned my addr 172.21.15.67:0/2827457796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4965600640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600bfa30 0x7f496014aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4966000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49601560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4964c00640 1 -- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 msgr2=0x7f49601560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4964c00640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49601560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4964c00640 1 -- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600bfa30 msgr2=0x7f496014aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4964c00640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600bfa30 0x7f496014aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4964c00640 1 -- 172.21.15.67:0/2827457796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600c5720 con 0x7f49600c35d0 2024-02-26T10:42:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.847+0000 7f4964c00640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 0x7f49600a9f80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.848+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c032860 con 0x7f49600c35d0 2024-02-26T10:42:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.848+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c032a20 con 0x7f49600c35d0 2024-02-26T10:42:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.848+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c032e10 con 0x7f49600c35d0 2024-02-26T10:42:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.848+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49600ab1e0 con 0x7f49600c35d0 2024-02-26T10:42:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.848+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49600c4600 con 0x7f49600c35d0 2024-02-26T10:42:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.849+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49600aafd0 con 0x7f49600c35d0 2024-02-26T10:42:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.850+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f494c04c060 con 0x7f49600c35d0 2024-02-26T10:42:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.851+0000 7f495d600640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494006b550 0x7f494006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.851+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f494c0abc10 con 0x7f49600c35d0 2024-02-26T10:42:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.851+0000 7f4965600640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494006b550 0x7f494006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.853+0000 7f4965600640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494006b550 0x7f494006da10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:56.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:56.856+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f494c040070 con 0x7f49600c35d0 2024-02-26T10:42:57.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.091+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4960074a30 con 0x7f49600c35d0 2024-02-26T10:42:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.092+0000 7f495d600640 1 -- 172.21.15.67:0/2827457796 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6185 (secure 0 0 0) 0x7f494c031e90 con 0x7f49600c35d0 2024-02-26T10:42:57.090 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:57.090 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:41:58.715593+0000","modified":"2024-02-26T10:42:01.803454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6450},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6450":{"gid":6450,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":5,"addr":"172.21.15.67:6835/2183069114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":2183069114},{"type":"v1","addr":"172.21.15.67:6835","nonce":2183069114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:42:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 53 2024-02-26T10:42:57.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494006b550 msgr2=0x7f494006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494006b550 0x7f494006da10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954008040 comp rx=0 tx=0).stop 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 msgr2=0x7f49600a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 0x7f49600a9f80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c00 tx=0x7f494c0046c0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 shutdown_connections 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4960152420 0x7f49601560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49600c35d0 0x7f49600a9f80 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f494006b550 0x7f494006da10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 --2- 172.21.15.67:0/2827457796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600bfa30 0x7f496014aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.094+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 >> 172.21.15.67:0/2827457796 conn(0x7f49600a84a0 msgr2=0x7f49600bff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.095+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 shutdown_connections 2024-02-26T10:42:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.095+0000 7f4968000640 1 -- 172.21.15.67:0/2827457796 wait complete. 2024-02-26T10:42:57.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:42:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.326+0000 7fb0f4e00640 1 Processor -- start 2024-02-26T10:42:57.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.326+0000 7fb0f4e00640 1 -- start start 2024-02-26T10:42:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0f4e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb0f006ac20 0x7fb0f006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0f4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0f0153630 0x7fb0f01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0f4e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f0155f10 con 0x7fb0f00c2310 2024-02-26T10:42:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0f4e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f0153a60 con 0x7fb0f0153630 2024-02-26T10:42:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0f4e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00c28d0 con 0x7fb0f006ac20 2024-02-26T10:42:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0efe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0f0153630 0x7fb0f01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0efe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0f0153630 0x7fb0f01524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34862/0 (socket says 172.21.15.67:34862) 2024-02-26T10:42:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.328+0000 7fb0efe00640 1 -- 172.21.15.67:0/3908653707 learned_addr learned my addr 172.21.15.67:0/3908653707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.329+0000 7fb0ef400640 1 --2- 172.21.15.67:0/3908653707 >> v2:172.21.15.82:3300/0 conn(0x7fb0f006ac20 0x7fb0f006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.329+0000 7fb0efe00640 1 -- 172.21.15.67:0/3908653707 >> v2:172.21.15.82:3300/0 conn(0x7fb0f006ac20 msgr2=0x7fb0f006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.329+0000 7fb0efe00640 1 --2- 172.21.15.67:0/3908653707 >> v2:172.21.15.82:3300/0 conn(0x7fb0f006ac20 0x7fb0f006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.329+0000 7fb0efe00640 1 -- 172.21.15.67:0/3908653707 >> v1:172.21.15.82:6789/0 conn(0x7fb0f00c2310 legacy=0x7fb0f0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:57.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.329+0000 7fb0efe00640 1 -- 172.21.15.67:0/3908653707 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f0155150 con 0x7fb0f0153630 2024-02-26T10:42:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.330+0000 7fb0efe00640 1 --2- 172.21.15.67:0/3908653707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f0153630 0x7fb0f01524b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb0f006ee00 tx=0x7fb0e802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2489752ae6ec001 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.330+0000 7fb0eea00640 1 -- 172.21.15.67:0/3908653707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e802c960 con 0x7fb0f0153630 2024-02-26T10:42:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.330+0000 7fb0eea00640 1 -- 172.21.15.67:0/3908653707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e802cb00 con 0x7fb0f0153630 2024-02-26T10:42:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.330+0000 7fb0eea00640 1 -- 172.21.15.67:0/3908653707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80343f0 con 0x7fb0f0153630 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 -- 172.21.15.67:0/3908653707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f0153630 msgr2=0x7fb0f01524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/3908653707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f0153630 0x7fb0f01524b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb0f006ee00 tx=0x7fb0e802bdc0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 -- 172.21.15.67:0/3908653707 shutdown_connections 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/3908653707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f0153630 0x7fb0f01524b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/3908653707 >> v2:172.21.15.82:3300/0 conn(0x7fb0f006ac20 0x7fb0f006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 -- 172.21.15.67:0/3908653707 >> 172.21.15.67:0/3908653707 conn(0x7fb0f00c63f0 msgr2=0x7fb0f00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 -- 172.21.15.67:0/3908653707 shutdown_connections 2024-02-26T10:42:57.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.331+0000 7fb0f4e00640 1 -- 172.21.15.67:0/3908653707 wait complete. 2024-02-26T10:42:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.333+0000 7fb0f4e00640 1 Processor -- start 2024-02-26T10:42:57.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.333+0000 7fb0f4e00640 1 -- start start 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0f4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f006ac20 0x7fb0f00abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0f4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 0x7fb0f01ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0f4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0f0153630 0x7fb0f014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0f4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f01559f0 con 0x7fb0f00c2310 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0f4e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0155710 con 0x7fb0f006ac20 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0f4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0148850 con 0x7fb0f0153630 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 0x7fb0f01ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0e6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 0x7fb0f01ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34870/0 (socket says 172.21.15.67:34870) 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.334+0000 7fb0e6a00640 1 -- 172.21.15.67:0/2871848669 learned_addr learned my addr 172.21.15.67:0/2871848669 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e6a00640 1 -- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0f0153630 msgr2=0x7fb0f014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0efe00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0f0153630 0x7fb0f014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0ef400640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f006ac20 0x7fb0f00abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e6a00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0f0153630 0x7fb0f014ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e6a00640 1 -- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f006ac20 msgr2=0x7fb0f00abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e6a00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f006ac20 0x7fb0f00abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e6a00640 1 -- 172.21.15.67:0/2871848669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f01ec7c0 con 0x7fb0f00c2310 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0ef400640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f006ac20 0x7fb0f00abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e6a00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 0x7fb0f01ebe80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800b420 tx=0x7fb0d800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8011830 con 0x7fb0f00c2310 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8016070 con 0x7fb0f00c2310 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f0072ad0 con 0x7fb0f00c2310 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.335+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8011c10 con 0x7fb0f00c2310 2024-02-26T10:42:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.336+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f01eca60 con 0x7fb0f00c2310 2024-02-26T10:42:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.337+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f00c20f0 con 0x7fb0f00c2310 2024-02-26T10:42:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.338+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0d801b460 con 0x7fb0f00c2310 2024-02-26T10:42:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.338+0000 7fb0e7e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0cc06b4d0 0x7fb0cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.339+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb0d8089f90 con 0x7fb0f00c2310 2024-02-26T10:42:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.339+0000 7fb0ef400640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0cc06b4d0 0x7fb0cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.340+0000 7fb0ef400640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0cc06b4d0 0x7fb0cc06d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0006fd0 tx=0x7fb0e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.343+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0d8020020 con 0x7fb0f00c2310 2024-02-26T10:42:57.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.541+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0f0074a10 con 0x7fb0f00c2310 2024-02-26T10:42:57.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.546+0000 7fb0e7e00640 1 -- 172.21.15.67:0/2871848669 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20308 (secure 0 0 0) 0x7fb0d8058b20 con 0x7fb0f00c2310 2024-02-26T10:42:57.544 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:42:57.544 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":80,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:42:21.580979+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-02-26T10:41:57.123912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-02-26T10:41:57.744647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:42:57.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.548+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0cc06b4d0 msgr2=0x7fb0cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.548+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0cc06b4d0 0x7fb0cc06d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0006fd0 tx=0x7fb0e0008040 comp rx=0 tx=0).stop 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 msgr2=0x7fb0f01ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 0x7fb0f01ebe80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800b420 tx=0x7fb0d800b8e0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 shutdown_connections 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb0f0153630 0x7fb0f014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f00c2310 0x7fb0f01ebe80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb0cc06b4d0 0x7fb0cc06d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 --2- 172.21.15.67:0/2871848669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f006ac20 0x7fb0f00abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 >> 172.21.15.67:0/2871848669 conn(0x7fb0f00c63f0 msgr2=0x7fb0f00c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 shutdown_connections 2024-02-26T10:42:57.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.549+0000 7fb0f4e00640 1 -- 172.21.15.67:0/2871848669 wait complete. 2024-02-26T10:42:57.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-26T10:42:57.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.791+0000 7f372fe00640 1 Processor -- start 2024-02-26T10:42:57.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.792+0000 7f372fe00640 1 -- start start 2024-02-26T10:42:57.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3730152380 0x7f37300c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37300c7020 0x7f3730155790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372fe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3730152760 con 0x7f373006d290 2024-02-26T10:42:57.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3730155cd0 con 0x7f3730152380 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37300c7550 con 0x7f37300c7020 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3730152380 0x7f37300c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3730152380 0x7f37300c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34886/0 (socket says 172.21.15.67:34886) 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.793+0000 7f372ea00640 1 -- 172.21.15.67:0/940083623 learned_addr learned my addr 172.21.15.67:0/940083623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.794+0000 7f372e000640 1 --2- 172.21.15.67:0/940083623 >> v2:172.21.15.82:3300/0 conn(0x7f37300c7020 0x7f3730155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.794+0000 7f372ea00640 1 -- 172.21.15.67:0/940083623 >> v2:172.21.15.82:3300/0 conn(0x7f37300c7020 msgr2=0x7f3730155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.794+0000 7f372ea00640 1 --2- 172.21.15.67:0/940083623 >> v2:172.21.15.82:3300/0 conn(0x7f37300c7020 0x7f3730155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.794+0000 7f372ea00640 1 -- 172.21.15.67:0/940083623 >> v1:172.21.15.82:6789/0 conn(0x7f373006d290 legacy=0x7f37300c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.794+0000 7f372ea00640 1 -- 172.21.15.67:0/940083623 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300c7830 con 0x7f3730152380 2024-02-26T10:42:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.794+0000 7f372e000640 1 --2- 172.21.15.67:0/940083623 >> v2:172.21.15.82:3300/0 conn(0x7f37300c7020 0x7f3730155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.795+0000 7f372ea00640 1 --2- 172.21.15.67:0/940083623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3730152380 0x7f37300c8510 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f37300acf40 tx=0x7f372002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4924bbaef239a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.795+0000 7f372d600640 1 -- 172.21.15.67:0/940083623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372002c9f0 con 0x7f3730152380 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.795+0000 7f372d600640 1 -- 172.21.15.67:0/940083623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372002cb90 con 0x7f3730152380 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.795+0000 7f372d600640 1 -- 172.21.15.67:0/940083623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37200343f0 con 0x7f3730152380 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 -- 172.21.15.67:0/940083623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3730152380 msgr2=0x7f37300c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 --2- 172.21.15.67:0/940083623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3730152380 0x7f37300c8510 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f37300acf40 tx=0x7f372002be50 comp rx=0 tx=0).stop 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 -- 172.21.15.67:0/940083623 shutdown_connections 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 --2- 172.21.15.67:0/940083623 >> v2:172.21.15.82:3300/0 conn(0x7f37300c7020 0x7f3730155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 --2- 172.21.15.67:0/940083623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3730152380 0x7f37300c8510 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 -- 172.21.15.67:0/940083623 >> 172.21.15.67:0/940083623 conn(0x7f37300ac1a0 msgr2=0x7f37300ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 -- 172.21.15.67:0/940083623 shutdown_connections 2024-02-26T10:42:57.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.796+0000 7f372fe00640 1 -- 172.21.15.67:0/940083623 wait complete. 2024-02-26T10:42:57.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.798+0000 7f372fe00640 1 Processor -- start 2024-02-26T10:42:57.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.798+0000 7f372fe00640 1 -- start start 2024-02-26T10:42:57.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373006d290 0x7f373014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 0x7f373014b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3730152380 0x7f37301ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300c56a0 con 0x7f37300c7020 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373006cd70 con 0x7f373006d290 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37300c5820 con 0x7f3730152380 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 0x7f373014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 0x7f373014b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34890/0 (socket says 172.21.15.67:34890) 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.799+0000 7f372e000640 1 -- 172.21.15.67:0/577060297 learned_addr learned my addr 172.21.15.67:0/577060297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372ea00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373006d290 0x7f373014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372f400640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3730152380 0x7f37301ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372e000640 1 -- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3730152380 msgr2=0x7f37301ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372e000640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3730152380 0x7f37301ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372e000640 1 -- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373006d290 msgr2=0x7f373014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372e000640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373006d290 0x7f373014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372e000640 1 -- 172.21.15.67:0/577060297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300ab3e0 con 0x7f37300c7020 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372f400640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3730152380 0x7f37301ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:42:57.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.800+0000 7f372e000640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 0x7f373014b000 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f371800c820 tx=0x7f371800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:57.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.801+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718015070 con 0x7f37300c7020 2024-02-26T10:42:57.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.801+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3718007a30 con 0x7f37300c7020 2024-02-26T10:42:57.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.801+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718007de0 con 0x7f37300c7020 2024-02-26T10:42:57.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.801+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37300aae20 con 0x7f37300c7020 2024-02-26T10:42:57.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.801+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37300ab8b0 con 0x7f37300c7020 2024-02-26T10:42:57.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.802+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3730074a30 con 0x7f37300c7020 2024-02-26T10:42:57.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.803+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3718018070 con 0x7f37300c7020 2024-02-26T10:42:57.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.804+0000 7f3727400640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f370c06b4d0 0x7f370c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:57.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.804+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f371808a380 con 0x7f37300c7020 2024-02-26T10:42:57.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.805+0000 7f372ea00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f370c06b4d0 0x7f370c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:57.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.807+0000 7f372ea00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f370c06b4d0 0x7f370c06d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3720035000 tx=0x7f3720030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:57.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:57.810+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3718058f30 con 0x7f37300c7020 2024-02-26T10:42:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.047+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f37300c7c50 con 0x7f37300c7020 2024-02-26T10:42:58.046 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:42:58.048+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:42:58.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.052+0000 7f3727400640 1 -- 172.21.15.67:0/577060297 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3718058830 con 0x7f37300c7020 2024-02-26T10:42:58.050 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f370c06b4d0 msgr2=0x7f370c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f370c06b4d0 0x7f370c06d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3720035000 tx=0x7f3720030000 comp rx=0 tx=0).stop 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 msgr2=0x7f373014b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 0x7f373014b000 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f371800c820 tx=0x7f371800cce0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 shutdown_connections 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3730152380 0x7f37301ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37300c7020 0x7f373014b000 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f370c06b4d0 0x7f370c06d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 --2- 172.21.15.67:0/577060297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f373006d290 0x7f373014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.055+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 >> 172.21.15.67:0/577060297 conn(0x7f37300ac1a0 msgr2=0x7f37300c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.056+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 shutdown_connections 2024-02-26T10:42:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.057+0000 7f372fe00640 1 -- 172.21.15.67:0/577060297 wait complete. 2024-02-26T10:42:58.065 INFO:tasks.ceph.mds.a.smithi067.stderr: -6032> 2024-02-26T10:42:03.171+0000 7fa307000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:42:58.076 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-26T10:42:58.129 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-02-26T10:42:58.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.272+0000 7f7fd7200640 1 Processor -- start 2024-02-26T10:42:58.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.273+0000 7f7fd7200640 1 -- start start 2024-02-26T10:42:58.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.274+0000 7f7fd7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81525b0 0x7f7fd8155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.274+0000 7f7fd7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd806c850 con 0x7f7fd80c5330 2024-02-26T10:42:58.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.274+0000 7f7fd7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd8155c90 con 0x7f7fd806bb80 2024-02-26T10:42:58.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.274+0000 7f7fd7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81552c0 con 0x7f7fd81525b0 2024-02-26T10:42:58.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.274+0000 7f7fd5400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7fd80c5330 0x7f7fd80c8520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49614/0 (socket says 172.21.15.67:49614) 2024-02-26T10:42:58.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.274+0000 7f7fd5400640 1 -- 172.21.15.67:0/2266768959 learned_addr learned my addr 172.21.15.67:0/2266768959 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81525b0 0x7f7fd8155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346787401 0 0) 0x7f7fd806c850 con 0x7f7fd80c5330 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb8003520 con 0x7f7fd80c5330 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1876999186 0 0) 0x7f7fb8003520 con 0x7f7fd80c5330 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81525b0 msgr2=0x7f7fd8155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81525b0 0x7f7fd8155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 >> v1:172.21.15.82:6789/0 conn(0x7f7fd806bb80 legacy=0x7f7fd80c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd806c290 con 0x7f7fd80c5330 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f7fc00019f0 con 0x7f7fd80c5330 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 >> v1:172.21.15.67:6789/0 conn(0x7f7fd80c5330 legacy=0x7f7fd80c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fb80039e0 0x7f7fb8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fb8004800 0x7f7fb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb8005180 0x7f7fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81552c0 con 0x7f7fb80039e0 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb800bfe0 con 0x7f7fb8005180 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.275+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb800c1b0 con 0x7f7fb8004800 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fb80039e0 0x7f7fb8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5400640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fb8004800 0x7f7fb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5e00640 1 -- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fb8004800 msgr2=0x7f7fb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fb8004800 0x7f7fb8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5e00640 1 -- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb8005180 msgr2=0x7f7fb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb8005180 0x7f7fb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.276+0000 7f7fd5e00640 1 -- 172.21.15.67:0/2266768959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8003520 con 0x7f7fb80039e0 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.277+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fb80039e0 0x7f7fb8003e00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009ac0 tx=0x7f7fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb5b59c66796d2b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.277+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8039070 con 0x7f7fb80039e0 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.277+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc802c6b0 con 0x7f7fb80039e0 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.277+0000 7f7fd4a00640 1 -- 172.21.15.67:0/2266768959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8009690 con 0x7f7fb80039e0 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd7200640 1 -- 172.21.15.67:0/2266768959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fb80039e0 msgr2=0x7f7fb8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2266768959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fb80039e0 0x7f7fb8003e00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009ac0 tx=0x7f7fb8005830 comp rx=0 tx=0).stop 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd6800640 1 -- 172.21.15.67:0/2266768959 reap_dead start 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd7200640 1 -- 172.21.15.67:0/2266768959 shutdown_connections 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd7200640 1 -- 172.21.15.67:0/2266768959 >> 172.21.15.67:0/2266768959 conn(0x7f7fd80c70a0 msgr2=0x7f7fd80c74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:58.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd7200640 1 -- 172.21.15.67:0/2266768959 shutdown_connections 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.278+0000 7f7fd7200640 1 -- 172.21.15.67:0/2266768959 wait complete. 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.280+0000 7f7fd7200640 1 Processor -- start 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.280+0000 7f7fd7200640 1 -- start start 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb80039e0 0x7f7fd81eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 0x7f7fd8156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81565e0 0x7f7fd814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd806b670 con 0x7f7fd81560b0 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8154f50 con 0x7f7fb80039e0 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd806b7f0 con 0x7f7fd81565e0 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 0x7f7fd8156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 0x7f7fd8156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34930/0 (socket says 172.21.15.67:34930) 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.281+0000 7f7fd5400640 1 -- 172.21.15.67:0/2877797957 learned_addr learned my addr 172.21.15.67:0/2877797957 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:58.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd6800640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81565e0 0x7f7fd814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb80039e0 0x7f7fd81eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5400640 1 -- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81565e0 msgr2=0x7f7fd814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5400640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81565e0 0x7f7fd814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5400640 1 -- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb80039e0 msgr2=0x7f7fd81eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5400640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb80039e0 0x7f7fd81eb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5400640 1 -- 172.21.15.67:0/2877797957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8158c40 con 0x7f7fd81560b0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd5400640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 0x7f7fd8156ec0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0002110 tx=0x7f7fc000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0019070 con 0x7f7fd81560b0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0004d60 con 0x7f7fd81560b0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00072d0 con 0x7f7fd81560b0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.282+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd8158f20 con 0x7f7fd81560b0 2024-02-26T10:42:58.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.283+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd81eab30 con 0x7f7fd81560b0 2024-02-26T10:42:58.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.284+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd8074a30 con 0x7f7fd81560b0 2024-02-26T10:42:58.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.285+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fc0007470 con 0x7f7fd81560b0 2024-02-26T10:42:58.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.286+0000 7f7fce000640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7fb406b500 0x7f7fb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:58.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.286+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f7fc008a6a0 con 0x7f7fd81560b0 2024-02-26T10:42:58.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.287+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7fb406b500 0x7f7fb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:58.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.288+0000 7f7fd5e00640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7fb406b500 0x7f7fb406d9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8036000 tx=0x7f7fc8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:58.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.291+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fc0059160 con 0x7f7fd81560b0 2024-02-26T10:42:58.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:58.529+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7fd80c5110 con 0x7f7fd81560b0 2024-02-26T10:42:59.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.055+0000 7f7fce000640 1 -- 172.21.15.67:0/2877797957 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7fc0058a60 con 0x7f7fd81560b0 2024-02-26T10:42:59.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7fb406b500 msgr2=0x7f7fb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7fb406b500 0x7f7fb406d9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8036000 tx=0x7f7fc8030000 comp rx=0 tx=0).stop 2024-02-26T10:42:59.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 msgr2=0x7f7fd8156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 0x7f7fd8156ec0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0002110 tx=0x7f7fc000b4a0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 shutdown_connections 2024-02-26T10:42:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fd81565e0 0x7f7fd814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7fd81560b0 0x7f7fd8156ec0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f7fb406b500 0x7f7fb406d9c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 --2- 172.21.15.67:0/2877797957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fb80039e0 0x7f7fd81eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 >> 172.21.15.67:0/2877797957 conn(0x7f7fd80c70a0 msgr2=0x7f7fd80c5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 shutdown_connections 2024-02-26T10:42:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.059+0000 7f7fd7200640 1 -- 172.21.15.67:0/2877797957 wait complete. 2024-02-26T10:42:59.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-26T10:42:59.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.301+0000 7f75cfe00640 1 Processor -- start 2024-02-26T10:42:59.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.301+0000 7f75cfe00640 1 -- start start 2024-02-26T10:42:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75d00abcf0 0x7f75d00ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d006e030 con 0x7f75d00c2610 2024-02-26T10:42:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cfe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d006e310 con 0x7f75d006db00 2024-02-26T10:42:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cfe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d006b330 con 0x7f75d00abcf0 2024-02-26T10:42:59.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75ce000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f75d00c2610 0x7f75d006ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49626/0 (socket says 172.21.15.67:49626) 2024-02-26T10:42:59.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75ce000640 1 -- 172.21.15.67:0/1903188006 learned_addr learned my addr 172.21.15.67:0/1903188006 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:42:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cf400640 1 --2- 172.21.15.67:0/1903188006 >> v2:172.21.15.82:3300/0 conn(0x7f75d00abcf0 0x7f75d00ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474937844 0 0) 0x7f75d006e030 con 0x7f75d00c2610 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.303+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b4003520 con 0x7f75d00c2610 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 178985312 0 0) 0x7f75b4003520 con 0x7f75d00c2610 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 >> v2:172.21.15.82:3300/0 conn(0x7f75d00abcf0 msgr2=0x7f75d00ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 --2- 172.21.15.67:0/1903188006 >> v2:172.21.15.82:3300/0 conn(0x7f75d00abcf0 0x7f75d00ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 >> v1:172.21.15.82:6789/0 conn(0x7f75d006db00 legacy=0x7f75d0151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d0070df0 con 0x7f75d00c2610 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f75b8001670 con 0x7f75d00c2610 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 >> v1:172.21.15.67:6789/0 conn(0x7f75d00c2610 legacy=0x7f75d006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b4004800 0x7f75b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d006b330 con 0x7f75b40051c0 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400c160 con 0x7f75b40039b0 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400c360 con 0x7f75b4004800 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.304+0000 7f75cf400640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cea00640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cf400640 1 -- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b4004800 msgr2=0x7f75b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75ce000640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b4004800 0x7f75b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cf400640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75b4004800 0x7f75b4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cf400640 1 -- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40039b0 msgr2=0x7f75b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cf400640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cf400640 1 -- 172.21.15.67:0/1903188006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4003520 con 0x7f75b40051c0 2024-02-26T10:42:59.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.305+0000 7f75cea00640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75b40039b0 0x7f75b4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:42:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.306+0000 7f75cf400640 1 -- 172.21.15.67:0/1903188006 reap_dead start 2024-02-26T10:42:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.306+0000 7f75cf400640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75b400b9c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009ac0 tx=0x7f75b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e24a1b75558fc4c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.306+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8009690 con 0x7f75b40051c0 2024-02-26T10:42:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.306+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8004020 con 0x7f75b40051c0 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.306+0000 7f75cd600640 1 -- 172.21.15.67:0/1903188006 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004410 con 0x7f75b40051c0 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 -- 172.21.15.67:0/1903188006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 msgr2=0x7f75b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75b400b9c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009ac0 tx=0x7f75b4005830 comp rx=0 tx=0).stop 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 -- 172.21.15.67:0/1903188006 shutdown_connections 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1903188006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75b400b9c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 -- 172.21.15.67:0/1903188006 >> 172.21.15.67:0/1903188006 conn(0x7f75d00c63f0 msgr2=0x7f75d00c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 -- 172.21.15.67:0/1903188006 shutdown_connections 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.307+0000 7f75cfe00640 1 -- 172.21.15.67:0/1903188006 wait complete. 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.309+0000 7f75cfe00640 1 Processor -- start 2024-02-26T10:42:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.309+0000 7f75cfe00640 1 -- start start 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75d01374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75d00c3fe0 0x7f75d0135c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d0136670 0x7f75d00bb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00ac140 con 0x7f75b40051c0 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cfe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00c2ac0 con 0x7f75d0136670 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cfe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00ac2c0 con 0x7f75d00c3fe0 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75d01374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75d01374d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34948/0 (socket says 172.21.15.67:34948) 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.310+0000 7f75cea00640 1 -- 172.21.15.67:0/1917441605 learned_addr learned my addr 172.21.15.67:0/1917441605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:42:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75ce000640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75d00c3fe0 0x7f75d0135c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cf400640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d0136670 0x7f75d00bb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cea00640 1 -- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75d00c3fe0 msgr2=0x7f75d0135c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cea00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75d00c3fe0 0x7f75d0135c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cea00640 1 -- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d0136670 msgr2=0x7f75d00bb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cea00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d0136670 0x7f75d00bb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cea00640 1 -- 172.21.15.67:0/1917441605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00c4b70 con 0x7f75b40051c0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cea00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75d01374d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f75c0002230 tx=0x7f75c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0003b80 con 0x7f75b40051c0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0018070 con 0x7f75b40051c0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00050d0 con 0x7f75b40051c0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.311+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d0136cd0 con 0x7f75b40051c0 2024-02-26T10:42:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.312+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d01361b0 con 0x7f75b40051c0 2024-02-26T10:42:59.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.313+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d0074a30 con 0x7f75b40051c0 2024-02-26T10:42:59.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.314+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75c0005270 con 0x7f75b40051c0 2024-02-26T10:42:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.314+0000 7f75c6a00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:42:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.315+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f75c008a620 con 0x7f75b40051c0 2024-02-26T10:42:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.315+0000 7f75ce000640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:42:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.316+0000 7f75ce000640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75d013f9d0 tx=0x7f75b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:42:59.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.319+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75c001a310 con 0x7f75b40051c0 2024-02-26T10:42:59.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:42:59.517+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75d006ea20 con 0x7f75b40051c0 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.067+0000 7f75c6a00640 1 -- 172.21.15.67:0/1917441605 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) v1 ==== 181+0+0 (secure 0 0 0) 0x7f75c0059060 con 0x7f75b40051c0 2024-02-26T10:43:00.065 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 msgr2=0x7f75ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75d013f9d0 tx=0x7f75b8009040 comp rx=0 tx=0).stop 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 msgr2=0x7f75d01374d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75d01374d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f75c0002230 tx=0x7f75c000ec80 comp rx=0 tx=0).stop 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 shutdown_connections 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f75ac06b4d0 0x7f75ac06d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d0136670 0x7f75d00bb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75d00c3fe0 0x7f75d0135c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 --2- 172.21.15.67:0/1917441605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40051c0 0x7f75d01374d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 >> 172.21.15.67:0/1917441605 conn(0x7f75d00c63f0 msgr2=0x7f75d0145c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.071+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 shutdown_connections 2024-02-26T10:43:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.072+0000 7f75cfe00640 1 -- 172.21.15.67:0/1917441605 wait complete. 2024-02-26T10:43:00.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-26T10:43:00.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.309+0000 7fa6edc00640 1 Processor -- start 2024-02-26T10:43:00.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.310+0000 7fa6edc00640 1 -- start start 2024-02-26T10:43:00.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.311+0000 7fa6edc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6e80c5290 0x7fa6e81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.311+0000 7fa6edc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6e80af8a0 0x7fa6e80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.311+0000 7fa6edc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e806c110 con 0x7fa6e8155880 2024-02-26T10:43:00.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.311+0000 7fa6edc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e806d000 con 0x7fa6e80c5290 2024-02-26T10:43:00.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.311+0000 7fa6edc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e80c72c0 con 0x7fa6e80af8a0 2024-02-26T10:43:00.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.311+0000 7fa6e7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6e80c5290 0x7fa6e81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6e7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6e80c5290 0x7fa6e81562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50438/0 (socket says 172.21.15.67:50438) 2024-02-26T10:43:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6e7e00640 1 -- 172.21.15.67:0/3703648023 learned_addr learned my addr 172.21.15.67:0/3703648023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6ed200640 1 --2- 172.21.15.67:0/3703648023 >> v2:172.21.15.82:3300/0 conn(0x7fa6e80af8a0 0x7fa6e80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6e7e00640 1 -- 172.21.15.67:0/3703648023 >> v2:172.21.15.82:3300/0 conn(0x7fa6e80af8a0 msgr2=0x7fa6e80afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6e7e00640 1 --2- 172.21.15.67:0/3703648023 >> v2:172.21.15.82:3300/0 conn(0x7fa6e80af8a0 0x7fa6e80afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6e7e00640 1 -- 172.21.15.67:0/3703648023 >> v1:172.21.15.82:6789/0 conn(0x7fa6e8155880 legacy=0x7fa6e806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.312+0000 7fa6e7e00640 1 -- 172.21.15.67:0/3703648023 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80c75a0 con 0x7fa6e80c5290 2024-02-26T10:43:00.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.314+0000 7fa6e7e00640 1 --2- 172.21.15.67:0/3703648023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81562f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80ca650 tx=0x7fa6d802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee674c7dc6041b74 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.314+0000 7fa6e6a00640 1 -- 172.21.15.67:0/3703648023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d802e070 con 0x7fa6e80c5290 2024-02-26T10:43:00.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.314+0000 7fa6e6a00640 1 -- 172.21.15.67:0/3703648023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8037070 con 0x7fa6e80c5290 2024-02-26T10:43:00.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.314+0000 7fa6e6a00640 1 -- 172.21.15.67:0/3703648023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8031a00 con 0x7fa6e80c5290 2024-02-26T10:43:00.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 -- 172.21.15.67:0/3703648023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 msgr2=0x7fa6e81562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 --2- 172.21.15.67:0/3703648023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81562f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80ca650 tx=0x7fa6d802b880 comp rx=0 tx=0).stop 2024-02-26T10:43:00.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 -- 172.21.15.67:0/3703648023 shutdown_connections 2024-02-26T10:43:00.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 --2- 172.21.15.67:0/3703648023 >> v2:172.21.15.82:3300/0 conn(0x7fa6e80af8a0 0x7fa6e80afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 --2- 172.21.15.67:0/3703648023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81562f0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 -- 172.21.15.67:0/3703648023 >> 172.21.15.67:0/3703648023 conn(0x7fa6e80ac230 msgr2=0x7fa6e80ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:00.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 -- 172.21.15.67:0/3703648023 shutdown_connections 2024-02-26T10:43:00.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.315+0000 7fa6edc00640 1 -- 172.21.15.67:0/3703648023 wait complete. 2024-02-26T10:43:00.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.317+0000 7fa6edc00640 1 Processor -- start 2024-02-26T10:43:00.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.317+0000 7fa6edc00640 1 -- start start 2024-02-26T10:43:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6edc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6e80af8a0 0x7fa6e80c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6edc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6edc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e8155880 0x7fa6e81514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6edc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e8155cf0 con 0x7fa6e80c5290 2024-02-26T10:43:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6edc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e806c3d0 con 0x7fa6e8155880 2024-02-26T10:43:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6edc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80c5670 con 0x7fa6e80af8a0 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50442/0 (socket says 172.21.15.67:50442) 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6e7400640 1 -- 172.21.15.67:0/439594614 learned_addr learned my addr 172.21.15.67:0/439594614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6e7e00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6e80af8a0 0x7fa6e80c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.318+0000 7fa6ed200640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e8155880 0x7fa6e81514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6e7400640 1 -- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6e80af8a0 msgr2=0x7fa6e80c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6e7400640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6e80af8a0 0x7fa6e80c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6e7400640 1 -- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e8155880 msgr2=0x7fa6e81514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6e7400640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e8155880 0x7fa6e81514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6e7400640 1 -- 172.21.15.67:0/439594614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80c12a0 con 0x7fa6e80c5290 2024-02-26T10:43:00.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6ed200640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e8155880 0x7fa6e81514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6e7400640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81547e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000b3f0 tx=0x7fa6d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0015020 con 0x7fa6e80c5290 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d0011830 con 0x7fa6e80c5290 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e8151fc0 con 0x7fa6e80c5290 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.319+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0011c20 con 0x7fa6e80c5290 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.320+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80c2c70 con 0x7fa6e80c5290 2024-02-26T10:43:00.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.321+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e8074a30 con 0x7fa6e80c5290 2024-02-26T10:43:00.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.322+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6d0018020 con 0x7fa6e80c5290 2024-02-26T10:43:00.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.323+0000 7fa6dfe00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6c406b550 0x7fa6c406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:00.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.323+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6213+0+0 (secure 0 0 0) 0x7fa6d0089ff0 con 0x7fa6e80c5290 2024-02-26T10:43:00.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.323+0000 7fa6e7e00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6c406b550 0x7fa6c406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:00.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.324+0000 7fa6e7e00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6c406b550 0x7fa6c406da10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8036000 tx=0x7fa6d803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:00.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.327+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6d001f030 con 0x7fa6e80c5290 2024-02-26T10:43:00.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:00.529+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa6e806d950 con 0x7fa6e80c5290 2024-02-26T10:43:01.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:01.040 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.077+0000 7fa6dfe00640 1 -- 172.21.15.67:0/439594614 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa6d00172c0 con 0x7fa6e80c5290 2024-02-26T10:43:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-02-26T10:43:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.082+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6c406b550 msgr2=0x7fa6c406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.082+0000 7fa6edc00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6c406b550 0x7fa6c406da10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8036000 tx=0x7fa6d803d040 comp rx=0 tx=0).stop 2024-02-26T10:43:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.082+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 msgr2=0x7fa6e81547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.082+0000 7fa6edc00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81547e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000b3f0 tx=0x7fa6d000b8b0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.085+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 shutdown_connections 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.085+0000 7fa6edc00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa6c406b550 0x7fa6c406da10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.085+0000 7fa6edc00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e8155880 0x7fa6e81514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.085+0000 7fa6edc00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6e80c5290 0x7fa6e81547e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.085+0000 7fa6edc00640 1 --2- 172.21.15.67:0/439594614 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6e80af8a0 0x7fa6e80c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.085+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 >> 172.21.15.67:0/439594614 conn(0x7fa6e80ac230 msgr2=0x7fa6e80aa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.087+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 shutdown_connections 2024-02-26T10:43:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.087+0000 7fa6edc00640 1 -- 172.21.15.67:0/439594614 wait complete. 2024-02-26T10:43:01.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.363+0000 7f1ca0e00640 1 Processor -- start 2024-02-26T10:43:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.364+0000 7f1ca0e00640 1 -- start start 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1ca0e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c9c0c8570 0x7f1c9c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c9c06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1ca0e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c0c5100 con 0x7f1c9c14ee60 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1ca0e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c14f240 con 0x7f1c9c0c8570 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c06c3f0 con 0x7f1c9c0bdd90 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1c9b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1c9c14ee60 0x7f1c9c1557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33234/0 (socket says 172.21.15.67:33234) 2024-02-26T10:43:01.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.365+0000 7f1c9b400640 1 -- 172.21.15.67:0/509631007 learned_addr learned my addr 172.21.15.67:0/509631007 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9aa00640 1 --2- 172.21.15.67:0/509631007 >> v2:172.21.15.82:3300/0 conn(0x7f1c9c0c8570 0x7f1c9c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9be00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c9c06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260311036 0 0) 0x7f1c9c0c5100 con 0x7f1c9c14ee60 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c80003520 con 0x7f1c9c14ee60 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 899671051 0 0) 0x7f1c80003520 con 0x7f1c9c14ee60 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 msgr2=0x7f1c9c06c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c9c06c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 >> v2:172.21.15.82:3300/0 conn(0x7f1c9c0c8570 msgr2=0x7f1c9c0c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 --2- 172.21.15.67:0/509631007 >> v2:172.21.15.82:3300/0 conn(0x7f1c9c0c8570 0x7f1c9c0c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0c53e0 con 0x7f1c9c14ee60 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f1c8c001670 con 0x7f1c9c14ee60 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.366+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 >> v1:172.21.15.67:6789/0 conn(0x7f1c9c14ee60 legacy=0x7f1c9c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9a000640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c800039b0 0x7f1c80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9a000640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9a000640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80004d40 0x7f1c800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c14f240 con 0x7f1c80004d40 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c06c3f0 con 0x7f1c800039b0 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8000ba40 con 0x7f1c9c0bdd90 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9be00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80004d40 0x7f1c800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9b400640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c800039b0 0x7f1c80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9aa00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9be00640 1 -- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 msgr2=0x7f1c80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9be00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9be00640 1 -- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c800039b0 msgr2=0x7f1c80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9be00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c800039b0 0x7f1c80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9be00640 1 -- 172.21.15.67:0/509631007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c80003520 con 0x7f1c80004d40 2024-02-26T10:43:01.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.367+0000 7f1c9aa00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0bdd90 0x7f1c80004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:01.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.368+0000 7f1c9be00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80004d40 0x7f1c800051c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009f90 tx=0x7f1c80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fff6b7bf40452b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94039070 con 0x7f1c80004d40 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9402cbd0 con 0x7f1c80004d40 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1c9a000640 1 -- 172.21.15.67:0/509631007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94009c10 con 0x7f1c80004d40 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1ca0e00640 1 -- 172.21.15.67:0/509631007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80004d40 msgr2=0x7f1c800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/509631007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80004d40 0x7f1c800051c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009f90 tx=0x7f1c80005830 comp rx=0 tx=0).stop 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1c9be00640 1 -- 172.21.15.67:0/509631007 reap_dead start 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1ca0e00640 1 -- 172.21.15.67:0/509631007 shutdown_connections 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1ca0e00640 1 -- 172.21.15.67:0/509631007 >> 172.21.15.67:0/509631007 conn(0x7f1c9c072ea0 msgr2=0x7f1c9c0ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.369+0000 7f1ca0e00640 1 -- 172.21.15.67:0/509631007 shutdown_connections 2024-02-26T10:43:01.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.370+0000 7f1ca0e00640 1 -- 172.21.15.67:0/509631007 wait complete. 2024-02-26T10:43:01.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.371+0000 7f1ca0e00640 1 Processor -- start 2024-02-26T10:43:01.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.371+0000 7f1ca0e00640 1 -- start start 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80004d40 0x7f1c9c1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0ad9a0 0x7f1c9c0c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1ca0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 0x7f1c9c0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0c7740 con 0x7f1c9c0ade00 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0c75c0 con 0x7f1c80004d40 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1ca0e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0c78c0 con 0x7f1c9c0ad9a0 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.372+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 0x7f1c9c0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 0x7f1c9c0c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50468/0 (socket says 172.21.15.67:50468) 2024-02-26T10:43:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 -- 172.21.15.67:0/2100235300 learned_addr learned my addr 172.21.15.67:0/2100235300 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:01.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9aa00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0ad9a0 0x7f1c9c0c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9b400640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80004d40 0x7f1c9c1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 -- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0ad9a0 msgr2=0x7f1c9c0c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0ad9a0 0x7f1c9c0c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 -- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80004d40 msgr2=0x7f1c9c1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80004d40 0x7f1c9c1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 -- 172.21.15.67:0/2100235300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0c40b0 con 0x7f1c9c0ade00 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9aa00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0ad9a0 0x7f1c9c0c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c9be00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 0x7f1c9c0c3510 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e230 tx=0x7f1c94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.373+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94039070 con 0x7f1c9c0ade00 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.374+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9403a040 con 0x7f1c9c0ade00 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.374+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c0c4390 con 0x7f1c9c0ade00 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.374+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94009c10 con 0x7f1c9c0ade00 2024-02-26T10:43:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.374+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c1576f0 con 0x7f1c9c0ade00 2024-02-26T10:43:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.375+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c074a30 con 0x7f1c9c0ade00 2024-02-26T10:43:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.376+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c94046020 con 0x7f1c9c0ade00 2024-02-26T10:43:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.377+0000 7f1c93400640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.377+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f1c940abfd0 con 0x7f1c9c0ade00 2024-02-26T10:43:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.378+0000 7f1c9b400640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.379+0000 7f1c9b400640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c001eb0 tx=0x7f1c8c000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.381+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c9407a710 con 0x7f1c9c0ade00 2024-02-26T10:43:01.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.582+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c9c06d9e0 con 0x7f1c9c0ade00 2024-02-26T10:43:01.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.585+0000 7f1c93400640 1 -- 172.21.15.67:0/2100235300 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f1c9407e7b0 con 0x7f1c9c0ade00 2024-02-26T10:43:01.583 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:01.583 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:43:01.068125+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6835/2183069114":"2024-02-27T10:42:58.049528+0000","172.21.15.67:6833/2183069114":"2024-02-27T10:42:58.049528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:01.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.587+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1c7c06b500 msgr2=0x7f1c7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.587+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c001eb0 tx=0x7f1c8c000ed0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.587+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 msgr2=0x7f1c9c0c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.587+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 0x7f1c9c0c3510 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e230 tx=0x7f1c94009750 comp rx=0 tx=0).stop 2024-02-26T10:43:01.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 shutdown_connections 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c9c0ade00 0x7f1c9c0c3510 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c9c0ad9a0 0x7f1c9c0c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 --2- 172.21.15.67:0/2100235300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80004d40 0x7f1c9c1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 >> 172.21.15.67:0/2100235300 conn(0x7f1c9c072ea0 msgr2=0x7f1c9c0a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 shutdown_connections 2024-02-26T10:43:01.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.588+0000 7f1ca0e00640 1 -- 172.21.15.67:0/2100235300 wait complete. 2024-02-26T10:43:01.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-26T10:38:28.395212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-02-26T10:42:17.695706+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-02-26T10:42:19.039564+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-26T10:43:01.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-26T10:43:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.830+0000 7f37f9c00640 1 Processor -- start 2024-02-26T10:43:01.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.831+0000 7f37f9c00640 1 -- start start 2024-02-26T10:43:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.832+0000 7f37f9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f40c5290 0x7f37f41562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.832+0000 7f37f9c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f37f4155880 0x7f37f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.832+0000 7f37f9c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f406c110 con 0x7f37f406ce30 2024-02-26T10:43:01.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.832+0000 7f37f9c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f40c72c0 con 0x7f37f4155880 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.832+0000 7f37f9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40c75a0 con 0x7f37f40c5290 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.832+0000 7f37f3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f37f4155880 0x7f37f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f37f4155880 0x7f37f406c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50472/0 (socket says 172.21.15.67:50472) 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3400640 1 -- 172.21.15.67:0/1007604883 learned_addr learned my addr 172.21.15.67:0/1007604883 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3e00640 1 --2- 172.21.15.67:0/1007604883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f40c5290 0x7f37f41562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3400640 1 -- 172.21.15.67:0/1007604883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f40c5290 msgr2=0x7f37f41562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3400640 1 --2- 172.21.15.67:0/1007604883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f40c5290 0x7f37f41562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3400640 1 -- 172.21.15.67:0/1007604883 >> v1:172.21.15.82:6789/0 conn(0x7f37f406ce30 legacy=0x7f37f40af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:01.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.833+0000 7f37f3400640 1 -- 172.21.15.67:0/1007604883 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40c7880 con 0x7f37f4155880 2024-02-26T10:43:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.834+0000 7f37f3400640 1 --2- 172.21.15.67:0/1007604883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f4155880 0x7f37f406c8f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009870 tx=0x7f37dc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2af4d99db8d43d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.835+0000 7f37f2a00640 1 -- 172.21.15.67:0/1007604883 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0317c0 con 0x7f37f4155880 2024-02-26T10:43:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.835+0000 7f37f2a00640 1 -- 172.21.15.67:0/1007604883 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc037070 con 0x7f37f4155880 2024-02-26T10:43:01.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.835+0000 7f37f2a00640 1 -- 172.21.15.67:0/1007604883 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc031b80 con 0x7f37f4155880 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.835+0000 7f37f9c00640 1 -- 172.21.15.67:0/1007604883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f4155880 msgr2=0x7f37f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.835+0000 7f37f9c00640 1 --2- 172.21.15.67:0/1007604883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f4155880 0x7f37f406c8f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009870 tx=0x7f37dc02b860 comp rx=0 tx=0).stop 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.836+0000 7f37f9c00640 1 -- 172.21.15.67:0/1007604883 shutdown_connections 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.836+0000 7f37f9c00640 1 --2- 172.21.15.67:0/1007604883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f4155880 0x7f37f406c8f0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.836+0000 7f37f9c00640 1 --2- 172.21.15.67:0/1007604883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f40c5290 0x7f37f41562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.836+0000 7f37f9c00640 1 -- 172.21.15.67:0/1007604883 >> 172.21.15.67:0/1007604883 conn(0x7f37f40ac230 msgr2=0x7f37f40ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.836+0000 7f37f9c00640 1 -- 172.21.15.67:0/1007604883 shutdown_connections 2024-02-26T10:43:01.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.836+0000 7f37f9c00640 1 -- 172.21.15.67:0/1007604883 wait complete. 2024-02-26T10:43:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.838+0000 7f37f9c00640 1 Processor -- start 2024-02-26T10:43:01.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.838+0000 7f37f9c00640 1 -- start start 2024-02-26T10:43:01.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.839+0000 7f37f9c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f40c5290 0x7f37f41eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.839+0000 7f37f9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f4155880 0x7f37f40ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.839+0000 7f37f9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 0x7f37f40c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.839+0000 7f37f9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40affb0 con 0x7f37f414f490 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.839+0000 7f37f9c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f4155cc0 con 0x7f37f40c5290 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.839+0000 7f37f9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40b0130 con 0x7f37f4155880 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 0x7f37f40c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 0x7f37f40c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50482/0 (socket says 172.21.15.67:50482) 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 -- 172.21.15.67:0/4145553826 learned_addr learned my addr 172.21.15.67:0/4145553826 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f3400640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f4155880 0x7f37f40ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f3e00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f40c5290 0x7f37f41eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 -- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f4155880 msgr2=0x7f37f40ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f4155880 0x7f37f40ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 -- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f40c5290 msgr2=0x7f37f41eeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f40c5290 0x7f37f41eeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.840+0000 7f37f9200640 1 -- 172.21.15.67:0/4145553826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f41eefe0 con 0x7f37f414f490 2024-02-26T10:43:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.841+0000 7f37f9200640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 0x7f37f40c3130 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00ba10 tx=0x7f37ec00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.841+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec019070 con 0x7f37f414f490 2024-02-26T10:43:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.841+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec00cae0 con 0x7f37f414f490 2024-02-26T10:43:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.841+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec014440 con 0x7f37f414f490 2024-02-26T10:43:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.841+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f40c99f0 con 0x7f37f414f490 2024-02-26T10:43:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.841+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f40c9fb0 con 0x7f37f414f490 2024-02-26T10:43:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.843+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f40c6970 con 0x7f37f414f490 2024-02-26T10:43:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.843+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37ec01c070 con 0x7f37f414f490 2024-02-26T10:43:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.845+0000 7f37ebe00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37d006b500 0x7f37d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.845+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f37ec089de0 con 0x7f37f414f490 2024-02-26T10:43:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.845+0000 7f37f3e00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37d006b500 0x7f37d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:01.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.846+0000 7f37f3e00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37d006b500 0x7f37d006d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006fd0 tx=0x7f37e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:01.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:01.849+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37f40c6970 con 0x7f37f414f490 2024-02-26T10:43:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.046+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37f4074a30 con 0x7f37f414f490 2024-02-26T10:43:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.047+0000 7f37ebe00640 1 -- 172.21.15.67:0/4145553826 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37f41eefe0 con 0x7f37f414f490 2024-02-26T10:43:02.044 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37d006b500 msgr2=0x7f37d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37d006b500 0x7f37d006d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f37e4006fd0 tx=0x7f37e4008040 comp rx=0 tx=0).stop 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 msgr2=0x7f37f40c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 0x7f37f40c3130 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00ba10 tx=0x7f37ec00bed0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 shutdown_connections 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37f414f490 0x7f37f40c3130 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37f4155880 0x7f37f40ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f37d006b500 0x7f37d006d9c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 --2- 172.21.15.67:0/4145553826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f40c5290 0x7f37f41eeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 >> 172.21.15.67:0/4145553826 conn(0x7f37f40ac230 msgr2=0x7f37f4152980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.049+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 shutdown_connections 2024-02-26T10:43:02.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.050+0000 7f37f9c00640 1 -- 172.21.15.67:0/4145553826 wait complete. 2024-02-26T10:43:02.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-02-26T10:43:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.292+0000 7f5a7b200640 1 Processor -- start 2024-02-26T10:43:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.293+0000 7f5a7b200640 1 -- start start 2024-02-26T10:43:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c0c5290 0x7f5a7c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a7c0af8a0 0x7f5a7c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c06c110 con 0x7f5a7c155880 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c06d000 con 0x7f5a7c0af8a0 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c72c0 con 0x7f5a7c0c5290 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a7c0af8a0 0x7f5a7c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a7c0af8a0 0x7f5a7c0afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50490/0 (socket says 172.21.15.67:50490) 2024-02-26T10:43:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.294+0000 7f5a7a800640 1 -- 172.21.15.67:0/2977464049 learned_addr learned my addr 172.21.15.67:0/2977464049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.295+0000 7f5a79e00640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c0c5290 0x7f5a7c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.295+0000 7f5a7a800640 1 -- 172.21.15.67:0/2977464049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c0c5290 msgr2=0x7f5a7c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.295+0000 7f5a7a800640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c0c5290 0x7f5a7c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.295+0000 7f5a7a800640 1 -- 172.21.15.67:0/2977464049 >> v1:172.21.15.67:6789/0 conn(0x7f5a7c155880 legacy=0x7f5a7c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.295+0000 7f5a7a800640 1 -- 172.21.15.67:0/2977464049 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0c75a0 con 0x7f5a7c0af8a0 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.295+0000 7f5a79e00640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c0c5290 0x7f5a7c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.296+0000 7f5a7a800640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0af8a0 0x7f5a7c0afd30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009870 tx=0x7f5a7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e136cace23a1c2b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.296+0000 7f5a78a00640 1 -- 172.21.15.67:0/2977464049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74035020 con 0x7f5a7c0af8a0 2024-02-26T10:43:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.296+0000 7f5a78a00640 1 -- 172.21.15.67:0/2977464049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a74031640 con 0x7f5a7c0af8a0 2024-02-26T10:43:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.296+0000 7f5a78a00640 1 -- 172.21.15.67:0/2977464049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7402d040 con 0x7f5a7c0af8a0 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 -- 172.21.15.67:0/2977464049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0af8a0 msgr2=0x7f5a7c0afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0af8a0 0x7f5a7c0afd30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009870 tx=0x7f5a7402b8f0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 -- 172.21.15.67:0/2977464049 shutdown_connections 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0af8a0 0x7f5a7c0afd30 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 --2- 172.21.15.67:0/2977464049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c0c5290 0x7f5a7c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 -- 172.21.15.67:0/2977464049 >> 172.21.15.67:0/2977464049 conn(0x7f5a7c0ac230 msgr2=0x7f5a7c0ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 -- 172.21.15.67:0/2977464049 shutdown_connections 2024-02-26T10:43:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.297+0000 7f5a7b200640 1 -- 172.21.15.67:0/2977464049 wait complete. 2024-02-26T10:43:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 Processor -- start 2024-02-26T10:43:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 -- start start 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a7c0af8a0 0x7f5a7c1ee900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c155880 0x7f5a7c1eb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c155cc0 con 0x7f5a7c0c5290 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c06c3d0 con 0x7f5a7c0af8a0 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.299+0000 7f5a7b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c5670 con 0x7f5a7c155880 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50504/0 (socket says 172.21.15.67:50504) 2024-02-26T10:43:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 -- 172.21.15.67:0/99539982 learned_addr learned my addr 172.21.15.67:0/99539982 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 -- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c155880 msgr2=0x7f5a7c1eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c155880 0x7f5a7c1eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 -- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a7c0af8a0 msgr2=0x7f5a7c1ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a7c0af8a0 0x7f5a7c1ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 -- 172.21.15.67:0/99539982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0c35f0 con 0x7f5a7c0c5290 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.300+0000 7f5a79400640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1544d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c730 tx=0x7f5a6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.301+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c016020 con 0x7f5a7c0c5290 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.301+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c006d50 con 0x7f5a7c0c5290 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.301+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c003bb0 con 0x7f5a7c0c5290 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.301+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c0c38d0 con 0x7f5a7c0c5290 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.301+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c1eab70 con 0x7f5a7c0c5290 2024-02-26T10:43:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.302+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c0c6970 con 0x7f5a7c0c5290 2024-02-26T10:43:02.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.303+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a6c00a850 con 0x7f5a7c0c5290 2024-02-26T10:43:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.304+0000 7f5a72000640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a5806b500 0x7f5a5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.304+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f5a6c089ea0 con 0x7f5a7c0c5290 2024-02-26T10:43:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.304+0000 7f5a79e00640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a5806b500 0x7f5a5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.305+0000 7f5a79e00640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a5806b500 0x7f5a5806d9c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006fd0 tx=0x7f5a64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.308+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a7c0c6970 con 0x7f5a7c0c5290 2024-02-26T10:43:02.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.513+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a7c1ea980 con 0x7f5a7c0c5290 2024-02-26T10:43:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.514+0000 7f5a72000640 1 -- 172.21.15.67:0/99539982 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v83) v1 ==== 105+0+11 (secure 0 0 0) 0x7f5a6c020020 con 0x7f5a7c0c5290 2024-02-26T10:43:02.512 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a5806b500 msgr2=0x7f5a5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a5806b500 0x7f5a5806d9c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006fd0 tx=0x7f5a64008040 comp rx=0 tx=0).stop 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 msgr2=0x7f5a7c1544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1544d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c730 tx=0x7f5a6c00cbf0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 shutdown_connections 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5a7c155880 0x7f5a7c1eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1544d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f5a5806b500 0x7f5a5806d9c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 --2- 172.21.15.67:0/99539982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a7c0af8a0 0x7f5a7c1ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 >> 172.21.15.67:0/99539982 conn(0x7f5a7c0ac230 msgr2=0x7f5a7c0c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 shutdown_connections 2024-02-26T10:43:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.517+0000 7f5a7b200640 1 -- 172.21.15.67:0/99539982 wait complete. 2024-02-26T10:43:02.533 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-02-26T10:43:02.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.751+0000 7fde85200640 1 Processor -- start 2024-02-26T10:43:02.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.751+0000 7fde85200640 1 -- start start 2024-02-26T10:43:02.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde85200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde80155c80 0x7fde800c1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde85200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800c3d20 con 0x7fde800c0b40 2024-02-26T10:43:02.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde85200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde80072bc0 con 0x7fde801558a0 2024-02-26T10:43:02.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde85200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c1e50 con 0x7fde80155c80 2024-02-26T10:43:02.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde7e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fde800c0b40 0x7fde800c1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33252/0 (socket says 172.21.15.67:33252) 2024-02-26T10:43:02.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde7e800640 1 -- 172.21.15.67:0/3224651538 learned_addr learned my addr 172.21.15.67:0/3224651538 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.753+0000 7fde7d400640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde80155c80 0x7fde800c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494794965 0 0) 0x7fde800c3d20 con 0x7fde800c0b40 2024-02-26T10:43:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde64003520 con 0x7fde800c0b40 2024-02-26T10:43:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933071691 0 0) 0x7fde64003520 con 0x7fde800c0b40 2024-02-26T10:43:02.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde80155c80 msgr2=0x7fde800c1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde80155c80 0x7fde800c1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 >> v1:172.21.15.82:6789/0 conn(0x7fde801558a0 legacy=0x7fde80071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80072ea0 con 0x7fde800c0b40 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fde780019f0 con 0x7fde800c0b40 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 >> v1:172.21.15.67:6789/0 conn(0x7fde800c0b40 legacy=0x7fde800c1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde640039e0 0x7fde64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde64004800 0x7fde64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde640051c0 0x7fde6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c1e50 con 0x7fde640039e0 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6400bf00 con 0x7fde64004800 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6400c080 con 0x7fde640051c0 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.754+0000 7fde7de00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde640039e0 0x7fde64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7e800640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde640051c0 0x7fde6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7d400640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde64004800 0x7fde64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7de00640 1 -- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde640051c0 msgr2=0x7fde6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:02.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7de00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde640051c0 0x7fde6400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7de00640 1 -- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde64004800 msgr2=0x7fde64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:02.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7de00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde64004800 0x7fde64004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.755+0000 7fde7de00640 1 -- 172.21.15.67:0/3224651538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64003520 con 0x7fde640039e0 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.756+0000 7fde7de00640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde640039e0 0x7fde64003e00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fde70001750 tx=0x7fde64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c84c1d0ed8ebaca server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.756+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70001370 con 0x7fde640039e0 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.756+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde700052d0 con 0x7fde640039e0 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.756+0000 7fde7ca00640 1 -- 172.21.15.67:0/3224651538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70003830 con 0x7fde640039e0 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde85200640 1 -- 172.21.15.67:0/3224651538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde640039e0 msgr2=0x7fde64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde85200640 1 --2- 172.21.15.67:0/3224651538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde640039e0 0x7fde64003e00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fde70001750 tx=0x7fde64005830 comp rx=0 tx=0).stop 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde7e800640 1 -- 172.21.15.67:0/3224651538 reap_dead start 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde85200640 1 -- 172.21.15.67:0/3224651538 shutdown_connections 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde85200640 1 -- 172.21.15.67:0/3224651538 >> 172.21.15.67:0/3224651538 conn(0x7fde800c7b00 msgr2=0x7fde800c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde85200640 1 -- 172.21.15.67:0/3224651538 shutdown_connections 2024-02-26T10:43:02.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.757+0000 7fde85200640 1 -- 172.21.15.67:0/3224651538 wait complete. 2024-02-26T10:43:02.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.759+0000 7fde85200640 1 Processor -- start 2024-02-26T10:43:02.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.759+0000 7fde85200640 1 -- start start 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde85200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640039e0 0x7fde80144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde85200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde800aaaa0 0x7fde80143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde85200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 0x7fde80157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde85200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c3a50 con 0x7fde800ab470 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde85200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c38d0 con 0x7fde640039e0 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde85200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c2110 con 0x7fde800aaaa0 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde7e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 0x7fde80157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde7e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 0x7fde80157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50520/0 (socket says 172.21.15.67:50520) 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde7e800640 1 -- 172.21.15.67:0/2202332301 learned_addr learned my addr 172.21.15.67:0/2202332301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:02.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.760+0000 7fde7de00640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640039e0 0x7fde80144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7d400640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde800aaaa0 0x7fde80143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7e800640 1 -- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde800aaaa0 msgr2=0x7fde80143430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7e800640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde800aaaa0 0x7fde80143430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7e800640 1 -- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640039e0 msgr2=0x7fde80144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7e800640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640039e0 0x7fde80144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7e800640 1 -- 172.21.15.67:0/2202332301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde801588e0 con 0x7fde800ab470 2024-02-26T10:43:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde7e800640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 0x7fde80157bf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fde78001580 tx=0x7fde7800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800bb80 con 0x7fde800ab470 2024-02-26T10:43:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800bd20 con 0x7fde800ab470 2024-02-26T10:43:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78007290 con 0x7fde800ab470 2024-02-26T10:43:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.761+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde801eb990 con 0x7fde800ab470 2024-02-26T10:43:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.762+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80143e40 con 0x7fde800ab470 2024-02-26T10:43:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.763+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde80156610 con 0x7fde800ab470 2024-02-26T10:43:02.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.764+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde7801b060 con 0x7fde800ab470 2024-02-26T10:43:02.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.765+0000 7fde76000640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fde6006b500 0x7fde6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:02.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.765+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fde7808a280 con 0x7fde800ab470 2024-02-26T10:43:02.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.765+0000 7fde7de00640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fde6006b500 0x7fde6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:02.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.766+0000 7fde7de00640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fde6006b500 0x7fde6006d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fde70033000 tx=0x7fde70009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:02.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.769+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde80156610 con 0x7fde800ab470 2024-02-26T10:43:02.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.968+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fde80074a30 con 0x7fde800ab470 2024-02-26T10:43:02.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.969+0000 7fde76000640 1 -- 172.21.15.67:0/2202332301 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v83) v1 ==== 101+0+11 (secure 0 0 0) 0x7fde78059000 con 0x7fde800ab470 2024-02-26T10:43:02.967 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fde6006b500 msgr2=0x7fde6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fde6006b500 0x7fde6006d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fde70033000 tx=0x7fde70009030 comp rx=0 tx=0).stop 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 msgr2=0x7fde80157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 0x7fde80157bf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fde78001580 tx=0x7fde7800b4a0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 shutdown_connections 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde800ab470 0x7fde80157bf0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde800aaaa0 0x7fde80143430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fde6006b500 0x7fde6006d9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 --2- 172.21.15.67:0/2202332301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640039e0 0x7fde80144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.972+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 >> 172.21.15.67:0/2202332301 conn(0x7fde800c7b00 msgr2=0x7fde800c46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.973+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 shutdown_connections 2024-02-26T10:43:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:02.973+0000 7fde85200640 1 -- 172.21.15.67:0/2202332301 wait complete. 2024-02-26T10:43:02.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:43:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.204+0000 7f6f2ec00640 1 Processor -- start 2024-02-26T10:43:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.204+0000 7f6f2ec00640 1 -- start start 2024-02-26T10:43:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.205+0000 7f6f2ec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f2806ac20 0x7f6f280a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ec00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f2806b000 0x7f6f2806db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f2806b460 con 0x7f6f28153610 2024-02-26T10:43:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ec00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f280a9a80 con 0x7f6f2806b000 2024-02-26T10:43:03.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f280a9d60 con 0x7f6f2806ac20 2024-02-26T10:43:03.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f2806b000 0x7f6f2806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6f2806b000 0x7f6f2806db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50522/0 (socket says 172.21.15.67:50522) 2024-02-26T10:43:03.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ce00640 1 -- 172.21.15.67:0/806790057 learned_addr learned my addr 172.21.15.67:0/806790057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2d800640 1 --2- 172.21.15.67:0/806790057 >> v2:172.21.15.82:3300/0 conn(0x7f6f2806ac20 0x7f6f280a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ce00640 1 -- 172.21.15.67:0/806790057 >> v2:172.21.15.82:3300/0 conn(0x7f6f2806ac20 msgr2=0x7f6f280a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ce00640 1 --2- 172.21.15.67:0/806790057 >> v2:172.21.15.82:3300/0 conn(0x7f6f2806ac20 0x7f6f280a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.206+0000 7f6f2ce00640 1 -- 172.21.15.67:0/806790057 >> v1:172.21.15.82:6789/0 conn(0x7f6f28153610 legacy=0x7f6f280abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.207+0000 7f6f2ce00640 1 -- 172.21.15.67:0/806790057 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280ab750 con 0x7f6f2806b000 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.207+0000 7f6f2d800640 1 --2- 172.21.15.67:0/806790057 >> v2:172.21.15.82:3300/0 conn(0x7f6f2806ac20 0x7f6f280a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:03.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.208+0000 7f6f2ce00640 1 --2- 172.21.15.67:0/806790057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f2806b000 0x7f6f2806db00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6f2806ecb0 tx=0x7f6f1402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fcd30f11196c5b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.208+0000 7f6f27e00640 1 -- 172.21.15.67:0/806790057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14035020 con 0x7f6f2806b000 2024-02-26T10:43:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.208+0000 7f6f27e00640 1 -- 172.21.15.67:0/806790057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14031870 con 0x7f6f2806b000 2024-02-26T10:43:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.208+0000 7f6f27e00640 1 -- 172.21.15.67:0/806790057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14031c40 con 0x7f6f2806b000 2024-02-26T10:43:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.208+0000 7f6f2ec00640 1 -- 172.21.15.67:0/806790057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f2806b000 msgr2=0x7f6f2806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.208+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/806790057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f2806b000 0x7f6f2806db00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6f2806ecb0 tx=0x7f6f1402b910 comp rx=0 tx=0).stop 2024-02-26T10:43:03.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.209+0000 7f6f2ec00640 1 -- 172.21.15.67:0/806790057 shutdown_connections 2024-02-26T10:43:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.209+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/806790057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f2806b000 0x7f6f2806db00 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.209+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/806790057 >> v2:172.21.15.82:3300/0 conn(0x7f6f2806ac20 0x7f6f280a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.209+0000 7f6f2ec00640 1 -- 172.21.15.67:0/806790057 >> 172.21.15.67:0/806790057 conn(0x7f6f280c63f0 msgr2=0x7f6f280c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.209+0000 7f6f2ec00640 1 -- 172.21.15.67:0/806790057 shutdown_connections 2024-02-26T10:43:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.209+0000 7f6f2ec00640 1 -- 172.21.15.67:0/806790057 wait complete. 2024-02-26T10:43:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.210+0000 7f6f2ec00640 1 Processor -- start 2024-02-26T10:43:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 -- start start 2024-02-26T10:43:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2806ac20 0x7f6f28152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f2806b000 0x7f6f281ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 0x7f6f28072750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28150410 con 0x7f6f28153610 2024-02-26T10:43:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28153a60 con 0x7f6f2806ac20 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.211+0000 7f6f2ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28150590 con 0x7f6f2806b000 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 0x7f6f28072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 0x7f6f28072750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50534/0 (socket says 172.21.15.67:50534) 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 -- 172.21.15.67:0/817665602 learned_addr learned my addr 172.21.15.67:0/817665602 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2ce00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f2806b000 0x7f6f281ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 -- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f2806b000 msgr2=0x7f6f281ee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f2806b000 0x7f6f281ee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 -- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2806ac20 msgr2=0x7f6f28152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2806ac20 0x7f6f28152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2e200640 1 -- 172.21.15.67:0/817665602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f281512c0 con 0x7f6f28153610 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.212+0000 7f6f2ce00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f2806b000 0x7f6f281ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.213+0000 7f6f2e200640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 0x7f6f28072750 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000b970 tx=0x7f6f2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.213+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20019070 con 0x7f6f28153610 2024-02-26T10:43:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.213+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2000cb40 con 0x7f6f28153610 2024-02-26T10:43:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.213+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20014440 con 0x7f6f28153610 2024-02-26T10:43:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.213+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f281515a0 con 0x7f6f28153610 2024-02-26T10:43:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.213+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f281486e0 con 0x7f6f28153610 2024-02-26T10:43:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.214+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f28074a30 con 0x7f6f28153610 2024-02-26T10:43:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.215+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f2001c070 con 0x7f6f28153610 2024-02-26T10:43:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.216+0000 7f6f25600640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f0806b500 0x7f6f0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.216+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f6f20089f30 con 0x7f6f28153610 2024-02-26T10:43:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.217+0000 7f6f2d800640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f0806b500 0x7f6f0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.218+0000 7f6f2d800640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f0806b500 0x7f6f0806d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c006fd0 tx=0x7f6f1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.220+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f20058cc0 con 0x7f6f28153610 2024-02-26T10:43:03.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.460+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f2814bbf0 con 0x7f6f28153610 2024-02-26T10:43:03.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.461+0000 7f6f25600640 1 -- 172.21.15.67:0/817665602 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6f200585c0 con 0x7f6f28153610 2024-02-26T10:43:03.459 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:03.459 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3395706465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3395706465},{"type":"v1","addr":"172.21.15.67:6835","nonce":3395706465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:43:03.460 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 55 2024-02-26T10:43:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f0806b500 msgr2=0x7f6f0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f0806b500 0x7f6f0806d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c006fd0 tx=0x7f6f1c008040 comp rx=0 tx=0).stop 2024-02-26T10:43:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 msgr2=0x7f6f28072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 0x7f6f28072750 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000b970 tx=0x7f6f2000be30 comp rx=0 tx=0).stop 2024-02-26T10:43:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 shutdown_connections 2024-02-26T10:43:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f28153610 0x7f6f28072750 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f2806b000 0x7f6f281ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6f0806b500 0x7f6f0806d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 --2- 172.21.15.67:0/817665602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f2806ac20 0x7f6f28152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.464+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 >> 172.21.15.67:0/817665602 conn(0x7f6f280c63f0 msgr2=0x7f6f280c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.465+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 shutdown_connections 2024-02-26T10:43:03.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.465+0000 7f6f2ec00640 1 -- 172.21.15.67:0/817665602 wait complete. 2024-02-26T10:43:03.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.697+0000 7fd2b0600640 1 Processor -- start 2024-02-26T10:43:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.697+0000 7fd2b0600640 1 -- start start 2024-02-26T10:43:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.699+0000 7fd2b0600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2a8072ea0 0x7fd2a8154e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.699+0000 7fd2b0600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2a8072a30 0x7fd2a81553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.699+0000 7fd2b0600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80c0a40 con 0x7fd2a806ae90 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.699+0000 7fd2b0600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a8073330 con 0x7fd2a8072a30 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.699+0000 7fd2b0600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a80c0d70 con 0x7fd2a8072ea0 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.699+0000 7fd2ae600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2a8072a30 0x7fd2a81553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ae600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd2a8072a30 0x7fd2a81553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50550/0 (socket says 172.21.15.67:50550) 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ae600640 1 -- 172.21.15.67:0/3996010293 learned_addr learned my addr 172.21.15.67:0/3996010293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2adc00640 1 --1- 172.21.15.67:0/3996010293 >> v1:172.21.15.67:6789/0 conn(0x7fd2a806ae90 0x7fd2a81521b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33258/0 (socket says 172.21.15.67:33258) 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ad200640 1 --2- 172.21.15.67:0/3996010293 >> v2:172.21.15.82:3300/0 conn(0x7fd2a8072ea0 0x7fd2a8154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ae600640 1 -- 172.21.15.67:0/3996010293 >> v2:172.21.15.82:3300/0 conn(0x7fd2a8072ea0 msgr2=0x7fd2a8154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ae600640 1 --2- 172.21.15.67:0/3996010293 >> v2:172.21.15.82:3300/0 conn(0x7fd2a8072ea0 0x7fd2a8154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ae600640 1 -- 172.21.15.67:0/3996010293 >> v1:172.21.15.67:6789/0 conn(0x7fd2a806ae90 legacy=0x7fd2a81521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2ae600640 1 -- 172.21.15.67:0/3996010293 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80c7010 con 0x7fd2a8072a30 2024-02-26T10:43:03.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.700+0000 7fd2a7e00640 1 -- 172.21.15.67:0/3996010293 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743684688 0 0) 0x7fd2a80c0a40 con 0x7fd2a806ae90 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.701+0000 7fd2ae600640 1 --2- 172.21.15.67:0/3996010293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8072a30 0x7fd2a81553b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80c86a0 tx=0x7fd2a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c69701eaaf0b29 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.701+0000 7fd2a7e00640 1 -- 172.21.15.67:0/3996010293 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0035020 con 0x7fd2a8072a30 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2a7e00640 1 -- 172.21.15.67:0/3996010293 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0031870 con 0x7fd2a8072a30 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2a7e00640 1 -- 172.21.15.67:0/3996010293 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0031c40 con 0x7fd2a8072a30 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 -- 172.21.15.67:0/3996010293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8072a30 msgr2=0x7fd2a81553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 --2- 172.21.15.67:0/3996010293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8072a30 0x7fd2a81553b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80c86a0 tx=0x7fd2a002b910 comp rx=0 tx=0).stop 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 -- 172.21.15.67:0/3996010293 shutdown_connections 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 --2- 172.21.15.67:0/3996010293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8072a30 0x7fd2a81553b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 --2- 172.21.15.67:0/3996010293 >> v2:172.21.15.82:3300/0 conn(0x7fd2a8072ea0 0x7fd2a8154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 -- 172.21.15.67:0/3996010293 >> 172.21.15.67:0/3996010293 conn(0x7fd2a80c7c40 msgr2=0x7fd2a806bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 -- 172.21.15.67:0/3996010293 shutdown_connections 2024-02-26T10:43:03.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.702+0000 7fd2b0600640 1 -- 172.21.15.67:0/3996010293 wait complete. 2024-02-26T10:43:03.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.704+0000 7fd2b0600640 1 Processor -- start 2024-02-26T10:43:03.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.704+0000 7fd2b0600640 1 -- start start 2024-02-26T10:43:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2b0600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2a806ae90 0x7fd2a8157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2b0600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8072a30 0x7fd2a8149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2b0600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 0x7fd2a81540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2b0600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a806b3f0 con 0x7fd2a8157f90 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2b0600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a806b270 con 0x7fd2a8072a30 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2b0600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81558f0 con 0x7fd2a806ae90 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2ae600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 0x7fd2a81540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2ae600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 0x7fd2a81540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50568/0 (socket says 172.21.15.67:50568) 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.705+0000 7fd2ae600640 1 -- 172.21.15.67:0/4063235669 learned_addr learned my addr 172.21.15.67:0/4063235669 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ad200640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8072a30 0x7fd2a8149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2adc00640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2a806ae90 0x7fd2a8157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ae600640 1 -- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2a806ae90 msgr2=0x7fd2a8157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ae600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2a806ae90 0x7fd2a8157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ae600640 1 -- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8072a30 msgr2=0x7fd2a8149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ae600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8072a30 0x7fd2a8149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ae600640 1 -- 172.21.15.67:0/4063235669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8159190 con 0x7fd2a8157f90 2024-02-26T10:43:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ad200640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8072a30 0x7fd2a8149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.706+0000 7fd2ae600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 0x7fd2a81540a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd2a002b950 tx=0x7fd2a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.707+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0040070 con 0x7fd2a8157f90 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.707+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0030c50 con 0x7fd2a8157f90 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.707+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0035040 con 0x7fd2a8157f90 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.707+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a81ee350 con 0x7fd2a8157f90 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.707+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a8151590 con 0x7fd2a8157f90 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.708+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8074a30 con 0x7fd2a8157f90 2024-02-26T10:43:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.709+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2a0038020 con 0x7fd2a8157f90 2024-02-26T10:43:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.710+0000 7fd2a5600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd28806b4d0 0x7fd28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.710+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fd2a00ac210 con 0x7fd2a8157f90 2024-02-26T10:43:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.711+0000 7fd2adc00640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd28806b4d0 0x7fd28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.712+0000 7fd2adc00640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd28806b4d0 0x7fd28806d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0012e0 tx=0x7fd29c007ef0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.714+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2a8159190 con 0x7fd2a8157f90 2024-02-26T10:43:03.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.914+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2a80c3930 con 0x7fd2a8157f90 2024-02-26T10:43:03.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.916+0000 7fd2a5600640 1 -- 172.21.15.67:0/4063235669 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16669 (secure 0 0 0) 0x7fd2a007af00 con 0x7fd2a8157f90 2024-02-26T10:43:03.914 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:03.914 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:43:01.068125+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6835/2183069114":"2024-02-27T10:42:58.049528+0000","172.21.15.67:6833/2183069114":"2024-02-27T10:42:58.049528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.918+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd28806b4d0 msgr2=0x7fd28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.918+0000 7fd2b0600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd28806b4d0 0x7fd28806d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0012e0 tx=0x7fd29c007ef0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.918+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 msgr2=0x7fd2a81540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.918+0000 7fd2b0600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 0x7fd2a81540a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd2a002b950 tx=0x7fd2a00046c0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 shutdown_connections 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a8157f90 0x7fd2a81540a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd28806b4d0 0x7fd28806d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8072a30 0x7fd2a8149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 --2- 172.21.15.67:0/4063235669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2a806ae90 0x7fd2a8157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 >> 172.21.15.67:0/4063235669 conn(0x7fd2a80c7c40 msgr2=0x7fd2a80c2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 shutdown_connections 2024-02-26T10:43:03.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:03.919+0000 7fd2b0600640 1 -- 172.21.15.67:0/4063235669 wait complete. 2024-02-26T10:43:03.934 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-02-26T10:43:03.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:43:04.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.152+0000 7f8b8a800640 1 Processor -- start 2024-02-26T10:43:04.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.152+0000 7f8b8a800640 1 -- start start 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.153+0000 7f8b8a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8b840c2310 0x7f8b84153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.153+0000 7f8b8a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b84155f10 con 0x7f8b8406ac20 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.153+0000 7f8b8a800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b840c28d0 con 0x7f8b8406b000 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.153+0000 7f8b8a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8406b430 con 0x7f8b840c2310 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b88a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8b840c2310 0x7f8b84153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b89400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8b8406ac20 0x7f8b84152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33264/0 (socket says 172.21.15.67:33264) 2024-02-26T10:43:04.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b89400640 1 -- 172.21.15.67:0/4248941291 learned_addr learned my addr 172.21.15.67:0/4248941291 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:04.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b88a00640 1 -- 172.21.15.67:0/4248941291 >> v1:172.21.15.82:6789/0 conn(0x7f8b8406b000 legacy=0x7f8b8415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:04.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b88a00640 1 -- 172.21.15.67:0/4248941291 >> v1:172.21.15.67:6789/0 conn(0x7f8b8406ac20 legacy=0x7f8b84152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b88a00640 1 -- 172.21.15.67:0/4248941291 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b84155150 con 0x7f8b840c2310 2024-02-26T10:43:04.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.154+0000 7f8b83e00640 1 -- 172.21.15.67:0/4248941291 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273653715 0 0) 0x7f8b84155f10 con 0x7f8b8406ac20 2024-02-26T10:43:04.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.155+0000 7f8b88a00640 1 --2- 172.21.15.67:0/4248941291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b840c2310 0x7f8b84153630 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8b740097a0 tx=0x7f8b7402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c872e0beba4d8fa server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b83e00640 1 -- 172.21.15.67:0/4248941291 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74038070 con 0x7f8b840c2310 2024-02-26T10:43:04.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b83e00640 1 -- 172.21.15.67:0/4248941291 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74031700 con 0x7f8b840c2310 2024-02-26T10:43:04.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b83e00640 1 -- 172.21.15.67:0/4248941291 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74031ad0 con 0x7f8b840c2310 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 -- 172.21.15.67:0/4248941291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b840c2310 msgr2=0x7f8b84153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4248941291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b840c2310 0x7f8b84153630 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8b740097a0 tx=0x7f8b7402b7a0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 -- 172.21.15.67:0/4248941291 shutdown_connections 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4248941291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b840c2310 0x7f8b84153630 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 -- 172.21.15.67:0/4248941291 >> 172.21.15.67:0/4248941291 conn(0x7f8b840c63f0 msgr2=0x7f8b840c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 -- 172.21.15.67:0/4248941291 shutdown_connections 2024-02-26T10:43:04.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.156+0000 7f8b8a800640 1 -- 172.21.15.67:0/4248941291 wait complete. 2024-02-26T10:43:04.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.158+0000 7f8b8a800640 1 Processor -- start 2024-02-26T10:43:04.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.158+0000 7f8b8a800640 1 -- start start 2024-02-26T10:43:04.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 0x7f8b840728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b8a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8406b000 0x7f8b841507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b8a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b840c2310 0x7f8b84141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b8a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b841559f0 con 0x7f8b8406ac20 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b8a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84155710 con 0x7f8b8406b000 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b8a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84148850 con 0x7f8b840c2310 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b89400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 0x7f8b840728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b89400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 0x7f8b840728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50582/0 (socket says 172.21.15.67:50582) 2024-02-26T10:43:04.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.159+0000 7f8b89400640 1 -- 172.21.15.67:0/4047143778 learned_addr learned my addr 172.21.15.67:0/4047143778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:04.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b88a00640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8406b000 0x7f8b841507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b89400640 1 -- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b840c2310 msgr2=0x7f8b84141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b89400640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b840c2310 0x7f8b84141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b89400640 1 -- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8406b000 msgr2=0x7f8b841507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b89400640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8406b000 0x7f8b841507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b89400640 1 -- 172.21.15.67:0/4047143778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840c19b0 con 0x7f8b8406ac20 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b88a00640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8406b000 0x7f8b841507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b89400640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 0x7f8b840728b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c460 tx=0x7f8b7800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78018070 con 0x7f8b8406ac20 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78007500 con 0x7f8b8406ac20 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78004a20 con 0x7f8b8406ac20 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.160+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b840725d0 con 0x7f8b8406ac20 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.161+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b84072df0 con 0x7f8b8406ac20 2024-02-26T10:43:04.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.162+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b84074a30 con 0x7f8b8406ac20 2024-02-26T10:43:04.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.163+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b78004bc0 con 0x7f8b8406ac20 2024-02-26T10:43:04.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.164+0000 7f8b81600640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b6806b500 0x7f8b6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.164+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f8b78089580 con 0x7f8b8406ac20 2024-02-26T10:43:04.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.165+0000 7f8b88a00640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b6806b500 0x7f8b6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.166+0000 7f8b88a00640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b6806b500 0x7f8b6806d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8b74035000 tx=0x7f8b74008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.168+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b78058390 con 0x7f8b8406ac20 2024-02-26T10:43:04.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.404+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b84142680 con 0x7f8b8406ac20 2024-02-26T10:43:04.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.406+0000 7f8b81600640 1 -- 172.21.15.67:0/4047143778 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8b78057c90 con 0x7f8b8406ac20 2024-02-26T10:43:04.403 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:04.404 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3395706465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3395706465},{"type":"v1","addr":"172.21.15.67:6835","nonce":3395706465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:43:04.404 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 55 2024-02-26T10:43:04.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b6806b500 msgr2=0x7f8b6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b6806b500 0x7f8b6806d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8b74035000 tx=0x7f8b74008030 comp rx=0 tx=0).stop 2024-02-26T10:43:04.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 msgr2=0x7f8b840728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 0x7f8b840728b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c460 tx=0x7f8b7800c920 comp rx=0 tx=0).stop 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 shutdown_connections 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b840c2310 0x7f8b84141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8b6806b500 0x7f8b6806d9c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8406b000 0x7f8b841507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 --2- 172.21.15.67:0/4047143778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b8406ac20 0x7f8b840728b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.408+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 >> 172.21.15.67:0/4047143778 conn(0x7f8b840c63f0 msgr2=0x7f8b8406e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.409+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 shutdown_connections 2024-02-26T10:43:04.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.409+0000 7f8b8a800640 1 -- 172.21.15.67:0/4047143778 wait complete. 2024-02-26T10:43:04.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:43:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.633+0000 7fbb1cc00640 1 Processor -- start 2024-02-26T10:43:04.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.633+0000 7fbb1cc00640 1 -- start start 2024-02-26T10:43:04.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb1cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 0x7fbb1806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb1cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb1806db00 0x7fbb180abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb1cc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1806df90 con 0x7fbb1806ac20 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb1cc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180a9a80 con 0x7fbb1806db00 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb1cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180a9d60 con 0x7fbb180a94a0 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb17400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbb1806ac20 0x7fbb18153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33266/0 (socket says 172.21.15.67:33266) 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb17400640 1 -- 172.21.15.67:0/1475017868 learned_addr learned my addr 172.21.15.67:0/1475017868 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.635+0000 7fbb17e00640 1 --2- 172.21.15.67:0/1475017868 >> v2:172.21.15.67:3300/0 conn(0x7fbb1806db00 0x7fbb180abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.636+0000 7fbb17e00640 1 -- 172.21.15.67:0/1475017868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 msgr2=0x7fbb1806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:04.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.636+0000 7fbb17e00640 1 --2- 172.21.15.67:0/1475017868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 0x7fbb1806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.636+0000 7fbb17e00640 1 -- 172.21.15.67:0/1475017868 >> v1:172.21.15.67:6789/0 conn(0x7fbb1806ac20 legacy=0x7fbb18153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.636+0000 7fbb16000640 1 -- 172.21.15.67:0/1475017868 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208535369 0 0) 0x7fbb1806df90 con 0x7fbb1806ac20 2024-02-26T10:43:04.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.636+0000 7fbb17e00640 1 -- 172.21.15.67:0/1475017868 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180ab750 con 0x7fbb1806db00 2024-02-26T10:43:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.637+0000 7fbb17e00640 1 --2- 172.21.15.67:0/1475017868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb180abf50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009d90 tx=0x7fbb1002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff60b3c94df7ecd6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.637+0000 7fbb16000640 1 -- 172.21.15.67:0/1475017868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10039070 con 0x7fbb1806db00 2024-02-26T10:43:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.637+0000 7fbb16000640 1 -- 172.21.15.67:0/1475017868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1002cc60 con 0x7fbb1806db00 2024-02-26T10:43:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.637+0000 7fbb16000640 1 -- 172.21.15.67:0/1475017868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10034660 con 0x7fbb1806db00 2024-02-26T10:43:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.637+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1475017868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 msgr2=0x7fbb180abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.637+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1475017868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb180abf50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009d90 tx=0x7fbb1002beb0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.638+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1475017868 shutdown_connections 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.638+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1475017868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb180abf50 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.638+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1475017868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 0x7fbb1806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.638+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1475017868 >> 172.21.15.67:0/1475017868 conn(0x7fbb180c63f0 msgr2=0x7fbb180c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.638+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1475017868 shutdown_connections 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.638+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1475017868 wait complete. 2024-02-26T10:43:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.639+0000 7fbb1cc00640 1 Processor -- start 2024-02-26T10:43:04.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.639+0000 7fbb1cc00640 1 -- start start 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.640+0000 7fbb1cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb1806ac20 0x7fbb181521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.640+0000 7fbb1cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb181ea160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.640+0000 7fbb1cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 0x7fbb181ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.640+0000 7fbb1cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18150410 con 0x7fbb1806db00 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.640+0000 7fbb1cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180ac490 con 0x7fbb1806ac20 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.640+0000 7fbb1cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18150590 con 0x7fbb180a94a0 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb181ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb181ea160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50604/0 (socket says 172.21.15.67:50604) 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 -- 172.21.15.67:0/1894854418 learned_addr learned my addr 172.21.15.67:0/1894854418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 -- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 msgr2=0x7fbb181ea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 0x7fbb181ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 -- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb1806ac20 msgr2=0x7fbb181521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb1806ac20 0x7fbb181521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 -- 172.21.15.67:0/1894854418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c55a0 con 0x7fbb1806db00 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb16a00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb181ea160 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400b410 tx=0x7fbb0400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04015020 con 0x7fbb1806db00 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb04011850 con 0x7fbb1806db00 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.641+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c5880 con 0x7fbb1806db00 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.642+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04011c40 con 0x7fbb1806db00 2024-02-26T10:43:04.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.642+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb181e9c40 con 0x7fbb1806db00 2024-02-26T10:43:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.643+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb180c0820 con 0x7fbb1806db00 2024-02-26T10:43:04.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.644+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb04018020 con 0x7fbb1806db00 2024-02-26T10:43:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.645+0000 7fbb0f400640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbaf806b4d0 0x7fbaf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:04.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.645+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fbb0408a290 con 0x7fbb1806db00 2024-02-26T10:43:04.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.646+0000 7fbb17400640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbaf806b4d0 0x7fbaf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:04.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.647+0000 7fbb17400640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbaf806b4d0 0x7fbaf806d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbb08007fd0 tx=0x7fbb08007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:04.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.649+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb0401f020 con 0x7fbb1806db00 2024-02-26T10:43:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.876+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb1806b9b0 con 0x7fbb1806db00 2024-02-26T10:43:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.877+0000 7fbb0f400640 1 -- 172.21.15.67:0/1894854418 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbb040172c0 con 0x7fbb1806db00 2024-02-26T10:43:04.875 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:04.875 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3395706465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3395706465},{"type":"v1","addr":"172.21.15.67:6835","nonce":3395706465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-26T10:42:20.013444+0000","modified":"2024-02-26T10:42:23.108546+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6447":{"gid":6447,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":10,"addr":"172.21.15.67:6834/2406048118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2406048118},{"type":"v1","addr":"172.21.15.67:6834","nonce":2406048118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-02-26T10:43:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 55 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbaf806b4d0 msgr2=0x7fbaf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbaf806b4d0 0x7fbaf806d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbb08007fd0 tx=0x7fbb08007920 comp rx=0 tx=0).stop 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 msgr2=0x7fbb181ea160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb181ea160 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400b410 tx=0x7fbb0400b8d0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 shutdown_connections 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb180a94a0 0x7fbb181ea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb1806db00 0x7fbb181ea160 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fbaf806b4d0 0x7fbaf806d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 --2- 172.21.15.67:0/1894854418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb1806ac20 0x7fbb181521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 >> 172.21.15.67:0/1894854418 conn(0x7fbb180c63f0 msgr2=0x7fbb180c1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 shutdown_connections 2024-02-26T10:43:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:04.880+0000 7fbb1cc00640 1 -- 172.21.15.67:0/1894854418 wait complete. 2024-02-26T10:43:04.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:05.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.109+0000 7f5108800640 1 Processor -- start 2024-02-26T10:43:05.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.109+0000 7f5108800640 1 -- start start 2024-02-26T10:43:05.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5108800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5100157d30 0x7f5100158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5108800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5100155400 0x7f5100155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5108800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5100154f50 con 0x7f51000bdef0 2024-02-26T10:43:05.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5108800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51000c7440 con 0x7f5100155400 2024-02-26T10:43:05.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5108800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51000c7720 con 0x7f5100157d30 2024-02-26T10:43:05.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5105400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f51000bdef0 0x7f51000c51a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33270/0 (socket says 172.21.15.67:33270) 2024-02-26T10:43:05.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5105400640 1 -- 172.21.15.67:0/1441880084 learned_addr learned my addr 172.21.15.67:0/1441880084 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:05.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.111+0000 7f5106800640 1 --2- 172.21.15.67:0/1441880084 >> v2:172.21.15.82:3300/0 conn(0x7f5100155400 0x7f5100155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5105e00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5100157d30 0x7f5100158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183799246 0 0) 0x7f5100154f50 con 0x7f51000bdef0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e8003520 con 0x7f51000bdef0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287518150 0 0) 0x7f50e8003520 con 0x7f51000bdef0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5100157d30 msgr2=0x7f5100158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5100157d30 0x7f5100158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 >> v2:172.21.15.82:3300/0 conn(0x7f5100155400 msgr2=0x7f5100155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 --2- 172.21.15.67:0/1441880084 >> v2:172.21.15.82:3300/0 conn(0x7f5100155400 0x7f5100155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000ad2a0 con 0x7f51000bdef0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f50f0001670 con 0x7f51000bdef0 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 >> v1:172.21.15.67:6789/0 conn(0x7f51000bdef0 legacy=0x7f51000c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000c7440 con 0x7f50e8004800 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000c7720 con 0x7f50e80039b0 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e800c030 con 0x7f50e80051c0 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.112+0000 7f5105e00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5100157d30 0x7f5100158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105400640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5106800640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105e00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105400640 1 -- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50e80051c0 msgr2=0x7f50e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105400640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f50e80051c0 0x7f50e800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105400640 1 -- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e80039b0 msgr2=0x7f50e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105400640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105400640 1 -- 172.21.15.67:0/1441880084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8003520 con 0x7f50e8004800 2024-02-26T10:43:05.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.113+0000 7f5105e00640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.114+0000 7f5105400640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f50f000c160 tx=0x7f50e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5abcd2b6dba03d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.114+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0007c80 con 0x7f50e8004800 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.114+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0030070 con 0x7f50e8004800 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.114+0000 7f5104a00640 1 -- 172.21.15.67:0/1441880084 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0005d30 con 0x7f50e8004800 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5108800640 1 -- 172.21.15.67:0/1441880084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 msgr2=0x7f50e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5108800640 1 --2- 172.21.15.67:0/1441880084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f50f000c160 tx=0x7f50e8005830 comp rx=0 tx=0).stop 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5106800640 1 -- 172.21.15.67:0/1441880084 reap_dead start 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5108800640 1 -- 172.21.15.67:0/1441880084 shutdown_connections 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5108800640 1 -- 172.21.15.67:0/1441880084 >> 172.21.15.67:0/1441880084 conn(0x7f5100072980 msgr2=0x7f51000ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5108800640 1 -- 172.21.15.67:0/1441880084 shutdown_connections 2024-02-26T10:43:05.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.115+0000 7f5108800640 1 -- 172.21.15.67:0/1441880084 wait complete. 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.117+0000 7f5108800640 1 Processor -- start 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.117+0000 7f5108800640 1 -- start start 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5108800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f510014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5108800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f510014c340 0x7f51001516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5108800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5100151c10 0x7f51000aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5108800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000c7a00 con 0x7f50e8004800 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5108800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5100158110 con 0x7f5100151c10 2024-02-26T10:43:05.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5108800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5100156330 con 0x7f510014c340 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5105e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f510014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5105e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f510014be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50626/0 (socket says 172.21.15.67:50626) 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5105e00640 1 -- 172.21.15.67:0/3611672913 learned_addr learned my addr 172.21.15.67:0/3611672913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5105e00640 1 -- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f510014c340 msgr2=0x7f51001516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.118+0000 7f5105400640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f510014c340 0x7f51001516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5105e00640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f510014c340 0x7f51001516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5106800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5100151c10 0x7f51000aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5105e00640 1 -- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5100151c10 msgr2=0x7f51000aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5105e00640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5100151c10 0x7f51000aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5105e00640 1 -- 172.21.15.67:0/3611672913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51001508d0 con 0x7f50e8004800 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5106800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5100151c10 0x7f51000aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:05.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5105e00640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f510014be00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x7f50e8004800 2024-02-26T10:43:05.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f4013610 con 0x7f50e8004800 2024-02-26T10:43:05.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f400f040 con 0x7f50e8004800 2024-02-26T10:43:05.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51000cb0a0 con 0x7f50e8004800 2024-02-26T10:43:05.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.119+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51001ebf10 con 0x7f50e8004800 2024-02-26T10:43:05.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.121+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5100152bd0 con 0x7f50e8004800 2024-02-26T10:43:05.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.122+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50f401a070 con 0x7f50e8004800 2024-02-26T10:43:05.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.122+0000 7f50fa000640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f50e406b4d0 0x7f50e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.123+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f50f4089be0 con 0x7f50e8004800 2024-02-26T10:43:05.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.123+0000 7f5105400640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f50e406b4d0 0x7f50e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.124+0000 7f5105400640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f50e406b4d0 0x7f50e406d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f50f0034000 tx=0x7f50f0007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.127+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5100152bd0 con 0x7f50e8004800 2024-02-26T10:43:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.328+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5100074a30 con 0x7f50e8004800 2024-02-26T10:43:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.331+0000 7f50fa000640 1 -- 172.21.15.67:0/3611672913 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f50f40589f0 con 0x7f50e8004800 2024-02-26T10:43:05.329 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:05.329 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:43:01.068125+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-02-26T10:42:17.695706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-02-26T10:42:19.039564+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6835/2183069114":"2024-02-27T10:42:58.049528+0000","172.21.15.67:6833/2183069114":"2024-02-27T10:42:58.049528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.333+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f50e406b4d0 msgr2=0x7f50e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.333+0000 7f5108800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f50e406b4d0 0x7f50e406d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f50f0034000 tx=0x7f50f0007000 comp rx=0 tx=0).stop 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 msgr2=0x7f510014be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f510014be00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f400e970 comp rx=0 tx=0).stop 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 shutdown_connections 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f50e406b4d0 0x7f50e406d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5100151c10 0x7f51000aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f510014c340 0x7f51001516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 --2- 172.21.15.67:0/3611672913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f510014be00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 >> 172.21.15.67:0/3611672913 conn(0x7f5100072980 msgr2=0x7f51000741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 shutdown_connections 2024-02-26T10:43:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.334+0000 7f5108800640 1 -- 172.21.15.67:0/3611672913 wait complete. 2024-02-26T10:43:05.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-02-26T10:43:05.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.583+0000 7fb5efc00640 1 Processor -- start 2024-02-26T10:43:05.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.583+0000 7fb5efc00640 1 -- start start 2024-02-26T10:43:05.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.584+0000 7fb5efc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5e8156810 0x7fb5e806e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.584+0000 7fb5efc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5e8072fe0 0x7fb5e80c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.584+0000 7fb5efc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8159210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.584+0000 7fb5efc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e815c8a0 con 0x7fb5e8156810 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.584+0000 7fb5efc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e815cb80 con 0x7fb5e8072fe0 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.584+0000 7fb5efc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e815ce60 con 0x7fb5e80c35d0 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5e8156810 0x7fb5e806e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5e8156810 0x7fb5e806e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50642/0 (socket says 172.21.15.67:50642) 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 -- 172.21.15.67:0/3950952 learned_addr learned my addr 172.21.15.67:0/3950952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 -- 172.21.15.67:0/3950952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 msgr2=0x7fb5e8159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 --2- 172.21.15.67:0/3950952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 -- 172.21.15.67:0/3950952 >> v2:172.21.15.82:3300/0 conn(0x7fb5e8072fe0 msgr2=0x7fb5e80c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 --2- 172.21.15.67:0/3950952 >> v2:172.21.15.82:3300/0 conn(0x7fb5e8072fe0 0x7fb5e80c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.585+0000 7fb5ed200640 1 -- 172.21.15.67:0/3950952 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e815d140 con 0x7fb5e8156810 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.586+0000 7fb5ed200640 1 --2- 172.21.15.67:0/3950952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8156810 0x7fb5e806e660 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb5e806bcc0 tx=0x7fb5dc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9afe271adba969d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5e7400640 1 -- 172.21.15.67:0/3950952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0317d0 con 0x7fb5e8156810 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5e7400640 1 -- 172.21.15.67:0/3950952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc037070 con 0x7fb5e8156810 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5e7400640 1 -- 172.21.15.67:0/3950952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc031b90 con 0x7fb5e8156810 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 -- 172.21.15.67:0/3950952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8156810 msgr2=0x7fb5e806e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3950952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8156810 0x7fb5e806e660 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb5e806bcc0 tx=0x7fb5dc02b870 comp rx=0 tx=0).stop 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 -- 172.21.15.67:0/3950952 shutdown_connections 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3950952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3950952 >> v2:172.21.15.82:3300/0 conn(0x7fb5e8072fe0 0x7fb5e80c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3950952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8156810 0x7fb5e806e660 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 -- 172.21.15.67:0/3950952 >> 172.21.15.67:0/3950952 conn(0x7fb5e80a84a0 msgr2=0x7fb5e80a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 -- 172.21.15.67:0/3950952 shutdown_connections 2024-02-26T10:43:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.587+0000 7fb5efc00640 1 -- 172.21.15.67:0/3950952 wait complete. 2024-02-26T10:43:05.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.589+0000 7fb5efc00640 1 Processor -- start 2024-02-26T10:43:05.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.589+0000 7fb5efc00640 1 -- start start 2024-02-26T10:43:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5efc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 0x7fb5e814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5efc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5efc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e8156810 0x7fb5e81eef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5efc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e806ec40 con 0x7fb5e8072fe0 2024-02-26T10:43:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5efc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e815d660 con 0x7fb5e8156810 2024-02-26T10:43:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5efc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80c0ba0 con 0x7fb5e80c35d0 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5ed200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 0x7fb5e814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5ed200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 0x7fb5e814c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50648/0 (socket says 172.21.15.67:50648) 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.590+0000 7fb5ed200640 1 -- 172.21.15.67:0/3967962387 learned_addr learned my addr 172.21.15.67:0/3967962387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5edc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e8156810 0x7fb5e81eef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5e7e00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5ed200640 1 -- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 msgr2=0x7fb5e8158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5ed200640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5ed200640 1 -- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e8156810 msgr2=0x7fb5e81eef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5ed200640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e8156810 0x7fb5e81eef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5ed200640 1 -- 172.21.15.67:0/3967962387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80c5610 con 0x7fb5e8072fe0 2024-02-26T10:43:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5e7e00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.591+0000 7fb5ed200640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 0x7fb5e814c9b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.592+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc032870 con 0x7fb5e8072fe0 2024-02-26T10:43:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.592+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc032a10 con 0x7fb5e8072fe0 2024-02-26T10:43:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.592+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc032e00 con 0x7fb5e8072fe0 2024-02-26T10:43:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.592+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e80c58f0 con 0x7fb5e8072fe0 2024-02-26T10:43:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.592+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e80c5eb0 con 0x7fb5e8072fe0 2024-02-26T10:43:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.593+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e80c61e0 con 0x7fb5e8072fe0 2024-02-26T10:43:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.594+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5dc04c060 con 0x7fb5e8072fe0 2024-02-26T10:43:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.595+0000 7fb5e4c00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb5c806b4d0 0x7fb5c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.595+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fb5dc0abc00 con 0x7fb5e8072fe0 2024-02-26T10:43:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.596+0000 7fb5e7e00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb5c806b4d0 0x7fb5c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.597+0000 7fb5e7e00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb5c806b4d0 0x7fb5c806d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80c2250 tx=0x7fb5d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.600+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5dc040070 con 0x7fb5e8072fe0 2024-02-26T10:43:05.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.837+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7fb5e8074a30 con 0x7fb5e8072fe0 2024-02-26T10:43:05.836 INFO:tasks.ceph.mon.a.smithi067.stderr:2024-02-26T10:43:05.839+0000 7ff2e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-26T10:43:05.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.842+0000 7fb5e4c00640 1 -- 172.21.15.67:0/3967962387 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 163+0+0 (secure 0 0 0) 0x7fb5dc07aa10 con 0x7fb5e8072fe0 2024-02-26T10:43:05.840 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb5c806b4d0 msgr2=0x7fb5c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb5c806b4d0 0x7fb5c806d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80c2250 tx=0x7fb5d4008040 comp rx=0 tx=0).stop 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 msgr2=0x7fb5e814c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 0x7fb5e814c9b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc0046c0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 shutdown_connections 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb5c806b4d0 0x7fb5c806d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e8156810 0x7fb5e81eef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5e80c35d0 0x7fb5e8158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 --2- 172.21.15.67:0/3967962387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5e8072fe0 0x7fb5e814c9b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 >> 172.21.15.67:0/3967962387 conn(0x7fb5e80a84a0 msgr2=0x7fb5e80c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 shutdown_connections 2024-02-26T10:43:05.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:05.846+0000 7fb5efc00640 1 -- 172.21.15.67:0/3967962387 wait complete. 2024-02-26T10:43:05.858 INFO:tasks.ceph.mds.c.smithi067.stderr: -5145> 2024-02-26T10:42:22.159+0000 7f1462a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-26T10:43:05.873 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-02-26T10:43:05.912 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-02-26T10:43:06.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.061+0000 7f829b000640 1 Processor -- start 2024-02-26T10:43:06.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.061+0000 7f829b000640 1 -- start start 2024-02-26T10:43:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.062+0000 7f829b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82941525b0 0x7f8294155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f829b000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f82940c5330 0x7f82940c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f829b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f829406bb80 0x7f82940c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f829b000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82941552c0 con 0x7f82940c5330 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f829b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829406c850 con 0x7f829406bb80 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f829b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8294155c90 con 0x7f82941525b0 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f8299200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f82940c5330 0x7f82940c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f8299200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f82940c5330 0x7f82940c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50680/0 (socket says 172.21.15.67:50680) 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f8299200640 1 -- 172.21.15.67:0/1631140094 learned_addr learned my addr 172.21.15.67:0/1631140094 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f829a600640 1 --2- 172.21.15.67:0/1631140094 >> v2:172.21.15.82:3300/0 conn(0x7f829406bb80 0x7f82940c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.063+0000 7f8299c00640 1 --2- 172.21.15.67:0/1631140094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82941525b0 0x7f8294155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.064+0000 7f8299200640 1 -- 172.21.15.67:0/1631140094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82941525b0 msgr2=0x7f8294155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.064+0000 7f8299200640 1 --2- 172.21.15.67:0/1631140094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82941525b0 0x7f8294155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.064+0000 7f8299200640 1 -- 172.21.15.67:0/1631140094 >> v2:172.21.15.82:3300/0 conn(0x7f829406bb80 msgr2=0x7f82940c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.064+0000 7f8299200640 1 --2- 172.21.15.67:0/1631140094 >> v2:172.21.15.82:3300/0 conn(0x7f829406bb80 0x7f82940c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.064+0000 7f8299200640 1 -- 172.21.15.67:0/1631140094 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940c8a60 con 0x7f82940c5330 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.065+0000 7f8299200640 1 --2- 172.21.15.67:0/1631140094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82940c5330 0x7f82940c8520 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f82940c50c0 tx=0x7f828002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59689168daa425a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.065+0000 7f8293e00640 1 -- 172.21.15.67:0/1631140094 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280038070 con 0x7f82940c5330 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.065+0000 7f8293e00640 1 -- 172.21.15.67:0/1631140094 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828002f050 con 0x7f82940c5330 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.065+0000 7f8293e00640 1 -- 172.21.15.67:0/1631140094 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280031970 con 0x7f82940c5330 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 -- 172.21.15.67:0/1631140094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82940c5330 msgr2=0x7f82940c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 --2- 172.21.15.67:0/1631140094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82940c5330 0x7f82940c8520 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f82940c50c0 tx=0x7f828002b7f0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 -- 172.21.15.67:0/1631140094 shutdown_connections 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 --2- 172.21.15.67:0/1631140094 >> v2:172.21.15.82:3300/0 conn(0x7f829406bb80 0x7f82940c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 --2- 172.21.15.67:0/1631140094 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f82940c5330 0x7f82940c8520 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 --2- 172.21.15.67:0/1631140094 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82941525b0 0x7f8294155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 -- 172.21.15.67:0/1631140094 >> 172.21.15.67:0/1631140094 conn(0x7f82940c70a0 msgr2=0x7f82940c74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 -- 172.21.15.67:0/1631140094 shutdown_connections 2024-02-26T10:43:06.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.066+0000 7f829b000640 1 -- 172.21.15.67:0/1631140094 wait complete. 2024-02-26T10:43:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.068+0000 7f829b000640 1 Processor -- start 2024-02-26T10:43:06.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.068+0000 7f829b000640 1 -- start start 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f829406bb80 0x7f8294149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82940c5330 0x7f8294158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 0x7f829414e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8294154f50 con 0x7f829414c850 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829406c570 con 0x7f829406bb80 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829406b670 con 0x7f82940c5330 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 0x7f829414e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:06.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 0x7f829414e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50686/0 (socket says 172.21.15.67:50686) 2024-02-26T10:43:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 -- 172.21.15.67:0/826638713 learned_addr learned my addr 172.21.15.67:0/826638713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 -- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82940c5330 msgr2=0x7f8294158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f8299200640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82940c5330 0x7f8294158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82940c5330 0x7f8294158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 -- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f829406bb80 msgr2=0x7f8294149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f829406bb80 0x7f8294149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.069+0000 7f829a600640 1 -- 172.21.15.67:0/826638713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829414ded0 con 0x7f829414c850 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.070+0000 7f829a600640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 0x7f829414e9a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f828c007d20 tx=0x7f828c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.070+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c014420 con 0x7f829414c850 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.070+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c019070 con 0x7f829414c850 2024-02-26T10:43:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.070+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829414e1b0 con 0x7f829414c850 2024-02-26T10:43:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.070+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c014820 con 0x7f829414c850 2024-02-26T10:43:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.070+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829414eee0 con 0x7f829414c850 2024-02-26T10:43:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.071+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829414e6d0 con 0x7f829414c850 2024-02-26T10:43:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.072+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f828c0149c0 con 0x7f829414c850 2024-02-26T10:43:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.073+0000 7f8291600640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f827406b500 0x7f827406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.073+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5885+0+0 (secure 0 0 0) 0x7f828c08aa20 con 0x7f829414c850 2024-02-26T10:43:06.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.074+0000 7f8299c00640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f827406b500 0x7f827406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:06.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.075+0000 7f8299c00640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f827406b500 0x7f827406d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f82880098f0 tx=0x7f8288008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:06.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.077+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f828c0597e0 con 0x7f829414c850 2024-02-26T10:43:06.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.320+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8294074a30 con 0x7f829414c850 2024-02-26T10:43:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.846+0000 7f8291600640 1 -- 172.21.15.67:0/826638713 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 105+0+0 (secure 0 0 0) 0x7f828c0590e0 con 0x7f829414c850 2024-02-26T10:43:06.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.849+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f827406b500 msgr2=0x7f827406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.849+0000 7f829b000640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f827406b500 0x7f827406d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f82880098f0 tx=0x7f8288008040 comp rx=0 tx=0).stop 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.849+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 msgr2=0x7f829414e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.849+0000 7f829b000640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 0x7f829414e9a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f828c007d20 tx=0x7f828c00e4b0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 shutdown_connections 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f829414c850 0x7f829414e9a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82940c5330 0x7f8294158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f827406b500 0x7f827406d9c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 --2- 172.21.15.67:0/826638713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f829406bb80 0x7f8294149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 >> 172.21.15.67:0/826638713 conn(0x7f82940c70a0 msgr2=0x7f82940c5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 shutdown_connections 2024-02-26T10:43:06.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:06.850+0000 7f829b000640 1 -- 172.21.15.67:0/826638713 wait complete. 2024-02-26T10:43:06.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-02-26T10:43:07.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.145+0000 7fa347e00640 1 Processor -- start 2024-02-26T10:43:07.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.145+0000 7fa347e00640 1 -- start start 2024-02-26T10:43:07.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.146+0000 7fa347e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa348152380 0x7fa3480c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3480c9b30 0x7fa34815f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa348152760 con 0x7fa3480c54d0 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3480cb4e0 con 0x7fa3480c9b30 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34806b900 con 0x7fa348152380 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3480c9b30 0x7fa34815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa346000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa3480c54d0 0x7fa3480c9420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33272/0 (socket says 172.21.15.67:33272) 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa3480c9b30 0x7fa34815f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50692/0 (socket says 172.21.15.67:50692) 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa346000640 1 -- 172.21.15.67:0/296896023 learned_addr learned my addr 172.21.15.67:0/296896023 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa346a00640 1 --2- 172.21.15.67:0/296896023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa348152380 0x7fa3480c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347400640 1 -- 172.21.15.67:0/296896023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa348152380 msgr2=0x7fa3480c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347400640 1 --2- 172.21.15.67:0/296896023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa348152380 0x7fa3480c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:07.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347400640 1 -- 172.21.15.67:0/296896023 >> v1:172.21.15.67:6789/0 conn(0x7fa3480c54d0 legacy=0x7fa3480c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:07.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.147+0000 7fa347400640 1 -- 172.21.15.67:0/296896023 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348159010 con 0x7fa3480c9b30 2024-02-26T10:43:07.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.149+0000 7fa347400640 1 --2- 172.21.15.67:0/296896023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3480c9b30 0x7fa34815f640 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa340009d90 tx=0x7fa34002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a09a6c3b7439c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:07.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.149+0000 7fa345600640 1 -- 172.21.15.67:0/296896023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340039070 con 0x7fa3480c9b30 2024-02-26T10:43:07.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.149+0000 7fa345600640 1 -- 172.21.15.67:0/296896023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34002cb40 con 0x7fa3480c9b30 2024-02-26T10:43:07.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.149+0000 7fa345600640 1 -- 172.21.15.67:0/296896023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340034440 con 0x7fa3480c9b30 2024-02-26T10:43:07.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.149+0000 7fa347e00640 1 -- 172.21.15.67:0/296896023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3480c9b30 msgr2=0x7fa34815f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:07.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.149+0000 7fa347e00640 1 --2- 172.21.15.67:0/296896023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3480c9b30 0x7fa34815f640 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa340009d90 tx=0x7fa34002bd90 comp rx=0 tx=0).stop 2024-02-26T10:43:07.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.150+0000 7fa347e00640 1 -- 172.21.15.67:0/296896023 shutdown_connections 2024-02-26T10:43:07.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.150+0000 7fa347e00640 1 --2- 172.21.15.67:0/296896023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3480c9b30 0x7fa34815f640 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:07.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.150+0000 7fa347e00640 1 --2- 172.21.15.67:0/296896023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa348152380 0x7fa3480c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:07.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.150+0000 7fa347e00640 1 -- 172.21.15.67:0/296896023 >> 172.21.15.67:0/296896023 conn(0x7fa3480ca610 msgr2=0x7fa3480caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:07.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.150+0000 7fa347e00640 1 -- 172.21.15.67:0/296896023 shutdown_connections 2024-02-26T10:43:07.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.150+0000 7fa347e00640 1 -- 172.21.15.67:0/296896023 wait complete. 2024-02-26T10:43:07.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.151+0000 7fa347e00640 1 Processor -- start 2024-02-26T10:43:07.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 -- start start 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3480c54d0 0x7fa348155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3480c9b30 0x7fa34814b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 0x7fa34814b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34806dca0 con 0x7fa348152380 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480c8bf0 con 0x7fa3480c9b30 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.152+0000 7fa347e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa348158c70 con 0x7fa3480c54d0 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 0x7fa34814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 0x7fa34814b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50694/0 (socket says 172.21.15.67:50694) 2024-02-26T10:43:07.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 -- 172.21.15.67:0/1205352033 learned_addr learned my addr 172.21.15.67:0/1205352033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa346a00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3480c54d0 0x7fa348155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 -- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3480c54d0 msgr2=0x7fa348155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa346000640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3480c9b30 0x7fa34814b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3480c54d0 0x7fa348155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 -- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3480c9b30 msgr2=0x7fa34814b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3480c9b30 0x7fa34814b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.153+0000 7fa347400640 1 -- 172.21.15.67:0/1205352033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3481eb210 con 0x7fa348152380 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.154+0000 7fa347400640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 0x7fa34814b550 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa340009d90 tx=0x7fa340004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.154+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340039050 con 0x7fa348152380 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.154+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa340045070 con 0x7fa348152380 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.154+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3481eb4f0 con 0x7fa348152380 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.154+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34003e830 con 0x7fa348152380 2024-02-26T10:43:07.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.154+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa348155860 con 0x7fa348152380 2024-02-26T10:43:07.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.155+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa348074a30 con 0x7fa348152380 2024-02-26T10:43:07.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.156+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa34003e9d0 con 0x7fa348152380 2024-02-26T10:43:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.157+0000 7fa33ea00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32406b4d0 0x7fa32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.157+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5885+0+0 (secure 0 0 0) 0x7fa3400acfe0 con 0x7fa348152380 2024-02-26T10:43:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.158+0000 7fa346a00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32406b4d0 0x7fa32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.159+0000 7fa346a00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32406b4d0 0x7fa32406d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa338006fd0 tx=0x7fa338008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:07.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.162+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa340032e20 con 0x7fa348152380 2024-02-26T10:43:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:07.363+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa3480c8510 con 0x7fa348152380 2024-02-26T10:43:08.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.124+0000 7fa33ea00640 1 -- 172.21.15.67:0/1205352033 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v85) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa34007bda0 con 0x7fa348152380 2024-02-26T10:43:08.121 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_metadata' removed 2024-02-26T10:43:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.127+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32406b4d0 msgr2=0x7fa32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.127+0000 7fa347e00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32406b4d0 0x7fa32406d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa338006fd0 tx=0x7fa338008040 comp rx=0 tx=0).stop 2024-02-26T10:43:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.127+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 msgr2=0x7fa34814b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.127+0000 7fa347e00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 0x7fa34814b550 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa340009d90 tx=0x7fa340004a40 comp rx=0 tx=0).stop 2024-02-26T10:43:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 shutdown_connections 2024-02-26T10:43:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa348152380 0x7fa34814b550 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fa32406b4d0 0x7fa32406d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3480c9b30 0x7fa34814b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 --2- 172.21.15.67:0/1205352033 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3480c54d0 0x7fa348155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 >> 172.21.15.67:0/1205352033 conn(0x7fa3480ca610 msgr2=0x7fa3480aa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.128+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 shutdown_connections 2024-02-26T10:43:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.129+0000 7fa347e00640 1 -- 172.21.15.67:0/1205352033 wait complete. 2024-02-26T10:43:08.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-02-26T10:43:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.360+0000 7f6825a00640 1 Processor -- start 2024-02-26T10:43:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.360+0000 7f6825a00640 1 -- start start 2024-02-26T10:43:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6825a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f68200adbe0 0x7f68200c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6825a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68201568b0 0x7f68200c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6825a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6820070e70 con 0x7f68200ae310 2024-02-26T10:43:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6825a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6820150650 con 0x7f68201568b0 2024-02-26T10:43:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6825a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6820070350 con 0x7f68200adbe0 2024-02-26T10:43:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6817e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f68200ae310 0x7f68201561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33288/0 (socket says 172.21.15.67:33288) 2024-02-26T10:43:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6817e00640 1 -- 172.21.15.67:0/2002492908 learned_addr learned my addr 172.21.15.67:0/2002492908 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f6825000640 1 --2- 172.21.15.67:0/2002492908 >> v2:172.21.15.67:3300/0 conn(0x7f68201568b0 0x7f68200c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.362+0000 7f681fe00640 1 --2- 172.21.15.67:0/2002492908 >> v2:172.21.15.82:3300/0 conn(0x7f68200adbe0 0x7f68200c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.363+0000 7f681f400640 1 -- 172.21.15.67:0/2002492908 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064656626 0 0) 0x7f6820070e70 con 0x7f68200ae310 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.363+0000 7f681f400640 1 -- 172.21.15.67:0/2002492908 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6804003520 con 0x7f68200ae310 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.363+0000 7f6825000640 1 -- 172.21.15.67:0/2002492908 >> v2:172.21.15.82:3300/0 conn(0x7f68200adbe0 msgr2=0x7f68200c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.363+0000 7f6825000640 1 --2- 172.21.15.67:0/2002492908 >> v2:172.21.15.82:3300/0 conn(0x7f68200adbe0 0x7f68200c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.363+0000 7f6825000640 1 -- 172.21.15.67:0/2002492908 >> v1:172.21.15.67:6789/0 conn(0x7f68200ae310 legacy=0x7f68201561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:08.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.363+0000 7f6825000640 1 -- 172.21.15.67:0/2002492908 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68201536d0 con 0x7f68201568b0 2024-02-26T10:43:08.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.364+0000 7f6825000640 1 --2- 172.21.15.67:0/2002492908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68201568b0 0x7f68200c69c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f68200c91c0 tx=0x7f681802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f70a193993f8dd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:08.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f681f400640 1 -- 172.21.15.67:0/2002492908 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818039070 con 0x7f68201568b0 2024-02-26T10:43:08.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f681f400640 1 -- 172.21.15.67:0/2002492908 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681802cbf0 con 0x7f68201568b0 2024-02-26T10:43:08.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f681f400640 1 -- 172.21.15.67:0/2002492908 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818034440 con 0x7f68201568b0 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f6825a00640 1 -- 172.21.15.67:0/2002492908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68201568b0 msgr2=0x7f68200c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f6825a00640 1 --2- 172.21.15.67:0/2002492908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68201568b0 0x7f68200c69c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f68200c91c0 tx=0x7f681802be40 comp rx=0 tx=0).stop 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f6825a00640 1 -- 172.21.15.67:0/2002492908 shutdown_connections 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f6825a00640 1 --2- 172.21.15.67:0/2002492908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68201568b0 0x7f68200c69c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f6825a00640 1 --2- 172.21.15.67:0/2002492908 >> v2:172.21.15.82:3300/0 conn(0x7f68200adbe0 0x7f68200c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.365+0000 7f6825a00640 1 -- 172.21.15.67:0/2002492908 >> 172.21.15.67:0/2002492908 conn(0x7f682006ac20 msgr2=0x7f682006b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.366+0000 7f6825a00640 1 -- 172.21.15.67:0/2002492908 shutdown_connections 2024-02-26T10:43:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.366+0000 7f6825a00640 1 -- 172.21.15.67:0/2002492908 wait complete. 2024-02-26T10:43:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.367+0000 7f6825a00640 1 Processor -- start 2024-02-26T10:43:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 -- start start 2024-02-26T10:43:08.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 0x7f68200a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:08.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68200ae310 0x7f682014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:08.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68201568b0 0x7f682014d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:08.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820156e00 con 0x7f68200adbe0 2024-02-26T10:43:08.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820070970 con 0x7f68200ae310 2024-02-26T10:43:08.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.368+0000 7f6825a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200adfc0 con 0x7f68201568b0 2024-02-26T10:43:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 0x7f68200a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 0x7f68200a82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50714/0 (socket says 172.21.15.67:50714) 2024-02-26T10:43:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 -- 172.21.15.67:0/3073904642 learned_addr learned my addr 172.21.15.67:0/3073904642 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f6825000640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68201568b0 0x7f682014d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f6817e00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68200ae310 0x7f682014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 -- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68201568b0 msgr2=0x7f682014d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68201568b0 0x7f682014d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 -- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68200ae310 msgr2=0x7f682014d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68200ae310 0x7f682014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.369+0000 7f681fe00640 1 -- 172.21.15.67:0/3073904642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68201eaa20 con 0x7f68200adbe0 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.370+0000 7f681fe00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 0x7f68200a82e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f681000d800 tx=0x7f681000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.370+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000b6c0 con 0x7f68200adbe0 2024-02-26T10:43:08.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.370+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810019070 con 0x7f68200adbe0 2024-02-26T10:43:08.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.370+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000ba80 con 0x7f68200adbe0 2024-02-26T10:43:08.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.370+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68201ead00 con 0x7f68200adbe0 2024-02-26T10:43:08.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.370+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68201eb2c0 con 0x7f68200adbe0 2024-02-26T10:43:08.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.371+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6820074a30 con 0x7f68200adbe0 2024-02-26T10:43:08.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.372+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f681000bc20 con 0x7f68200adbe0 2024-02-26T10:43:08.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.373+0000 7f681cc00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b4d0 0x7f67fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:08.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.373+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5445+0+0 (secure 0 0 0) 0x7f681008a5c0 con 0x7f68200adbe0 2024-02-26T10:43:08.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.374+0000 7f6817e00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b4d0 0x7f67fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:08.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.375+0000 7f6817e00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b4d0 0x7f67fc06d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6820155990 tx=0x7f6808009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:08.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.378+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6810059530 con 0x7f68200adbe0 2024-02-26T10:43:08.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:08.588+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68200705b0 con 0x7f68200adbe0 2024-02-26T10:43:09.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.143+0000 7f681cc00640 1 -- 172.21.15.67:0/3073904642 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v86) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6810058e30 con 0x7f68200adbe0 2024-02-26T10:43:09.141 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs2_data' removed 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.150+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b4d0 msgr2=0x7f67fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.150+0000 7f6825a00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b4d0 0x7f67fc06d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6820155990 tx=0x7f6808009040 comp rx=0 tx=0).stop 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.150+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 msgr2=0x7f68200a82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.150+0000 7f6825a00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 0x7f68200a82e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f681000d800 tx=0x7f681000dcc0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.151+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 shutdown_connections 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.151+0000 7f6825a00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68201568b0 0x7f682014d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.151+0000 7f6825a00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f67fc06b4d0 0x7f67fc06d990 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6820155990 tx=0x7f6808009040 comp rx=0 tx=0).stop 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.151+0000 7f6825a00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68200ae310 0x7f682014d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.151+0000 7f6825a00640 1 --2- 172.21.15.67:0/3073904642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68200adbe0 0x7f68200a82e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.151+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 >> 172.21.15.67:0/3073904642 conn(0x7f682006ac20 msgr2=0x7f68200c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.153+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 shutdown_connections 2024-02-26T10:43:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.153+0000 7f6825a00640 1 -- 172.21.15.67:0/3073904642 wait complete. 2024-02-26T10:43:09.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-02-26T10:43:09.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.371+0000 7f3022600640 1 Processor -- start 2024-02-26T10:43:09.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.371+0000 7f3022600640 1 -- start start 2024-02-26T10:43:09.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f3022600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f301c071ff0 0x7f301c0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f3022600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c0c3b00 con 0x7f301c0c4630 2024-02-26T10:43:09.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f3022600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c0c3de0 con 0x7f301c0c0b40 2024-02-26T10:43:09.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f3022600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c0c38d0 con 0x7f301c071ff0 2024-02-26T10:43:09.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f301be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f301c071ff0 0x7f301c0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f301be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f301c071ff0 0x7f301c0c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43950/0 (socket says 172.21.15.67:43950) 2024-02-26T10:43:09.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f301be00640 1 -- 172.21.15.67:0/3196183690 learned_addr learned my addr 172.21.15.67:0/3196183690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:09.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.373+0000 7f301b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f301c0c4630 0x7f301c154ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43514/0 (socket says 172.21.15.67:43514) 2024-02-26T10:43:09.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.374+0000 7f301be00640 1 -- 172.21.15.67:0/3196183690 >> v1:172.21.15.82:6789/0 conn(0x7f301c0c0b40 legacy=0x7f301c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.374+0000 7f301be00640 1 -- 172.21.15.67:0/3196183690 >> v1:172.21.15.67:6789/0 conn(0x7f301c0c4630 legacy=0x7f301c154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.374+0000 7f301be00640 1 -- 172.21.15.67:0/3196183690 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0c19c0 con 0x7f301c071ff0 2024-02-26T10:43:09.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.375+0000 7f301be00640 1 --2- 172.21.15.67:0/3196183690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c071ff0 0x7f301c0c4a10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f3014009e50 tx=0x7f301402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83d6650ed1e74b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.375+0000 7f301a000640 1 -- 172.21.15.67:0/3196183690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301402cbb0 con 0x7f301c071ff0 2024-02-26T10:43:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.375+0000 7f301a000640 1 -- 172.21.15.67:0/3196183690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301402cd50 con 0x7f301c071ff0 2024-02-26T10:43:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.376+0000 7f301a000640 1 -- 172.21.15.67:0/3196183690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014034610 con 0x7f301c071ff0 2024-02-26T10:43:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.376+0000 7f3022600640 1 -- 172.21.15.67:0/3196183690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c071ff0 msgr2=0x7f301c0c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.376+0000 7f3022600640 1 --2- 172.21.15.67:0/3196183690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c071ff0 0x7f301c0c4a10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f3014009e50 tx=0x7f301402c010 comp rx=0 tx=0).stop 2024-02-26T10:43:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.376+0000 7f3022600640 1 -- 172.21.15.67:0/3196183690 shutdown_connections 2024-02-26T10:43:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.376+0000 7f3022600640 1 --2- 172.21.15.67:0/3196183690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c071ff0 0x7f301c0c4a10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.376+0000 7f3022600640 1 -- 172.21.15.67:0/3196183690 >> 172.21.15.67:0/3196183690 conn(0x7f301c0c7b00 msgr2=0x7f301c0c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.377+0000 7f3022600640 1 -- 172.21.15.67:0/3196183690 shutdown_connections 2024-02-26T10:43:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.377+0000 7f3022600640 1 -- 172.21.15.67:0/3196183690 wait complete. 2024-02-26T10:43:09.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.378+0000 7f3022600640 1 Processor -- start 2024-02-26T10:43:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.378+0000 7f3022600640 1 -- start start 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.379+0000 7f3022600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f301c071ff0 0x7f301c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.379+0000 7f3022600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f301c0c0b40 0x7f301c1ee200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.379+0000 7f3022600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 0x7f301c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.379+0000 7f3022600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c152770 con 0x7f301c0c4630 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.379+0000 7f3022600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c072490 con 0x7f301c071ff0 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.379+0000 7f3022600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c1528f0 con 0x7f301c0c0b40 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 0x7f301c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 0x7f301c156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43964/0 (socket says 172.21.15.67:43964) 2024-02-26T10:43:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 -- 172.21.15.67:0/3409609069 learned_addr learned my addr 172.21.15.67:0/3409609069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301b400640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f301c071ff0 0x7f301c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 -- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f301c0c0b40 msgr2=0x7f301c1ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f301c0c0b40 0x7f301c1ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 -- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f301c071ff0 msgr2=0x7f301c1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f301c071ff0 0x7f301c1587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 -- 172.21.15.67:0/3409609069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c159340 con 0x7f301c0c4630 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301b400640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f301c071ff0 0x7f301c1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.380+0000 7f301be00640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 0x7f301c156f60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f3014009e50 tx=0x7f3014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.381+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014043020 con 0x7f301c0c4630 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.381+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301403e450 con 0x7f301c0c4630 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.381+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014009510 con 0x7f301c0c4630 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.381+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c1eea40 con 0x7f301c0c4630 2024-02-26T10:43:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.381+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c158fd0 con 0x7f301c0c4630 2024-02-26T10:43:09.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.382+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c074a30 con 0x7f301c0c4630 2024-02-26T10:43:09.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.383+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3014046020 con 0x7f301c0c4630 2024-02-26T10:43:09.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.384+0000 7f3013400640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.384+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f30140ac860 con 0x7f301c0c4630 2024-02-26T10:43:09.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.385+0000 7f301b400640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.386+0000 7f301b400640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ff806b4d0 0x7f2ff806d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f300c000f60 tx=0x7f300c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:09.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.389+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f301404d030 con 0x7f301c0c4630 2024-02-26T10:43:09.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.591+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f301c156cc0 con 0x7f301c0c4630 2024-02-26T10:43:09.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.598+0000 7f3013400640 1 -- 172.21.15.67:0/3409609069 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f301407b970 con 0x7f301c0c4630 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ff806b4d0 msgr2=0x7f2ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ff806b4d0 0x7f2ff806d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f300c000f60 tx=0x7f300c009040 comp rx=0 tx=0).stop 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 msgr2=0x7f301c156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 0x7f301c156f60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f3014009e50 tx=0x7f3014004a40 comp rx=0 tx=0).stop 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 shutdown_connections 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f301c0c4630 0x7f301c156f60 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f301c0c0b40 0x7f301c1ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 --2- 172.21.15.67:0/3409609069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f301c071ff0 0x7f301c1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.601+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 >> 172.21.15.67:0/3409609069 conn(0x7f301c0c7b00 msgr2=0x7f301c14f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.602+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 shutdown_connections 2024-02-26T10:43:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.602+0000 7f3022600640 1 -- 172.21.15.67:0/3409609069 wait complete. 2024-02-26T10:43:09.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap' 2024-02-26T10:43:09.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.821+0000 7fcfb2800640 1 Processor -- start 2024-02-26T10:43:09.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.822+0000 7fcfb2800640 1 -- start start 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.823+0000 7fcfb2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfac152460 0x7fcfac158d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.823+0000 7fcfb2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcfac156230 0x7fcfac0c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.823+0000 7fcfb2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac155840 con 0x7fcfac0c54e0 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.823+0000 7fcfb2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac158330 con 0x7fcfac156230 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.823+0000 7fcfb2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac1588f0 con 0x7fcfac152460 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcfac156230 0x7fcfac0c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcfac156230 0x7fcfac0c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43970/0 (socket says 172.21.15.67:43970) 2024-02-26T10:43:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 -- 172.21.15.67:0/4156994339 learned_addr learned my addr 172.21.15.67:0/4156994339 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 -- 172.21.15.67:0/4156994339 >> v2:172.21.15.82:3300/0 conn(0x7fcfac152460 msgr2=0x7fcfac158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 --2- 172.21.15.67:0/4156994339 >> v2:172.21.15.82:3300/0 conn(0x7fcfac152460 0x7fcfac158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 -- 172.21.15.67:0/4156994339 >> v1:172.21.15.82:6789/0 conn(0x7fcfac0c54e0 legacy=0x7fcfac154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.824+0000 7fcfb0a00640 1 -- 172.21.15.67:0/4156994339 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac155b20 con 0x7fcfac156230 2024-02-26T10:43:09.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.825+0000 7fcfb0a00640 1 --2- 172.21.15.67:0/4156994339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac156230 0x7fcfac0c4fa0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0c6fd0 tx=0x7fcfa002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d87dad7ae6e14b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfabe00640 1 -- 172.21.15.67:0/4156994339 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa002c960 con 0x7fcfac156230 2024-02-26T10:43:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfabe00640 1 -- 172.21.15.67:0/4156994339 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa002cb00 con 0x7fcfac156230 2024-02-26T10:43:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfabe00640 1 -- 172.21.15.67:0/4156994339 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00343f0 con 0x7fcfac156230 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfb2800640 1 -- 172.21.15.67:0/4156994339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac156230 msgr2=0x7fcfac0c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfb2800640 1 --2- 172.21.15.67:0/4156994339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac156230 0x7fcfac0c4fa0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0c6fd0 tx=0x7fcfa002bdc0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfb2800640 1 -- 172.21.15.67:0/4156994339 shutdown_connections 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfb2800640 1 --2- 172.21.15.67:0/4156994339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac156230 0x7fcfac0c4fa0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfb2800640 1 --2- 172.21.15.67:0/4156994339 >> v2:172.21.15.82:3300/0 conn(0x7fcfac152460 0x7fcfac158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.826+0000 7fcfb2800640 1 -- 172.21.15.67:0/4156994339 >> 172.21.15.67:0/4156994339 conn(0x7fcfac0ac1b0 msgr2=0x7fcfac0ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.827+0000 7fcfb2800640 1 -- 172.21.15.67:0/4156994339 shutdown_connections 2024-02-26T10:43:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.827+0000 7fcfb2800640 1 -- 172.21.15.67:0/4156994339 wait complete. 2024-02-26T10:43:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.828+0000 7fcfb2800640 1 Processor -- start 2024-02-26T10:43:09.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.828+0000 7fcfb2800640 1 -- start start 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.829+0000 7fcfb2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 0x7fcfac1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.829+0000 7fcfb2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfac152460 0x7fcfac0ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.829+0000 7fcfb2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfac156230 0x7fcfac153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.829+0000 7fcfb2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac155490 con 0x7fcfac0c54e0 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.829+0000 7fcfb2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1592c0 con 0x7fcfac152460 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.829+0000 7fcfb2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0af080 con 0x7fcfac156230 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 0x7fcfac1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 0x7fcfac1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43976/0 (socket says 172.21.15.67:43976) 2024-02-26T10:43:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 -- 172.21.15.67:0/3293916967 learned_addr learned my addr 172.21.15.67:0/3293916967 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 -- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfac156230 msgr2=0x7fcfac153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfac156230 0x7fcfac153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb0a00640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfac152460 0x7fcfac0ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 -- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfac152460 msgr2=0x7fcfac0ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfac152460 0x7fcfac0ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 -- 172.21.15.67:0/3293916967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac1eeb40 con 0x7fcfac0c54e0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.830+0000 7fcfb1400640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 0x7fcfac1503d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00b410 tx=0x7fcf9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.831+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c015020 con 0x7fcfac0c54e0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.831+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c011850 con 0x7fcfac0c54e0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.831+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c011c40 con 0x7fcfac0c54e0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.831+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac1edd80 con 0x7fcfac0c54e0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.831+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac1ee340 con 0x7fcfac0c54e0 2024-02-26T10:43:09.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.832+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac074a30 con 0x7fcfac0c54e0 2024-02-26T10:43:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.833+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf9c018020 con 0x7fcfac0c54e0 2024-02-26T10:43:09.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.834+0000 7fcfa9600640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf9006b4d0 0x7fcf9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.834+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcf9c089ea0 con 0x7fcfac0c54e0 2024-02-26T10:43:09.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.835+0000 7fcfb0a00640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf9006b4d0 0x7fcf9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.836+0000 7fcfb0a00640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf9006b4d0 0x7fcf9006d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0035000 tx=0x7fcfa0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:09.839+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf9c01f020 con 0x7fcfac0c54e0 2024-02-26T10:43:10.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.114+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]} v 0) v1 -- 0x7fcfac0c7850 con 0x7fcfac0c54e0 2024-02-26T10:43:10.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.159+0000 7fcfa9600640 1 -- 172.21.15.67:0/3293916967 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]}]=0 v278) v1 ==== 159+0+0 (secure 0 0 0) 0x7fcf9c0172c0 con 0x7fcfac0c54e0 2024-02-26T10:43:10.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf9006b4d0 msgr2=0x7fcf9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:10.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf9006b4d0 0x7fcf9006d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0035000 tx=0x7fcfa0030000 comp rx=0 tx=0).stop 2024-02-26T10:43:10.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 msgr2=0x7fcfac1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:10.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 0x7fcfac1503d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00b410 tx=0x7fcf9c00b8d0 comp rx=0 tx=0).stop 2024-02-26T10:43:10.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 shutdown_connections 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfac156230 0x7fcfac153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcf9006b4d0 0x7fcf9006d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfac152460 0x7fcfac0ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 --2- 172.21.15.67:0/3293916967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcfac0c54e0 0x7fcfac1503d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 >> 172.21.15.67:0/3293916967 conn(0x7fcfac0ac1b0 msgr2=0x7fcfac1512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 shutdown_connections 2024-02-26T10:43:10.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:10.162+0000 7fcfb2800640 1 -- 172.21.15.67:0/3293916967 wait complete. 2024-02-26T10:43:10.179 INFO:tasks.cephfs_test_runner:test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) ... ERROR 2024-02-26T10:43:10.179 INFO:tasks.cephfs_test_runner: 2024-02-26T10:43:10.179 INFO:tasks.cephfs_test_runner:====================================================================== 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner:ERROR: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/test_multifs_auth.py", line 95, in test_r_with_fsname_and_no_path_in_cap 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: self.captesters[0].run_mds_cap_tests(PERM) 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/caps_helper.py", line 262, in run_mds_cap_tests 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: self.conduct_pos_test_for_read_caps() 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/caps_helper.py", line 274, in conduct_pos_test_for_read_caps 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: contents = self.mount.read_file(self.path, sudo_read) 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/mount.py", line 749, in read_file 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: return self.run_shell(args=args, omit_sudo=False).stdout.getvalue().strip() 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/mount.py", line 784, in run_shell 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run 2024-02-26T10:43:10.180 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: r.wait() 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi067 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec cat /home/ubuntu/cephtest/mnt.0/testdir/testfile)' 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner:Ran 3 tests in 236.487s 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner: 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner:====================================================================== 2024-02-26T10:43:10.181 INFO:tasks.cephfs_test_runner:ERROR: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/test_multifs_auth.py", line 95, in test_r_with_fsname_and_no_path_in_cap 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: self.captesters[0].run_mds_cap_tests(PERM) 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/caps_helper.py", line 262, in run_mds_cap_tests 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: self.conduct_pos_test_for_read_caps() 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/caps_helper.py", line 274, in conduct_pos_test_for_read_caps 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: contents = self.mount.read_file(self.path, sudo_read) 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/mount.py", line 749, in read_file 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: return self.run_shell(args=args, omit_sudo=False).stdout.getvalue().strip() 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs/mount.py", line 784, in run_shell 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: r.wait() 2024-02-26T10:43:10.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait 2024-02-26T10:43:10.183 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-02-26T10:43:10.183 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-02-26T10:43:10.183 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-02-26T10:43:10.183 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi067 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec cat /home/ubuntu/cephtest/mnt.0/testdir/testfile)' 2024-02-26T10:43:10.183 INFO:tasks.cephfs_test_runner: 2024-02-26T10:43:10.183 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-02-26T10:43:10.393 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f7ce41d0343b44149a953eb69c385139 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_6085b91fefeb74bf4310344a87d81d3217ef7893/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-02-26T10:43:10.394 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-02-26T10:43:10.402 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-26T10:43:10.409 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-26T10:43:10.409 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-26T10:43:10.409 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:43:10.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053433,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249184k,nr_inodes=812296,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498372k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-02-26T10:43:10.471 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-02-26T10:43:10.471 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-26T10:43:10.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498336k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-02-26T10:43:10.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:10.530 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-02-26T10:43:10.589 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-26T10:43:10.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:10.590 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-26T10:43:10.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:10.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-02-26T10:43:10.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:10.826 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-02-26T10:43:10.881 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-26T10:43:10.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:10.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-26T10:43:10.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-26T10:43:10.974 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-02-26T10:43:11.095 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-02-26T10:43:11.105 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-26T10:43:11.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.338+0000 7f954f000640 1 Processor -- start 2024-02-26T10:43:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.338+0000 7f954f000640 1 -- start start 2024-02-26T10:43:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.339+0000 7f954f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f954806e660 0x7f9548159920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.339+0000 7f954f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954815c8a0 con 0x7f9548156810 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.339+0000 7f954f000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954815cb80 con 0x7f9548072fe0 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.339+0000 7f954f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954815ce60 con 0x7f954806e660 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.339+0000 7f954d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f954806e660 0x7f9548159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.339+0000 7f9547e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9548156810 0x7f95480c3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43516/0 (socket says 172.21.15.67:43516) 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.340+0000 7f954d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f954806e660 0x7f9548159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43988/0 (socket says 172.21.15.67:43988) 2024-02-26T10:43:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.340+0000 7f9547e00640 1 -- 172.21.15.67:0/3843918373 learned_addr learned my addr 172.21.15.67:0/3843918373 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.340+0000 7f954d000640 1 -- 172.21.15.67:0/3843918373 >> v1:172.21.15.82:6789/0 conn(0x7f9548072fe0 legacy=0x7f9548159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.340+0000 7f954d000640 1 -- 172.21.15.67:0/3843918373 >> v1:172.21.15.67:6789/0 conn(0x7f9548156810 legacy=0x7f95480c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.340+0000 7f954d000640 1 -- 172.21.15.67:0/3843918373 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954815d140 con 0x7f954806e660 2024-02-26T10:43:11.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.341+0000 7f954d000640 1 --2- 172.21.15.67:0/3843918373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954806e660 0x7f9548159920 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9540009d90 tx=0x7f954002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171aa0f4ce103f2c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:11.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.341+0000 7f9546a00640 1 -- 172.21.15.67:0/3843918373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540039070 con 0x7f954806e660 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.341+0000 7f9546a00640 1 -- 172.21.15.67:0/3843918373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954002cbd0 con 0x7f954806e660 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.341+0000 7f9546a00640 1 -- 172.21.15.67:0/3843918373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540034440 con 0x7f954806e660 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 -- 172.21.15.67:0/3843918373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954806e660 msgr2=0x7f9548159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 --2- 172.21.15.67:0/3843918373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954806e660 0x7f9548159920 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9540009d90 tx=0x7f954002be20 comp rx=0 tx=0).stop 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 -- 172.21.15.67:0/3843918373 shutdown_connections 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 --2- 172.21.15.67:0/3843918373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f954806e660 0x7f9548159920 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 -- 172.21.15.67:0/3843918373 >> 172.21.15.67:0/3843918373 conn(0x7f95480a84a0 msgr2=0x7f95480a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 -- 172.21.15.67:0/3843918373 shutdown_connections 2024-02-26T10:43:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.342+0000 7f954f000640 1 -- 172.21.15.67:0/3843918373 wait complete. 2024-02-26T10:43:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.344+0000 7f954f000640 1 Processor -- start 2024-02-26T10:43:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.344+0000 7f954f000640 1 -- start start 2024-02-26T10:43:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f954806e660 0x7f9548150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954f000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9548072fe0 0x7f95481f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 0x7f95480aa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954806ec40 con 0x7f9548156810 2024-02-26T10:43:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954f000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954815d660 con 0x7f9548072fe0 2024-02-26T10:43:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480c37a0 con 0x7f954806e660 2024-02-26T10:43:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 0x7f95480aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 0x7f95480aa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43990/0 (socket says 172.21.15.67:43990) 2024-02-26T10:43:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.345+0000 7f954d000640 1 -- 172.21.15.67:0/3553675365 learned_addr learned my addr 172.21.15.67:0/3553675365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f9547e00640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f954806e660 0x7f9548150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f9547400640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9548072fe0 0x7f95481f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954d000640 1 -- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f954806e660 msgr2=0x7f9548150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954d000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f954806e660 0x7f9548150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954d000640 1 -- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9548072fe0 msgr2=0x7f95481f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954d000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9548072fe0 0x7f95481f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954d000640 1 -- 172.21.15.67:0/3553675365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480ab430 con 0x7f9548156810 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f9547e00640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f954806e660 0x7f9548150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f9547400640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9548072fe0 0x7f95481f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954d000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 0x7f95480aa610 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f9540037040 tx=0x7f9540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540039050 con 0x7f9548156810 2024-02-26T10:43:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954003a040 con 0x7f9548156810 2024-02-26T10:43:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954003e610 con 0x7f9548156810 2024-02-26T10:43:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.346+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954815b940 con 0x7f9548156810 2024-02-26T10:43:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.347+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954815bf00 con 0x7f9548156810 2024-02-26T10:43:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.348+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548074a30 con 0x7f9548156810 2024-02-26T10:43:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.349+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9540047020 con 0x7f9548156810 2024-02-26T10:43:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.349+0000 7f953fe00640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f952806b520 0x7f952806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.350+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f95400ac230 con 0x7f9548156810 2024-02-26T10:43:11.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.350+0000 7f9547e00640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f952806b520 0x7f952806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.351+0000 7f9547e00640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f952806b520 0x7f952806d9e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9538000f60 tx=0x7f9538009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.355+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f954004e020 con 0x7f9548156810 2024-02-26T10:43:11.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.552+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f954806e440 con 0x7f952806b520 2024-02-26T10:43:11.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.554+0000 7f953fe00640 1 -- 172.21.15.67:0/3553675365 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13505 (secure 0 0 0) 0x7f954806e440 con 0x7f952806b520 2024-02-26T10:43:11.552 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:11.552 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:43:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f952806b520 msgr2=0x7f952806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f952806b520 0x7f952806d9e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9538000f60 tx=0x7f9538009040 comp rx=0 tx=0).stop 2024-02-26T10:43:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 msgr2=0x7f95480aa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 0x7f95480aa610 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f9540037040 tx=0x7f9540004a40 comp rx=0 tx=0).stop 2024-02-26T10:43:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 shutdown_connections 2024-02-26T10:43:11.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9548156810 0x7f95480aa610 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f952806b520 0x7f952806d9e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9548072fe0 0x7f95481f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 --2- 172.21.15.67:0/3553675365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f954806e660 0x7f9548150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 >> 172.21.15.67:0/3553675365 conn(0x7f95480a84a0 msgr2=0x7f95480c23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:11.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 shutdown_connections 2024-02-26T10:43:11.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.557+0000 7f954f000640 1 -- 172.21.15.67:0/3553675365 wait complete. 2024-02-26T10:43:11.575 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-02-26T10:43:10.433342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258772,"kb_used_data":9332,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715948,"statfs":{"total":773094113280,"available":772829130752,"internally_reserved":0,"allocated":9555968,"data_stored":4951125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12392,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-64,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002649"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":291,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-26T10:43:05.850585+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:43:05.850585+0000","last_peered":"2024-02-26T10:43:05.850585+0000","last_clean":"2024-02-26T10:43:05.850585+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:43:05.850585+0000","last_undegraded":"2024-02-26T10:43:05.850585+0000","last_fullsized":"2024-02-26T10:43:05.850585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1638400,"data_stored":1059452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":469624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":469042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":480552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542204,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":480450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":462973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":469110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1634304,"data_stored":1059922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:11.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:11.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.802+0000 7fd237600640 1 Processor -- start 2024-02-26T10:43:11.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.802+0000 7fd237600640 1 -- start start 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd237600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd23014f3d0 0x7fd23006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd237600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd230071700 0x7fd230071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd237600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23006ed10 con 0x7fd2300c2870 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd237600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23006dd80 con 0x7fd230071700 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd237600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd230155c00 con 0x7fd23014f3d0 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd235600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd230071700 0x7fd230071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd235600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd230071700 0x7fd230071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43996/0 (socket says 172.21.15.67:43996) 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd235600640 1 -- 172.21.15.67:0/1798228924 learned_addr learned my addr 172.21.15.67:0/1798228924 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:11.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd22fe00640 1 --1- 172.21.15.67:0/1798228924 >> v1:172.21.15.67:6789/0 conn(0x7fd2300c2870 0x7fd230155160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43526/0 (socket says 172.21.15.67:43526) 2024-02-26T10:43:11.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.804+0000 7fd234c00640 1 --2- 172.21.15.67:0/1798228924 >> v2:172.21.15.82:3300/0 conn(0x7fd23014f3d0 0x7fd23006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.805+0000 7fd235600640 1 -- 172.21.15.67:0/1798228924 >> v2:172.21.15.82:3300/0 conn(0x7fd23014f3d0 msgr2=0x7fd23006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:11.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.805+0000 7fd235600640 1 --2- 172.21.15.67:0/1798228924 >> v2:172.21.15.82:3300/0 conn(0x7fd23014f3d0 0x7fd23006e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.805+0000 7fd235600640 1 -- 172.21.15.67:0/1798228924 >> v1:172.21.15.67:6789/0 conn(0x7fd2300c2870 legacy=0x7fd230155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.805+0000 7fd235600640 1 -- 172.21.15.67:0/1798228924 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230155ee0 con 0x7fd230071700 2024-02-26T10:43:11.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.806+0000 7fd235600640 1 --2- 172.21.15.67:0/1798228924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230071700 0x7fd230071b30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd2300c41c0 tx=0x7fd22802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc48fe48c73baa6d server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:11.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.806+0000 7fd22f400640 1 -- 172.21.15.67:0/1798228924 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22802ec90 con 0x7fd230071700 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.806+0000 7fd22f400640 1 -- 172.21.15.67:0/1798228924 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22802ee30 con 0x7fd230071700 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.806+0000 7fd22f400640 1 -- 172.21.15.67:0/1798228924 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280365f0 con 0x7fd230071700 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 -- 172.21.15.67:0/1798228924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230071700 msgr2=0x7fd230071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 --2- 172.21.15.67:0/1798228924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230071700 0x7fd230071b30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd2300c41c0 tx=0x7fd22802e0f0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 -- 172.21.15.67:0/1798228924 shutdown_connections 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 --2- 172.21.15.67:0/1798228924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd230071700 0x7fd230071b30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 --2- 172.21.15.67:0/1798228924 >> v2:172.21.15.82:3300/0 conn(0x7fd23014f3d0 0x7fd23006e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 -- 172.21.15.67:0/1798228924 >> 172.21.15.67:0/1798228924 conn(0x7fd2300a84a0 msgr2=0x7fd2300a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:11.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 -- 172.21.15.67:0/1798228924 shutdown_connections 2024-02-26T10:43:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.807+0000 7fd237600640 1 -- 172.21.15.67:0/1798228924 wait complete. 2024-02-26T10:43:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.809+0000 7fd237600640 1 Processor -- start 2024-02-26T10:43:11.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.809+0000 7fd237600640 1 -- start start 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd237600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd230071700 0x7fd2301dc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd237600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2300c2870 0x7fd230153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd237600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 0x7fd230153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd237600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2300c1810 con 0x7fd23014f3d0 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd237600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23006e2b0 con 0x7fd230071700 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd237600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd230155870 con 0x7fd2300c2870 2024-02-26T10:43:11.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd235600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 0x7fd230153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd235600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 0x7fd230153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43998/0 (socket says 172.21.15.67:43998) 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd235600640 1 -- 172.21.15.67:0/2550758268 learned_addr learned my addr 172.21.15.67:0/2550758268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd234c00640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd230071700 0x7fd2301dc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.810+0000 7fd22fe00640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2300c2870 0x7fd230153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd235600640 1 -- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2300c2870 msgr2=0x7fd230153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd235600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2300c2870 0x7fd230153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd235600640 1 -- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd230071700 msgr2=0x7fd2301dc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:11.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd235600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd230071700 0x7fd2301dc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd235600640 1 -- 172.21.15.67:0/2550758268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2301556a0 con 0x7fd23014f3d0 2024-02-26T10:43:11.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd235600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 0x7fd230153cb0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd22802e700 tx=0x7fd228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228045070 con 0x7fd23014f3d0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228041640 con 0x7fd23014f3d0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23014c1c0 con 0x7fd23014f3d0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228009510 con 0x7fd23014f3d0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.811+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23014c460 con 0x7fd23014f3d0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.812+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd230074a30 con 0x7fd23014f3d0 2024-02-26T10:43:11.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.814+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd228048070 con 0x7fd23014f3d0 2024-02-26T10:43:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.815+0000 7fd22cc00640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd21006b550 0x7fd21006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.815+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd2280ae8d0 con 0x7fd23014f3d0 2024-02-26T10:43:11.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.815+0000 7fd234c00640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd21006b550 0x7fd21006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:11.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.816+0000 7fd234c00640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd21006b550 0x7fd21006da10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:11.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:11.819+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd22807db80 con 0x7fd23014f3d0 2024-02-26T10:43:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.014+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2300a9590 con 0x7fd21006b550 2024-02-26T10:43:12.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.016+0000 7fd22cc00640 1 -- 172.21.15.67:0/2550758268 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13505 (secure 0 0 0) 0x7fd2300a9590 con 0x7fd21006b550 2024-02-26T10:43:12.014 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:43:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd21006b550 msgr2=0x7fd21006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd21006b550 0x7fd21006da10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224008040 comp rx=0 tx=0).stop 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 msgr2=0x7fd230153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 0x7fd230153cb0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd22802e700 tx=0x7fd228004a40 comp rx=0 tx=0).stop 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 shutdown_connections 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd23014f3d0 0x7fd230153cb0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2300c2870 0x7fd230153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd21006b550 0x7fd21006da10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 --2- 172.21.15.67:0/2550758268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd230071700 0x7fd2301dc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 >> 172.21.15.67:0/2550758268 conn(0x7fd2300a84a0 msgr2=0x7fd2300c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 shutdown_connections 2024-02-26T10:43:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.019+0000 7fd237600640 1 -- 172.21.15.67:0/2550758268 wait complete. 2024-02-26T10:43:12.034 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-02-26T10:43:10.433342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258772,"kb_used_data":9332,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715948,"statfs":{"total":773094113280,"available":772829130752,"internally_reserved":0,"allocated":9555968,"data_stored":4951125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12392,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-64,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002649"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":291,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-26T10:43:05.850585+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:43:05.850585+0000","last_peered":"2024-02-26T10:43:05.850585+0000","last_clean":"2024-02-26T10:43:05.850585+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:43:05.850585+0000","last_undegraded":"2024-02-26T10:43:05.850585+0000","last_fullsized":"2024-02-26T10:43:05.850585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1638400,"data_stored":1059452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":469624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":469042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":480552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542204,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":480450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":462973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":469110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1634304,"data_stored":1059922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:12.035 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-26T10:43:12.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:12.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.244+0000 7f8cd3200640 1 Processor -- start 2024-02-26T10:43:12.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.244+0000 7f8cd3200640 1 -- start start 2024-02-26T10:43:12.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc152380 0x7f8ccc0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ccc06d290 0x7f8ccc0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd3200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc152760 con 0x7f8ccc0c7020 2024-02-26T10:43:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c7550 con 0x7f8ccc06d290 2024-02-26T10:43:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c7830 con 0x7f8ccc152380 2024-02-26T10:43:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ccc06d290 0x7f8ccc0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ccc06d290 0x7f8ccc0c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44008/0 (socket says 172.21.15.67:44008) 2024-02-26T10:43:12.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.246+0000 7f8cd2800640 1 -- 172.21.15.67:0/1172860411 learned_addr learned my addr 172.21.15.67:0/1172860411 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:12.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.247+0000 7f8cd1e00640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc152380 0x7f8ccc0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.247+0000 7f8cd2800640 1 -- 172.21.15.67:0/1172860411 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc152380 msgr2=0x7f8ccc0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.247+0000 7f8cd2800640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc152380 0x7f8ccc0c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.247+0000 7f8cd2800640 1 -- 172.21.15.67:0/1172860411 >> v1:172.21.15.82:6789/0 conn(0x7f8ccc0c7020 legacy=0x7f8ccc155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.247+0000 7f8cd2800640 1 -- 172.21.15.67:0/1172860411 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc06ca90 con 0x7f8ccc06d290 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.247+0000 7f8cd1e00640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc152380 0x7f8ccc0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.248+0000 7f8cd2800640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc06d290 0x7f8ccc0c4f90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009870 tx=0x7f8cc802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdde1ad189f62e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.248+0000 7f8cd0a00640 1 -- 172.21.15.67:0/1172860411 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80317c0 con 0x7f8ccc06d290 2024-02-26T10:43:12.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.248+0000 7f8cd0a00640 1 -- 172.21.15.67:0/1172860411 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8037070 con 0x7f8ccc06d290 2024-02-26T10:43:12.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.248+0000 7f8cd0a00640 1 -- 172.21.15.67:0/1172860411 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8031b80 con 0x7f8ccc06d290 2024-02-26T10:43:12.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 -- 172.21.15.67:0/1172860411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc06d290 msgr2=0x7f8ccc0c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc06d290 0x7f8ccc0c4f90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009870 tx=0x7f8cc802b860 comp rx=0 tx=0).stop 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 -- 172.21.15.67:0/1172860411 shutdown_connections 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc06d290 0x7f8ccc0c4f90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1172860411 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc152380 0x7f8ccc0c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 -- 172.21.15.67:0/1172860411 >> 172.21.15.67:0/1172860411 conn(0x7f8ccc0ac1a0 msgr2=0x7f8ccc0ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 -- 172.21.15.67:0/1172860411 shutdown_connections 2024-02-26T10:43:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.249+0000 7f8cd3200640 1 -- 172.21.15.67:0/1172860411 wait complete. 2024-02-26T10:43:12.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.251+0000 7f8cd3200640 1 Processor -- start 2024-02-26T10:43:12.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.251+0000 7f8cd3200640 1 -- start start 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc06d290 0x7f8ccc1ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ccc0c7020 0x7f8ccc14b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 0x7f8ccc14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc06d8a0 con 0x7f8ccc152380 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc06d720 con 0x7f8ccc0c7020 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc06da20 con 0x7f8ccc06d290 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 0x7f8ccc14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 0x7f8ccc14abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44010/0 (socket says 172.21.15.67:44010) 2024-02-26T10:43:12.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.252+0000 7f8cd2800640 1 -- 172.21.15.67:0/1345793872 learned_addr learned my addr 172.21.15.67:0/1345793872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd1e00640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc06d290 0x7f8ccc1ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd1400640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ccc0c7020 0x7f8ccc14b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd2800640 1 -- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc06d290 msgr2=0x7f8ccc1ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd2800640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc06d290 0x7f8ccc1ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd2800640 1 -- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ccc0c7020 msgr2=0x7f8ccc14b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd2800640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ccc0c7020 0x7f8ccc14b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd2800640 1 -- 172.21.15.67:0/1345793872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc1eea20 con 0x7f8ccc152380 2024-02-26T10:43:12.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.253+0000 7f8cd2800640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 0x7f8ccc14abe0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009870 tx=0x7f8cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.254+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8032860 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.254+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8032a00 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.254+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc0af9f0 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.254+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8032df0 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.254+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1ee570 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.255+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc074a30 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.256+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cc80453f0 con 0x7f8ccc152380 2024-02-26T10:43:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.257+0000 7f8cc6000640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.257+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8cc80ac290 con 0x7f8ccc152380 2024-02-26T10:43:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.257+0000 7f8cd1e00640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.259+0000 7f8cd1e00640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8cb006b4d0 0x7f8cb006d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007510 tx=0x7f8cc00074a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.261+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cc8040070 con 0x7f8ccc152380 2024-02-26T10:43:12.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.462+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ccc1eed00 con 0x7f8cb006b4d0 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.465+0000 7f8cc6000640 1 -- 172.21.15.67:0/1345793872 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13505 (secure 0 0 0) 0x7f8ccc1eed00 con 0x7f8cb006b4d0 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:12.462 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:43:12.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.467+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8cb006b4d0 msgr2=0x7f8cb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.467+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8cb006b4d0 0x7f8cb006d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007510 tx=0x7f8cc00074a0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.467+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 msgr2=0x7f8ccc14abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.467+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 0x7f8ccc14abe0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009870 tx=0x7f8cc80046c0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 shutdown_connections 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ccc152380 0x7f8ccc14abe0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ccc0c7020 0x7f8ccc14b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 --2- 172.21.15.67:0/1345793872 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ccc06d290 0x7f8ccc1ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 >> 172.21.15.67:0/1345793872 conn(0x7f8ccc0ac1a0 msgr2=0x7f8ccc14d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 shutdown_connections 2024-02-26T10:43:12.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.468+0000 7f8cd3200640 1 -- 172.21.15.67:0/1345793872 wait complete. 2024-02-26T10:43:12.485 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-02-26T10:43:12.434051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258772,"kb_used_data":9332,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715948,"statfs":{"total":773094113280,"available":772829130752,"internally_reserved":0,"allocated":9555968,"data_stored":4951125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12392,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-64,"num_write_kb":-34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.587594"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":291,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-26T10:43:05.850585+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:43:05.850585+0000","last_peered":"2024-02-26T10:43:05.850585+0000","last_clean":"2024-02-26T10:43:05.850585+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:43:05.850585+0000","last_undegraded":"2024-02-26T10:43:05.850585+0000","last_fullsized":"2024-02-26T10:43:05.850585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1638400,"data_stored":1059452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":469624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":469042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542204,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":480552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542204,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":480450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":462973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":469110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542204,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1634304,"data_stored":1059922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:12.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-26T10:43:12.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.686+0000 7fd3c4800640 1 Processor -- start 2024-02-26T10:43:12.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.686+0000 7fd3c4800640 1 -- start start 2024-02-26T10:43:12.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.687+0000 7fd3c4800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc157d30 0x7fd3bc158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.687+0000 7fd3c4800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd3bc0bdef0 0x7fd3bc0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.687+0000 7fd3c4800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc154f50 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.687+0000 7fd3c4800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c7440 con 0x7fd3bc0bdef0 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.687+0000 7fd3c4800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c7720 con 0x7fd3bc157d30 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd3bc0bdef0 0x7fd3bc0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c2800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd3bc0c56e0 0x7fd3bc155400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43532/0 (socket says 172.21.15.67:43532) 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd3bc0bdef0 0x7fd3bc0c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44016/0 (socket says 172.21.15.67:44016) 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c2800640 1 -- 172.21.15.67:0/1110628616 learned_addr learned my addr 172.21.15.67:0/1110628616 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1e00640 1 --2- 172.21.15.67:0/1110628616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc157d30 0x7fd3bc158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1400640 1 -- 172.21.15.67:0/1110628616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc157d30 msgr2=0x7fd3bc158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1400640 1 --2- 172.21.15.67:0/1110628616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc157d30 0x7fd3bc158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1400640 1 -- 172.21.15.67:0/1110628616 >> v1:172.21.15.67:6789/0 conn(0x7fd3bc0c56e0 legacy=0x7fd3bc155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c0a00640 1 -- 172.21.15.67:0/1110628616 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607107437 0 0) 0x7fd3bc154f50 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.688+0000 7fd3c1400640 1 -- 172.21.15.67:0/1110628616 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc0ad2a0 con 0x7fd3bc0bdef0 2024-02-26T10:43:12.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.690+0000 7fd3c1400640 1 --2- 172.21.15.67:0/1110628616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0bdef0 0x7fd3bc0c51a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0097a0 tx=0x7fd3ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9547b232d98b9e server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.690+0000 7fd3c0a00640 1 -- 172.21.15.67:0/1110628616 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0315a0 con 0x7fd3bc0bdef0 2024-02-26T10:43:12.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.690+0000 7fd3c0a00640 1 -- 172.21.15.67:0/1110628616 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac037070 con 0x7fd3bc0bdef0 2024-02-26T10:43:12.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.690+0000 7fd3c0a00640 1 -- 172.21.15.67:0/1110628616 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac031960 con 0x7fd3bc0bdef0 2024-02-26T10:43:12.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 -- 172.21.15.67:0/1110628616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0bdef0 msgr2=0x7fd3bc0c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1110628616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0bdef0 0x7fd3bc0c51a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0097a0 tx=0x7fd3ac02b840 comp rx=0 tx=0).stop 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 -- 172.21.15.67:0/1110628616 shutdown_connections 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1110628616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0bdef0 0x7fd3bc0c51a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1110628616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc157d30 0x7fd3bc158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 -- 172.21.15.67:0/1110628616 >> 172.21.15.67:0/1110628616 conn(0x7fd3bc072980 msgr2=0x7fd3bc0ac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 -- 172.21.15.67:0/1110628616 shutdown_connections 2024-02-26T10:43:12.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.691+0000 7fd3c4800640 1 -- 172.21.15.67:0/1110628616 wait complete. 2024-02-26T10:43:12.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.693+0000 7fd3c4800640 1 Processor -- start 2024-02-26T10:43:12.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.693+0000 7fd3c4800640 1 -- start start 2024-02-26T10:43:12.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c4800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc0bdef0 0x7fd3bc1e1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c4800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 0x7fd3bc150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c4800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3bc157d30 0x7fd3bc151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c4800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc156330 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c4800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c7a00 con 0x7fd3bc157d30 2024-02-26T10:43:12.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c4800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc155b10 con 0x7fd3bc0bdef0 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 0x7fd3bc150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 0x7fd3bc150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44028/0 (socket says 172.21.15.67:44028) 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c1400640 1 -- 172.21.15.67:0/1547247352 learned_addr learned my addr 172.21.15.67:0/1547247352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c1e00640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc0bdef0 0x7fd3bc1e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c2800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3bc157d30 0x7fd3bc151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.694+0000 7fd3c1400640 1 -- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc0bdef0 msgr2=0x7fd3bc1e1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c1400640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc0bdef0 0x7fd3bc1e1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c1400640 1 -- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3bc157d30 msgr2=0x7fd3bc151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c1400640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3bc157d30 0x7fd3bc151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c1400640 1 -- 172.21.15.67:0/1547247352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc0c3810 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c1e00640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc0bdef0 0x7fd3bc1e1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:12.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c2800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3bc157d30 0x7fd3bc151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:12.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c1400640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 0x7fd3bc150cc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0098c0 tx=0x7fd3ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac032630 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac0327d0 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac032ba0 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.695+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc0c3af0 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.696+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc1e1c80 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.697+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc074a30 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.698+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3ac04e430 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.698+0000 7fd3b6000640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3a006b4d0 0x7fd3a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:12.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.698+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd3ac02f030 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.699+0000 7fd3c1e00640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3a006b4d0 0x7fd3a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:12.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.700+0000 7fd3c1e00640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3a006b4d0 0x7fd3a006d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0006fd0 tx=0x7fd3b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:12.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.703+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3ac040070 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.903+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3bc0acef0 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.905+0000 7fd3b6000640 1 -- 172.21.15.67:0/1547247352 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13079 (secure 0 0 0) 0x7fd3ac07af40 con 0x7fd3bc0c56e0 2024-02-26T10:43:12.903 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:12.903 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":86,"fsid":"f4a3a702-2ce8-46d3-b32d-e4f6872b8b58","created":"2024-02-26T10:38:15.047297+0000","modified":"2024-02-26T10:43:09.140744+0000","last_up_change":"2024-02-26T10:38:27.389640+0000","last_in_change":"2024-02-26T10:38:18.735515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-26T10:38:28.395212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"239e9e71-e60d-4d32-b8a3-4a3dce0021cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6801","nonce":1950536715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6803","nonce":1950536715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6807","nonce":1950536715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1950536715},{"type":"v1","addr":"172.21.15.67:6805","nonce":1950536715}]},"public_addr":"172.21.15.67:6801/1950536715","cluster_addr":"172.21.15.67:6803/1950536715","heartbeat_back_addr":"172.21.15.67:6807/1950536715","heartbeat_front_addr":"172.21.15.67:6805/1950536715","state":["exists","up"]},{"osd":1,"uuid":"f6dd1fff-5c4f-4c8a-954a-49f49124cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6817","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6819","nonce":1684785661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6824","nonce":1684785661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6829","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6831","nonce":1684785661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1684785661},{"type":"v1","addr":"172.21.15.67:6828","nonce":1684785661}]},"public_addr":"172.21.15.67:6819/1684785661","cluster_addr":"172.21.15.67:6824/1684785661","heartbeat_back_addr":"172.21.15.67:6831/1684785661","heartbeat_front_addr":"172.21.15.67:6828/1684785661","state":["exists","up"]},{"osd":2,"uuid":"d5b606f9-72f7-49f5-9c8c-cf21d2fd263d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6812","nonce":2562492504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6815","nonce":2562492504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6827","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6830","nonce":2562492504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2562492504},{"type":"v1","addr":"172.21.15.67:6821","nonce":2562492504}]},"public_addr":"172.21.15.67:6812/2562492504","cluster_addr":"172.21.15.67:6815/2562492504","heartbeat_back_addr":"172.21.15.67:6830/2562492504","heartbeat_front_addr":"172.21.15.67:6821/2562492504","state":["exists","up"]},{"osd":3,"uuid":"6f93e7e6-8121-4666-8641-c3c6d1e43fea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6809","nonce":755501566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6813","nonce":755501566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6823","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6825","nonce":755501566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":755501566},{"type":"v1","addr":"172.21.15.67:6820","nonce":755501566}]},"public_addr":"172.21.15.67:6809/755501566","cluster_addr":"172.21.15.67:6813/755501566","heartbeat_back_addr":"172.21.15.67:6825/755501566","heartbeat_front_addr":"172.21.15.67:6820/755501566","state":["exists","up"]},{"osd":4,"uuid":"e6238c1b-6838-4531-86e8-8decc09c7a29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6817","nonce":900472103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6819","nonce":900472103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6823","nonce":900472103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":900472103},{"type":"v1","addr":"172.21.15.82:6821","nonce":900472103}]},"public_addr":"172.21.15.82:6817/900472103","cluster_addr":"172.21.15.82:6819/900472103","heartbeat_back_addr":"172.21.15.82:6823/900472103","heartbeat_front_addr":"172.21.15.82:6821/900472103","state":["exists","up"]},{"osd":5,"uuid":"5f87545d-44e7-41af-bbcc-306a21a8159d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6801","nonce":431290960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6803","nonce":431290960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6807","nonce":431290960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":431290960},{"type":"v1","addr":"172.21.15.82:6805","nonce":431290960}]},"public_addr":"172.21.15.82:6801/431290960","cluster_addr":"172.21.15.82:6803/431290960","heartbeat_back_addr":"172.21.15.82:6807/431290960","heartbeat_front_addr":"172.21.15.82:6805/431290960","state":["exists","up"]},{"osd":6,"uuid":"b510a5fe-eeb5-4ae9-807e-95e4e00db1cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6809","nonce":1261740917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6811","nonce":1261740917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6815","nonce":1261740917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1261740917},{"type":"v1","addr":"172.21.15.82:6813","nonce":1261740917}]},"public_addr":"172.21.15.82:6809/1261740917","cluster_addr":"172.21.15.82:6811/1261740917","heartbeat_back_addr":"172.21.15.82:6815/1261740917","heartbeat_front_addr":"172.21.15.82:6813/1261740917","state":["exists","up"]},{"osd":7,"uuid":"dcd292f3-26a3-47f5-ba3d-ebb61f72348b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6825","nonce":3253207267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6827","nonce":3253207267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6831","nonce":3253207267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3253207267},{"type":"v1","addr":"172.21.15.82:6829","nonce":3253207267}]},"public_addr":"172.21.15.82:6825/3253207267","cluster_addr":"172.21.15.82:6827/3253207267","heartbeat_back_addr":"172.21.15.82:6831/3253207267","heartbeat_front_addr":"172.21.15.82:6829/3253207267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.049502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.310038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.346674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.113661+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:26.708574+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.259054+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.038047+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-26T10:38:24.109194+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6834/2406048118":"2024-02-27T10:43:05.839681+0000","172.21.15.67:6832/2406048118":"2024-02-27T10:43:05.839681+0000","172.21.15.67:6835/2183069114":"2024-02-27T10:42:58.049528+0000","172.21.15.67:6833/2183069114":"2024-02-27T10:42:58.049528+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-26T10:43:12.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3a006b4d0 msgr2=0x7fd3a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3a006b4d0 0x7fd3a006d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0006fd0 tx=0x7fd3b0008040 comp rx=0 tx=0).stop 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 msgr2=0x7fd3bc150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 0x7fd3bc150cc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0098c0 tx=0x7fd3ac0046c0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 shutdown_connections 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3a006b4d0 0x7fd3a006d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3bc157d30 0x7fd3bc151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3bc0c56e0 0x7fd3bc150cc0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 --2- 172.21.15.67:0/1547247352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3bc0bdef0 0x7fd3bc1e1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:12.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 >> 172.21.15.67:0/1547247352 conn(0x7fd3bc072980 msgr2=0x7fd3bc06c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:12.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 shutdown_connections 2024-02-26T10:43:12.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:12.908+0000 7fd3c4800640 1 -- 172.21.15.67:0/1547247352 wait complete. 2024-02-26T10:43:13.924 INFO:tasks.ceph:Scrubbing osd.0 2024-02-26T10:43:13.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:14.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.164+0000 7f76f2a00640 1 Processor -- start 2024-02-26T10:43:14.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.164+0000 7f76f2a00640 1 -- start start 2024-02-26T10:43:14.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1503a0 0x7f76ec0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f76ec06e2d0 0x7f76ec0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f2a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ec150780 con 0x7f76ec0a9430 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f2a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ec0a98c0 con 0x7f76ec06e2d0 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0a9ba0 con 0x7f76ec1503a0 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f0c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f76ec0a9430 0x7f76ec1535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43536/0 (socket says 172.21.15.67:43536) 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f0c00640 1 -- 172.21.15.67:0/844815676 learned_addr learned my addr 172.21.15.67:0/844815676 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.166+0000 7f76f2000640 1 --2- 172.21.15.67:0/844815676 >> v2:172.21.15.67:3300/0 conn(0x7f76ec06e2d0 0x7f76ec0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.167+0000 7f76f1600640 1 --2- 172.21.15.67:0/844815676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1503a0 0x7f76ec0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.167+0000 7f76f2000640 1 -- 172.21.15.67:0/844815676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1503a0 msgr2=0x7f76ec0ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:14.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.167+0000 7f76f2000640 1 --2- 172.21.15.67:0/844815676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1503a0 0x7f76ec0ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.167+0000 7f76ebe00640 1 -- 172.21.15.67:0/844815676 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296854187 0 0) 0x7f76ec150780 con 0x7f76ec0a9430 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.167+0000 7f76f2000640 1 -- 172.21.15.67:0/844815676 >> v1:172.21.15.67:6789/0 conn(0x7f76ec0a9430 legacy=0x7f76ec1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.167+0000 7f76f2000640 1 -- 172.21.15.67:0/844815676 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0a9e80 con 0x7f76ec06e2d0 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.168+0000 7f76f2000640 1 --2- 172.21.15.67:0/844815676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec0c2e10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009d90 tx=0x7f76e402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e585339781eb2ddc server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.168+0000 7f76ebe00640 1 -- 172.21.15.67:0/844815676 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4039070 con 0x7f76ec06e2d0 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.168+0000 7f76ebe00640 1 -- 172.21.15.67:0/844815676 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e402cc50 con 0x7f76ec06e2d0 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.168+0000 7f76ebe00640 1 -- 172.21.15.67:0/844815676 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4034440 con 0x7f76ec06e2d0 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 -- 172.21.15.67:0/844815676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 msgr2=0x7f76ec0c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 --2- 172.21.15.67:0/844815676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec0c2e10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009d90 tx=0x7f76e402bea0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 -- 172.21.15.67:0/844815676 shutdown_connections 2024-02-26T10:43:14.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 --2- 172.21.15.67:0/844815676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec0c2e10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 --2- 172.21.15.67:0/844815676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1503a0 0x7f76ec0ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 -- 172.21.15.67:0/844815676 >> 172.21.15.67:0/844815676 conn(0x7f76ec0c6380 msgr2=0x7f76ec0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 -- 172.21.15.67:0/844815676 shutdown_connections 2024-02-26T10:43:14.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.169+0000 7f76f2a00640 1 -- 172.21.15.67:0/844815676 wait complete. 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.171+0000 7f76f2a00640 1 Processor -- start 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.171+0000 7f76f2a00640 1 -- start start 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec1e9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec1503a0 0x7f76ec1de300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1debf0 0x7f76ec151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0c0f80 con 0x7f76ec06e2d0 2024-02-26T10:43:14.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec06ddb0 con 0x7f76ec1503a0 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec06e760 con 0x7f76ec1debf0 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec1e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec1e9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44046/0 (socket says 172.21.15.67:44046) 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 -- 172.21.15.67:0/2493610867 learned_addr learned my addr 172.21.15.67:0/2493610867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 -- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1debf0 msgr2=0x7f76ec151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f76ec1debf0 0x7f76ec151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f0c00640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec1503a0 0x7f76ec1de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 -- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec1503a0 msgr2=0x7f76ec1de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec1503a0 0x7f76ec1de300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.172+0000 7f76f1600640 1 -- 172.21.15.67:0/2493610867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec1513d0 con 0x7f76ec06e2d0 2024-02-26T10:43:14.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.173+0000 7f76f1600640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec1e9300 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f76e000d620 tx=0x7f76e000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.173+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000b620 con 0x7f76ec06e2d0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.173+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0019070 con 0x7f76ec06e2d0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.173+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec1516b0 con 0x7f76ec06e2d0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.173+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000ba00 con 0x7f76ec06e2d0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.173+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec1e8b00 con 0x7f76ec06e2d0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.174+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76ec0c36a0 con 0x7f76ec06e2d0 2024-02-26T10:43:14.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.175+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76e000bba0 con 0x7f76ec06e2d0 2024-02-26T10:43:14.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.176+0000 7f76e9600640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.176+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f76e008acd0 con 0x7f76ec06e2d0 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.176+0000 7f76e9600640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f76cc071460 0x7f76cc073880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.176+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 --> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76e0023e50 con 0x7f76cc071460 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.176+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e008b160 con 0x7f76ec06e2d0 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.177+0000 7f76f2000640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f76cc071460 0x7f76cc073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.177+0000 7f76f0c00640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.177+0000 7f76f2000640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f76cc071460 0x7f76cc073880 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.178+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== osd.0 v2:172.21.15.67:6800/1950536715 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f76e0023e50 con 0x7f76cc071460 2024-02-26T10:43:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.178+0000 7f76f0c00640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f76ec1dc810 tx=0x7f76d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.194+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 --> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f76ec074a30 con 0x7f76cc071460 2024-02-26T10:43:14.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.194+0000 7f76e9600640 1 -- 172.21.15.67:0/2493610867 <== osd.0 v2:172.21.15.67:6800/1950536715 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f76ec074a30 con 0x7f76cc071460 2024-02-26T10:43:14.192 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:14.192 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.194+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f76cc071460 msgr2=0x7f76cc073880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.194+0000 7f76f2a00640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:6800/1950536715,v1:172.21.15.67:6801/1950536715] conn(0x7f76cc071460 0x7f76cc073880 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 msgr2=0x7f76cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f76cc06b500 0x7f76cc06d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f76ec1dc810 tx=0x7f76d8009040 comp rx=0 tx=0).stop 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 msgr2=0x7f76ec1e9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 --2- 172.21.15.67:0/2493610867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76ec06e2d0 0x7f76ec1e9300 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f76e000d620 tx=0x7f76e000dae0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2000640 1 -- 172.21.15.67:0/2493610867 reap_dead start 2024-02-26T10:43:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 shutdown_connections 2024-02-26T10:43:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 >> 172.21.15.67:0/2493610867 conn(0x7f76ec0c6380 msgr2=0x7f76ec0c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 shutdown_connections 2024-02-26T10:43:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.195+0000 7f76f2a00640 1 -- 172.21.15.67:0/2493610867 wait complete. 2024-02-26T10:43:14.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-02-26T10:43:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.428+0000 7f8f3ba00640 1 Processor -- start 2024-02-26T10:43:14.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.428+0000 7f8f3ba00640 1 -- start start 2024-02-26T10:43:14.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.430+0000 7f8f3ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 0x7f8f340c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.430+0000 7f8f3ba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8f3406e410 0x7f8f340c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.430+0000 7f8f3ba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f34152e50 con 0x7f8f34156230 2024-02-26T10:43:14.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.430+0000 7f8f3ba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f341506e0 con 0x7f8f3406e410 2024-02-26T10:43:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.430+0000 7f8f3ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34153a60 con 0x7f8f34150300 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 0x7f8f340c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 0x7f8f340c2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.67:37936/0 (socket says 172.21.15.67:37936) 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 -- 172.21.15.67:0/4051269412 learned_addr learned my addr 172.21.15.67:0/4051269412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f33e00640 1 --2- 172.21.15.67:0/4051269412 >> v2:172.21.15.82:3300/0 conn(0x7f8f3406e410 0x7f8f340c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 -- 172.21.15.67:0/4051269412 >> v2:172.21.15.82:3300/0 conn(0x7f8f3406e410 msgr2=0x7f8f340c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 --2- 172.21.15.67:0/4051269412 >> v2:172.21.15.82:3300/0 conn(0x7f8f3406e410 0x7f8f340c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 -- 172.21.15.67:0/4051269412 >> v1:172.21.15.82:6789/0 conn(0x7f8f34156230 legacy=0x7f8f340aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.431+0000 7f8f39000640 1 -- 172.21.15.67:0/4051269412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34153130 con 0x7f8f34150300 2024-02-26T10:43:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.433+0000 7f8f39000640 1 --2- 172.21.15.67:0/4051269412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 0x7f8f340c2ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009870 tx=0x7f8f2802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe5f0e4c2c036f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.434+0000 7f8f33400640 1 -- 172.21.15.67:0/4051269412 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280317c0 con 0x7f8f34150300 2024-02-26T10:43:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.434+0000 7f8f33400640 1 -- 172.21.15.67:0/4051269412 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f28037070 con 0x7f8f34150300 2024-02-26T10:43:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.434+0000 7f8f33400640 1 -- 172.21.15.67:0/4051269412 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28031b80 con 0x7f8f34150300 2024-02-26T10:43:14.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 -- 172.21.15.67:0/4051269412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 msgr2=0x7f8f340c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/4051269412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 0x7f8f340c2ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009870 tx=0x7f8f2802b860 comp rx=0 tx=0).stop 2024-02-26T10:43:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 -- 172.21.15.67:0/4051269412 shutdown_connections 2024-02-26T10:43:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/4051269412 >> v2:172.21.15.82:3300/0 conn(0x7f8f3406e410 0x7f8f340c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/4051269412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f34150300 0x7f8f340c2ff0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 -- 172.21.15.67:0/4051269412 >> 172.21.15.67:0/4051269412 conn(0x7f8f340a84a0 msgr2=0x7f8f340a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 -- 172.21.15.67:0/4051269412 shutdown_connections 2024-02-26T10:43:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.435+0000 7f8f3ba00640 1 -- 172.21.15.67:0/4051269412 wait complete. 2024-02-26T10:43:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.437+0000 7f8f3ba00640 1 Processor -- start 2024-02-26T10:43:14.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.437+0000 7f8f3ba00640 1 -- start start 2024-02-26T10:43:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f3ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3406e410 0x7f8f34151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f3ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34150300 0x7f8f3414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f3ba00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 0x7f8f3414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f3ba00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3406dd80 con 0x7f8f34156230 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f3ba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f341536f0 con 0x7f8f34150300 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f3ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340ab2b0 con 0x7f8f3406e410 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f39a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 0x7f8f3414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f39a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 0x7f8f3414bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44060/0 (socket says 172.21.15.67:44060) 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f39a00640 1 -- 172.21.15.67:0/3859220517 learned_addr learned my addr 172.21.15.67:0/3859220517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f39000640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3406e410 0x7f8f34151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.439+0000 7f8f33e00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34150300 0x7f8f3414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f39a00640 1 -- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3406e410 msgr2=0x7f8f34151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f39a00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3406e410 0x7f8f34151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f39a00640 1 -- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34150300 msgr2=0x7f8f3414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f39a00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34150300 0x7f8f3414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f39a00640 1 -- 172.21.15.67:0/3859220517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34146e20 con 0x7f8f34156230 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f39a00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 0x7f8f3414bf00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c680 tx=0x7f8f2c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c003b70 con 0x7f8f34156230 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c003d10 con 0x7f8f34156230 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c005310 con 0x7f8f34156230 2024-02-26T10:43:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.440+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f341520e0 con 0x7f8f34156230 2024-02-26T10:43:14.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.441+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f341526a0 con 0x7f8f34156230 2024-02-26T10:43:14.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.442+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f341528a0 con 0x7f8f34156230 2024-02-26T10:43:14.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.443+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f2c0054b0 con 0x7f8f34156230 2024-02-26T10:43:14.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.444+0000 7f8f30c00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f1406b4d0 0x7f8f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.444+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8f2c089d60 con 0x7f8f34156230 2024-02-26T10:43:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.445+0000 7f8f39000640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f1406b4d0 0x7f8f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.446+0000 7f8f39000640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f1406b4d0 0x7f8f1406d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f28036000 tx=0x7f8f2803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.449+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f341528a0 con 0x7f8f34156230 2024-02-26T10:43:14.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.646+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f34147100 con 0x7f8f1406b4d0 2024-02-26T10:43:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.647+0000 7f8f30c00640 1 -- 172.21.15.67:0/3859220517 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8f34147100 con 0x7f8f1406b4d0 2024-02-26T10:43:14.644 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 0 to deep-scrub 2024-02-26T10:43:14.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f1406b4d0 msgr2=0x7f8f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f1406b4d0 0x7f8f1406d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f28036000 tx=0x7f8f2803d040 comp rx=0 tx=0).stop 2024-02-26T10:43:14.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 msgr2=0x7f8f3414bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 0x7f8f3414bf00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c680 tx=0x7f8f2c00cb40 comp rx=0 tx=0).stop 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 shutdown_connections 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f34156230 0x7f8f3414bf00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f8f1406b4d0 0x7f8f1406d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f34150300 0x7f8f3414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 --2- 172.21.15.67:0/3859220517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3406e410 0x7f8f34151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 >> 172.21.15.67:0/3859220517 conn(0x7f8f340a84a0 msgr2=0x7f8f340aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 shutdown_connections 2024-02-26T10:43:14.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.650+0000 7f8f3ba00640 1 -- 172.21.15.67:0/3859220517 wait complete. 2024-02-26T10:43:14.667 INFO:tasks.ceph:Scrubbing osd.1 2024-02-26T10:43:14.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:14.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.897+0000 7f8a1ee00640 1 Processor -- start 2024-02-26T10:43:14.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.897+0000 7f8a1ee00640 1 -- start start 2024-02-26T10:43:14.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a1ee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8a181591b0 0x7f8a18158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a1ee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a180c0ef0 0x7f8a18070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a1ee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a18072ce0 con 0x7f8a18071410 2024-02-26T10:43:14.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a1ee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a18073010 con 0x7f8a180c0ef0 2024-02-26T10:43:14.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a1ee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1814f490 con 0x7f8a181591b0 2024-02-26T10:43:14.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a17400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a180c0ef0 0x7f8a18070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a17400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a180c0ef0 0x7f8a18070ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44062/0 (socket says 172.21.15.67:44062) 2024-02-26T10:43:14.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a17400640 1 -- 172.21.15.67:0/1497892542 learned_addr learned my addr 172.21.15.67:0/1497892542 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:14.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a17e00640 1 --2- 172.21.15.67:0/1497892542 >> v2:172.21.15.82:3300/0 conn(0x7f8a181591b0 0x7f8a18158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.899+0000 7f8a17400640 1 -- 172.21.15.67:0/1497892542 >> v2:172.21.15.82:3300/0 conn(0x7f8a181591b0 msgr2=0x7f8a18158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:14.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.900+0000 7f8a17400640 1 --2- 172.21.15.67:0/1497892542 >> v2:172.21.15.82:3300/0 conn(0x7f8a181591b0 0x7f8a18158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.900+0000 7f8a17400640 1 -- 172.21.15.67:0/1497892542 >> v1:172.21.15.82:6789/0 conn(0x7f8a18071410 legacy=0x7f8a1806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:14.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.900+0000 7f8a17400640 1 -- 172.21.15.67:0/1497892542 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a18155bf0 con 0x7f8a180c0ef0 2024-02-26T10:43:14.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.901+0000 7f8a17400640 1 --2- 172.21.15.67:0/1497892542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a180c0ef0 0x7f8a18070ed0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8a180c3bf0 tx=0x7f8a0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34063ffebc8eb560 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.901+0000 7f8a16a00640 1 -- 172.21.15.67:0/1497892542 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0002e070 con 0x7f8a180c0ef0 2024-02-26T10:43:14.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.901+0000 7f8a16a00640 1 -- 172.21.15.67:0/1497892542 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00037070 con 0x7f8a180c0ef0 2024-02-26T10:43:14.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.901+0000 7f8a16a00640 1 -- 172.21.15.67:0/1497892542 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00031a00 con 0x7f8a180c0ef0 2024-02-26T10:43:14.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 -- 172.21.15.67:0/1497892542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a180c0ef0 msgr2=0x7f8a18070ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 --2- 172.21.15.67:0/1497892542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a180c0ef0 0x7f8a18070ed0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8a180c3bf0 tx=0x7f8a0002b880 comp rx=0 tx=0).stop 2024-02-26T10:43:14.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 -- 172.21.15.67:0/1497892542 shutdown_connections 2024-02-26T10:43:14.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 --2- 172.21.15.67:0/1497892542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a180c0ef0 0x7f8a18070ed0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 --2- 172.21.15.67:0/1497892542 >> v2:172.21.15.82:3300/0 conn(0x7f8a181591b0 0x7f8a18158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 -- 172.21.15.67:0/1497892542 >> 172.21.15.67:0/1497892542 conn(0x7f8a180c6da0 msgr2=0x7f8a180c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 -- 172.21.15.67:0/1497892542 shutdown_connections 2024-02-26T10:43:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.902+0000 7f8a1ee00640 1 -- 172.21.15.67:0/1497892542 wait complete. 2024-02-26T10:43:14.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.903+0000 7f8a1ee00640 1 Processor -- start 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 -- start start 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a18071410 0x7f8a181de860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a180c0ef0 0x7f8a18151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a181591b0 0x7f8a18151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1806e6e0 con 0x7f8a181591b0 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1806e560 con 0x7f8a180c0ef0 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.904+0000 7f8a1ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180a8ba0 con 0x7f8a18071410 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a181591b0 0x7f8a18151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a181591b0 0x7f8a18151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44068/0 (socket says 172.21.15.67:44068) 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 -- 172.21.15.67:0/3751066650 learned_addr learned my addr 172.21.15.67:0/3751066650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a17e00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a18071410 0x7f8a181de860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a17400640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a180c0ef0 0x7f8a18151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 -- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a18071410 msgr2=0x7f8a181de860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8a18071410 0x7f8a181de860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 -- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a180c0ef0 msgr2=0x7f8a18151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a180c0ef0 0x7f8a18151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a1ce00640 1 -- 172.21.15.67:0/3751066650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a181eb820 con 0x7f8a181591b0 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.905+0000 7f8a17400640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a180c0ef0 0x7f8a18151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.906+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a181591b0 0x7f8a18151e60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000c700 tx=0x7f8a1000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.906+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10016020 con 0x7f8a181591b0 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.906+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1000aa50 con 0x7f8a181591b0 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.906+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000ae40 con 0x7f8a181591b0 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.906+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a18150740 con 0x7f8a181591b0 2024-02-26T10:43:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.906+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181deda0 con 0x7f8a181591b0 2024-02-26T10:43:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.907+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a18074a30 con 0x7f8a181591b0 2024-02-26T10:43:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.908+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a10019020 con 0x7f8a181591b0 2024-02-26T10:43:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.909+0000 7f8a0fe00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f89f406b4d0 0x7f89f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.909+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8a10089f50 con 0x7f8a181591b0 2024-02-26T10:43:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.909+0000 7f8a0fe00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7f89f4071430 0x7f89f4073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.909+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 --> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89f4073f00 con 0x7f89f4071430 2024-02-26T10:43:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.909+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1008c510 con 0x7f8a181591b0 2024-02-26T10:43:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.909+0000 7f8a17400640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7f89f4071430 0x7f89f4073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.910+0000 7f8a17e00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f89f406b4d0 0x7f89f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.910+0000 7f8a17400640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7f89f4071430 0x7f89f4073850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.911+0000 7f8a17e00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f89f406b4d0 0x7f89f406d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8a08006fd0 tx=0x7f8a08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:14.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.911+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== osd.1 v2:172.21.15.67:6817/1684785661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f89f4073f00 con 0x7f89f4071430 2024-02-26T10:43:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.925+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 --> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8a1806eb10 con 0x7f89f4071430 2024-02-26T10:43:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a0fe00640 1 -- 172.21.15.67:0/3751066650 <== osd.1 v2:172.21.15.67:6817/1684785661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8a1806eb10 con 0x7f89f4071430 2024-02-26T10:43:14.923 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:14.923 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:14.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7f89f4071430 msgr2=0x7f89f4073850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:6817/1684785661,v1:172.21.15.67:6819/1684785661] conn(0x7f89f4071430 0x7f89f4073850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f89f406b4d0 msgr2=0x7f89f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f89f406b4d0 0x7f89f406d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8a08006fd0 tx=0x7f8a08008040 comp rx=0 tx=0).stop 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a181591b0 msgr2=0x7f8a18151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 --2- 172.21.15.67:0/3751066650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a181591b0 0x7f8a18151e60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000c700 tx=0x7f8a1000cbc0 comp rx=0 tx=0).stop 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ce00640 1 -- 172.21.15.67:0/3751066650 reap_dead start 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 shutdown_connections 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.926+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 >> 172.21.15.67:0/3751066650 conn(0x7f8a180c6da0 msgr2=0x7f8a180c1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.927+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 shutdown_connections 2024-02-26T10:43:14.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:14.927+0000 7f8a1ee00640 1 -- 172.21.15.67:0/3751066650 wait complete. 2024-02-26T10:43:14.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-02-26T10:43:15.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.152+0000 7ff5da000640 1 Processor -- start 2024-02-26T10:43:15.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.152+0000 7ff5da000640 1 -- start start 2024-02-26T10:43:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5da000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff5d4150280 0x7ff5d40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5da000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40a5f50 con 0x7ff5d4152c60 2024-02-26T10:43:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5da000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d4156e00 con 0x7ff5d40718e0 2024-02-26T10:43:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5da000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40c1a60 con 0x7ff5d4150280 2024-02-26T10:43:15.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5d3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff5d4150280 0x7ff5d40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5d3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff5d4150280 0x7ff5d40713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44074/0 (socket says 172.21.15.67:44074) 2024-02-26T10:43:15.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5d3400640 1 -- 172.21.15.67:0/1620760465 learned_addr learned my addr 172.21.15.67:0/1620760465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:15.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.154+0000 7ff5caa00640 1 --1- 172.21.15.67:0/1620760465 >> v1:172.21.15.67:6789/0 conn(0x7ff5d4152c60 0x7ff5d40c37e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43548/0 (socket says 172.21.15.67:43548) 2024-02-26T10:43:15.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.155+0000 7ff5d3400640 1 -- 172.21.15.67:0/1620760465 >> v1:172.21.15.82:6789/0 conn(0x7ff5d40718e0 legacy=0x7ff5d415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:15.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.155+0000 7ff5d3400640 1 -- 172.21.15.67:0/1620760465 >> v1:172.21.15.67:6789/0 conn(0x7ff5d4152c60 legacy=0x7ff5d40c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.155+0000 7ff5d3400640 1 -- 172.21.15.67:0/1620760465 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4150660 con 0x7ff5d4150280 2024-02-26T10:43:15.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.155+0000 7ff5d2a00640 1 -- 172.21.15.67:0/1620760465 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88330753 0 0) 0x7ff5d40a5f50 con 0x7ff5d4152c60 2024-02-26T10:43:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.156+0000 7ff5d3400640 1 --2- 172.21.15.67:0/1620760465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d4150280 0x7ff5d40713a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009e50 tx=0x7ff5c402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97ba8ed05337aab server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.156+0000 7ff5d2a00640 1 -- 172.21.15.67:0/1620760465 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c402c9f0 con 0x7ff5d4150280 2024-02-26T10:43:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.156+0000 7ff5d2a00640 1 -- 172.21.15.67:0/1620760465 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c402cb90 con 0x7ff5d4150280 2024-02-26T10:43:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.156+0000 7ff5d2a00640 1 -- 172.21.15.67:0/1620760465 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40343f0 con 0x7ff5d4150280 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 -- 172.21.15.67:0/1620760465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d4150280 msgr2=0x7ff5d40713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 --2- 172.21.15.67:0/1620760465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d4150280 0x7ff5d40713a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009e50 tx=0x7ff5c402be50 comp rx=0 tx=0).stop 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 -- 172.21.15.67:0/1620760465 shutdown_connections 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 --2- 172.21.15.67:0/1620760465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d4150280 0x7ff5d40713a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 -- 172.21.15.67:0/1620760465 >> 172.21.15.67:0/1620760465 conn(0x7ff5d40c6e90 msgr2=0x7ff5d40c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 -- 172.21.15.67:0/1620760465 shutdown_connections 2024-02-26T10:43:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.157+0000 7ff5da000640 1 -- 172.21.15.67:0/1620760465 wait complete. 2024-02-26T10:43:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.159+0000 7ff5da000640 1 Processor -- start 2024-02-26T10:43:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.159+0000 7ff5da000640 1 -- start start 2024-02-26T10:43:15.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5da000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 0x7ff5d40aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5da000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5d4150280 0x7ff5d41edf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5da000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5d4152c60 0x7ff5d414cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5da000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c3ef0 con 0x7ff5d40718e0 2024-02-26T10:43:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5da000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c15d0 con 0x7ff5d4152c60 2024-02-26T10:43:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5da000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c4070 con 0x7ff5d4150280 2024-02-26T10:43:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 0x7ff5d40aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 0x7ff5d40aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44078/0 (socket says 172.21.15.67:44078) 2024-02-26T10:43:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3400640 1 -- 172.21.15.67:0/2341789931 learned_addr learned my addr 172.21.15.67:0/2341789931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5caa00640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5d4150280 0x7ff5d41edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3400640 1 -- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5d4150280 msgr2=0x7ff5d41edf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3400640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5d4150280 0x7ff5d41edf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3400640 1 -- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5d4152c60 msgr2=0x7ff5d414cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.160+0000 7ff5d3e00640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5d4152c60 0x7ff5d414cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5d3400640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5d4152c60 0x7ff5d414cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5d3400640 1 -- 172.21.15.67:0/2341789931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40c0a30 con 0x7ff5d40718e0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5d3e00640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5d4152c60 0x7ff5d414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5d3400640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 0x7ff5d40aca00 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4036040 tx=0x7ff5c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4044020 con 0x7ff5d40718e0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4040470 con 0x7ff5d40718e0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d41eea10 con 0x7ff5d40718e0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4040860 con 0x7ff5d40718e0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.161+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d40a86b0 con 0x7ff5d40718e0 2024-02-26T10:43:15.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.163+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d40c7270 con 0x7ff5d40718e0 2024-02-26T10:43:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.163+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5c4047020 con 0x7ff5d40718e0 2024-02-26T10:43:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.164+0000 7ff5cbe00640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff5b006b4d0 0x7ff5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.164+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff5c40acba0 con 0x7ff5d40718e0 2024-02-26T10:43:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.165+0000 7ff5caa00640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff5b006b4d0 0x7ff5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.166+0000 7ff5caa00640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff5b006b4d0 0x7ff5b006d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc007fd0 tx=0x7ff5bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.169+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5d40c7270 con 0x7ff5d40718e0 2024-02-26T10:43:15.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.369+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5d40a9860 con 0x7ff5b006b4d0 2024-02-26T10:43:15.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.370+0000 7ff5cbe00640 1 -- 172.21.15.67:0/2341789931 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff5d40a9860 con 0x7ff5b006b4d0 2024-02-26T10:43:15.367 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 1 to deep-scrub 2024-02-26T10:43:15.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.372+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff5b006b4d0 msgr2=0x7ff5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.372+0000 7ff5da000640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff5b006b4d0 0x7ff5b006d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc007fd0 tx=0x7ff5bc009040 comp rx=0 tx=0).stop 2024-02-26T10:43:15.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.372+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 msgr2=0x7ff5d40aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.372+0000 7ff5da000640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 0x7ff5d40aca00 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4036040 tx=0x7ff5c4004a40 comp rx=0 tx=0).stop 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 shutdown_connections 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7ff5b006b4d0 0x7ff5b006d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5d4152c60 0x7ff5d414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5d4150280 0x7ff5d41edf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 --2- 172.21.15.67:0/2341789931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5d40718e0 0x7ff5d40aca00 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 >> 172.21.15.67:0/2341789931 conn(0x7ff5d40c6e90 msgr2=0x7ff5d4152000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 shutdown_connections 2024-02-26T10:43:15.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.373+0000 7ff5da000640 1 -- 172.21.15.67:0/2341789931 wait complete. 2024-02-26T10:43:15.392 INFO:tasks.ceph:Scrubbing osd.2 2024-02-26T10:43:15.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:15.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.621+0000 7f9a4f200640 1 Processor -- start 2024-02-26T10:43:15.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.621+0000 7f9a4f200640 1 -- start start 2024-02-26T10:43:15.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.622+0000 7f9a4f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a500c78b0 0x7f9a50155940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.622+0000 7f9a4f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a50158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.622+0000 7f9a4f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500c59f0 con 0x7f9a501590b0 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.622+0000 7f9a4f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5006d670 con 0x7f9a500c78b0 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.622+0000 7f9a4f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500c5170 con 0x7f9a500c8570 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a500c78b0 0x7f9a50155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a500c78b0 0x7f9a50155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44094/0 (socket says 172.21.15.67:44094) 2024-02-26T10:43:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 -- 172.21.15.67:0/861113859 learned_addr learned my addr 172.21.15.67:0/861113859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4d400640 1 --2- 172.21.15.67:0/861113859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a50158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 -- 172.21.15.67:0/861113859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 msgr2=0x7f9a50158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 --2- 172.21.15.67:0/861113859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a50158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 -- 172.21.15.67:0/861113859 >> v1:172.21.15.82:6789/0 conn(0x7f9a501590b0 legacy=0x7f9a50151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.623+0000 7f9a4de00640 1 -- 172.21.15.67:0/861113859 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500c5430 con 0x7f9a500c78b0 2024-02-26T10:43:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.624+0000 7f9a4de00640 1 --2- 172.21.15.67:0/861113859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a500c78b0 0x7f9a50155940 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009e50 tx=0x7f9a4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460ca6c2b939406 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.625+0000 7f9a4ca00640 1 -- 172.21.15.67:0/861113859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4002ca70 con 0x7f9a500c78b0 2024-02-26T10:43:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.625+0000 7f9a4ca00640 1 -- 172.21.15.67:0/861113859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4002cc10 con 0x7f9a500c78b0 2024-02-26T10:43:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.625+0000 7f9a4ca00640 1 -- 172.21.15.67:0/861113859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a400343f0 con 0x7f9a500c78b0 2024-02-26T10:43:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.625+0000 7f9a4f200640 1 -- 172.21.15.67:0/861113859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a500c78b0 msgr2=0x7f9a50155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.625+0000 7f9a4f200640 1 --2- 172.21.15.67:0/861113859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a500c78b0 0x7f9a50155940 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009e50 tx=0x7f9a4002bed0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.626+0000 7f9a4f200640 1 -- 172.21.15.67:0/861113859 shutdown_connections 2024-02-26T10:43:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.626+0000 7f9a4f200640 1 --2- 172.21.15.67:0/861113859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a50158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.626+0000 7f9a4f200640 1 --2- 172.21.15.67:0/861113859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a500c78b0 0x7f9a50155940 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.626+0000 7f9a4f200640 1 -- 172.21.15.67:0/861113859 >> 172.21.15.67:0/861113859 conn(0x7f9a50072770 msgr2=0x7f9a50072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.626+0000 7f9a4f200640 1 -- 172.21.15.67:0/861113859 shutdown_connections 2024-02-26T10:43:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.626+0000 7f9a4f200640 1 -- 172.21.15.67:0/861113859 wait complete. 2024-02-26T10:43:15.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.627+0000 7f9a4f200640 1 Processor -- start 2024-02-26T10:43:15.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 -- start start 2024-02-26T10:43:15.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a500c78b0 0x7f9a500af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a500aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a501590b0 0x7f9a500aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5006da80 con 0x7f9a501590b0 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50158650 con 0x7f9a500c78b0 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.628+0000 7f9a4f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5006dc00 con 0x7f9a500c8570 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a501590b0 0x7f9a500aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a501590b0 0x7f9a500aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44098/0 (socket says 172.21.15.67:44098) 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 -- 172.21.15.67:0/3718072496 learned_addr learned my addr 172.21.15.67:0/3718072496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4d400640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a500aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4de00640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a500c78b0 0x7f9a500af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 -- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 msgr2=0x7f9a500aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a500c8570 0x7f9a500aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 -- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a500c78b0 msgr2=0x7f9a500af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a500c78b0 0x7f9a500af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4e800640 1 -- 172.21.15.67:0/3718072496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500ac100 con 0x7f9a501590b0 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.629+0000 7f9a4de00640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a500c78b0 0x7f9a500af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.630+0000 7f9a4e800640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a501590b0 0x7f9a500aa590 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b440 tx=0x7f9a4800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.630+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48015020 con 0x7f9a501590b0 2024-02-26T10:43:15.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.630+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a48011830 con 0x7f9a501590b0 2024-02-26T10:43:15.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.630+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48011c20 con 0x7f9a501590b0 2024-02-26T10:43:15.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.630+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a500aaad0 con 0x7f9a501590b0 2024-02-26T10:43:15.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.630+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a500ac3a0 con 0x7f9a501590b0 2024-02-26T10:43:15.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.631+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a50074a30 con 0x7f9a501590b0 2024-02-26T10:43:15.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.632+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a48018020 con 0x7f9a501590b0 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.633+0000 7f9a46000640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9a2c06b500 0x7f9a2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.633+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9a48089c30 con 0x7f9a501590b0 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.633+0000 7f9a46000640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f9a2c071460 0x7f9a2c073880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.633+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 --> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a48007e50 con 0x7f9a2c071460 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.633+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a4808e2a0 con 0x7f9a501590b0 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.634+0000 7f9a4d400640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f9a2c071460 0x7f9a2c073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.634+0000 7f9a4de00640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9a2c06b500 0x7f9a2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.634+0000 7f9a4d400640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f9a2c071460 0x7f9a2c073880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.635+0000 7f9a4de00640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9a2c06b500 0x7f9a2c06d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9a40035000 tx=0x7f9a40030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.635+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== osd.2 v2:172.21.15.67:6811/2562492504 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9a48007e50 con 0x7f9a2c071460 2024-02-26T10:43:15.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.650+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 --> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a5006d860 con 0x7f9a2c071460 2024-02-26T10:43:15.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.650+0000 7f9a46000640 1 -- 172.21.15.67:0/3718072496 <== osd.2 v2:172.21.15.67:6811/2562492504 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9a5006d860 con 0x7f9a2c071460 2024-02-26T10:43:15.648 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:15.649 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:15.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f9a2c071460 msgr2=0x7f9a2c073880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:6811/2562492504,v1:172.21.15.67:6812/2562492504] conn(0x7f9a2c071460 0x7f9a2c073880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9a2c06b500 msgr2=0x7f9a2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f9a2c06b500 0x7f9a2c06d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9a40035000 tx=0x7f9a40030000 comp rx=0 tx=0).stop 2024-02-26T10:43:15.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a501590b0 msgr2=0x7f9a500aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 --2- 172.21.15.67:0/3718072496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a501590b0 0x7f9a500aa590 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b440 tx=0x7f9a4800b900 comp rx=0 tx=0).stop 2024-02-26T10:43:15.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4e800640 1 -- 172.21.15.67:0/3718072496 reap_dead start 2024-02-26T10:43:15.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 shutdown_connections 2024-02-26T10:43:15.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.651+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 >> 172.21.15.67:0/3718072496 conn(0x7f9a50072770 msgr2=0x7f9a501514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.652+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 shutdown_connections 2024-02-26T10:43:15.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.652+0000 7f9a4f200640 1 -- 172.21.15.67:0/3718072496 wait complete. 2024-02-26T10:43:15.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-02-26T10:43:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.893+0000 7fb4e9000640 1 Processor -- start 2024-02-26T10:43:15.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.893+0000 7fb4e9000640 1 -- start start 2024-02-26T10:43:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.894+0000 7fb4e9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e40c5820 0x7fb4e4155a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.894+0000 7fb4e9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4e414ebd0 0x7fb4e414f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.894+0000 7fb4e9000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e4155fb0 con 0x7fb4e40c3730 2024-02-26T10:43:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.894+0000 7fb4e9000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e40c7100 con 0x7fb4e414ebd0 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.894+0000 7fb4e9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4158810 con 0x7fb4e40c5820 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4e414ebd0 0x7fb4e414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4e414ebd0 0x7fb4e414f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44104/0 (socket says 172.21.15.67:44104) 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 -- 172.21.15.67:0/1628752152 learned_addr learned my addr 172.21.15.67:0/1628752152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e1e00640 1 --2- 172.21.15.67:0/1628752152 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e40c5820 0x7fb4e4155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 -- 172.21.15.67:0/1628752152 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e40c5820 msgr2=0x7fb4e4155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 --2- 172.21.15.67:0/1628752152 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e40c5820 0x7fb4e4155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 -- 172.21.15.67:0/1628752152 >> v1:172.21.15.82:6789/0 conn(0x7fb4e40c3730 legacy=0x7fb4e4158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.895+0000 7fb4e2800640 1 -- 172.21.15.67:0/1628752152 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40c6860 con 0x7fb4e414ebd0 2024-02-26T10:43:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.896+0000 7fb4e2800640 1 --2- 172.21.15.67:0/1628752152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e414ebd0 0x7fb4e414f000 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e50 tx=0x7fb4dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e69377a61cf82d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.896+0000 7fb4e0a00640 1 -- 172.21.15.67:0/1628752152 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc02ca70 con 0x7fb4e414ebd0 2024-02-26T10:43:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e0a00640 1 -- 172.21.15.67:0/1628752152 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc02cc10 con 0x7fb4e414ebd0 2024-02-26T10:43:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e0a00640 1 -- 172.21.15.67:0/1628752152 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0343f0 con 0x7fb4e414ebd0 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 -- 172.21.15.67:0/1628752152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e414ebd0 msgr2=0x7fb4e414f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 --2- 172.21.15.67:0/1628752152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e414ebd0 0x7fb4e414f000 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e50 tx=0x7fb4dc02bed0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 -- 172.21.15.67:0/1628752152 shutdown_connections 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 --2- 172.21.15.67:0/1628752152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e414ebd0 0x7fb4e414f000 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 --2- 172.21.15.67:0/1628752152 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e40c5820 0x7fb4e4155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 -- 172.21.15.67:0/1628752152 >> 172.21.15.67:0/1628752152 conn(0x7fb4e406b040 msgr2=0x7fb4e406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 -- 172.21.15.67:0/1628752152 shutdown_connections 2024-02-26T10:43:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.897+0000 7fb4e9000640 1 -- 172.21.15.67:0/1628752152 wait complete. 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.899+0000 7fb4e9000640 1 Processor -- start 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.899+0000 7fb4e9000640 1 -- start start 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 0x7fb4e4157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e40c5820 0x7fb4e41532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e414ebd0 0x7fb4e4154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e9000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c6530 con 0x7fb4e40c3730 2024-02-26T10:43:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c6e20 con 0x7fb4e40c5820 2024-02-26T10:43:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40a7fc0 con 0x7fb4e414ebd0 2024-02-26T10:43:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 0x7fb4e4157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 0x7fb4e4157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44120/0 (socket says 172.21.15.67:44120) 2024-02-26T10:43:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.900+0000 7fb4e1e00640 1 -- 172.21.15.67:0/544889402 learned_addr learned my addr 172.21.15.67:0/544889402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e1e00640 1 -- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e414ebd0 msgr2=0x7fb4e4154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e1e00640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e414ebd0 0x7fb4e4154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e1e00640 1 -- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e40c5820 msgr2=0x7fb4e41532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e1e00640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e40c5820 0x7fb4e41532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e1e00640 1 -- 172.21.15.67:0/544889402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e41584a0 con 0x7fb4e40c3730 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e1e00640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 0x7fb4e4157f60 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c7f0 tx=0x7fb4d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4015070 con 0x7fb4e40c3730 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4004240 con 0x7fb4e40c3730 2024-02-26T10:43:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4007620 con 0x7fb4e40c3730 2024-02-26T10:43:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.901+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e41537f0 con 0x7fb4e40c3730 2024-02-26T10:43:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.902+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e4157770 con 0x7fb4e40c3730 2024-02-26T10:43:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.903+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e4074a30 con 0x7fb4e40c3730 2024-02-26T10:43:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.904+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4d40079d0 con 0x7fb4e40c3730 2024-02-26T10:43:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.905+0000 7fb4da000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb4c406b4d0 0x7fb4c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.905+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb4d4089fd0 con 0x7fb4e40c3730 2024-02-26T10:43:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.905+0000 7fb4e1400640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb4c406b4d0 0x7fb4c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:15.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.906+0000 7fb4e1400640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb4c406b4d0 0x7fb4c406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0006fd0 tx=0x7fb4d0006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:15.909+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4d40590e0 con 0x7fb4e40c3730 2024-02-26T10:43:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.105+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4e40c3010 con 0x7fb4c406b4d0 2024-02-26T10:43:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.106+0000 7fb4da000640 1 -- 172.21.15.67:0/544889402 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb4e40c3010 con 0x7fb4c406b4d0 2024-02-26T10:43:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 2 to deep-scrub 2024-02-26T10:43:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb4c406b4d0 msgr2=0x7fb4c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb4c406b4d0 0x7fb4c406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0006fd0 tx=0x7fb4d0006bc0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 msgr2=0x7fb4e4157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 0x7fb4e4157f60 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c7f0 tx=0x7fb4d400ccb0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 shutdown_connections 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4e414ebd0 0x7fb4e4154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fb4c406b4d0 0x7fb4c406d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e40c5820 0x7fb4e41532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 --2- 172.21.15.67:0/544889402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e40c3730 0x7fb4e4157f60 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 >> 172.21.15.67:0/544889402 conn(0x7fb4e406b040 msgr2=0x7fb4e4158ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 shutdown_connections 2024-02-26T10:43:16.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.109+0000 7fb4e9000640 1 -- 172.21.15.67:0/544889402 wait complete. 2024-02-26T10:43:16.127 INFO:tasks.ceph:Scrubbing osd.3 2024-02-26T10:43:16.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.355+0000 7fd3d3e00640 1 Processor -- start 2024-02-26T10:43:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.355+0000 7fd3d3e00640 1 -- start start 2024-02-26T10:43:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 0x7fd3d40c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d406e9e0 con 0x7fd3d41503a0 2024-02-26T10:43:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d4150780 con 0x7fd3d40a9430 2024-02-26T10:43:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40ab8d0 con 0x7fd3d40ab480 2024-02-26T10:43:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd3d41503a0 0x7fd3d41535a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43554/0 (socket says 172.21.15.67:43554) 2024-02-26T10:43:16.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d2a00640 1 -- 172.21.15.67:0/809443042 learned_addr learned my addr 172.21.15.67:0/809443042 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:16.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d3400640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 0x7fd3d40c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973604374 0 0) 0x7fd3d406e9e0 con 0x7fd3d41503a0 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.357+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b8003520 con 0x7fd3d41503a0 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105786203 0 0) 0x7fd3b8003520 con 0x7fd3d41503a0 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 msgr2=0x7fd3d40c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 0x7fd3d40c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 >> v1:172.21.15.82:6789/0 conn(0x7fd3d40a9430 legacy=0x7fd3d406e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40a98c0 con 0x7fd3d41503a0 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fd3c40019f0 con 0x7fd3d41503a0 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 >> v1:172.21.15.67:6789/0 conn(0x7fd3d41503a0 legacy=0x7fd3d41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b80039e0 0x7fd3b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 0x7fd3b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8004d40 0x7fd3b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40ab8d0 con 0x7fd3b8004d40 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800baa0 con 0x7fd3b80039e0 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800bca0 con 0x7fd3d40ab480 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.358+0000 7fd3d3400640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8004d40 0x7fd3b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d2000640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 0x7fd3b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d2a00640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b80039e0 0x7fd3b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d3400640 1 -- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 msgr2=0x7fd3b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d3400640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d40ab480 0x7fd3b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d3400640 1 -- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b80039e0 msgr2=0x7fd3b8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d3400640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b80039e0 0x7fd3b8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d3400640 1 -- 172.21.15.67:0/809443042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8003520 con 0x7fd3b8004d40 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.359+0000 7fd3d2a00640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b80039e0 0x7fd3b8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.360+0000 7fd3d3400640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8004d40 0x7fd3b80051c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009a90 tx=0x7fd3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d13ce63aa584b64 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.360+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc039070 con 0x7fd3b8004d40 2024-02-26T10:43:16.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.360+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc02c680 con 0x7fd3b8004d40 2024-02-26T10:43:16.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.360+0000 7fd3d1600640 1 -- 172.21.15.67:0/809443042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc009690 con 0x7fd3b8004d40 2024-02-26T10:43:16.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3e00640 1 -- 172.21.15.67:0/809443042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8004d40 msgr2=0x7fd3b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3e00640 1 --2- 172.21.15.67:0/809443042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8004d40 0x7fd3b80051c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009a90 tx=0x7fd3b8005830 comp rx=0 tx=0).stop 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3400640 1 -- 172.21.15.67:0/809443042 reap_dead start 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3e00640 1 -- 172.21.15.67:0/809443042 shutdown_connections 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3e00640 1 -- 172.21.15.67:0/809443042 >> 172.21.15.67:0/809443042 conn(0x7fd3d40c6380 msgr2=0x7fd3d40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3e00640 1 -- 172.21.15.67:0/809443042 shutdown_connections 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.361+0000 7fd3d3e00640 1 -- 172.21.15.67:0/809443042 wait complete. 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.363+0000 7fd3d3e00640 1 Processor -- start 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.363+0000 7fd3d3e00640 1 -- start start 2024-02-26T10:43:16.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d40 0x7fd3d41ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3d4156880 0x7fd3d40c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d41de7d0 0x7fd3d41dc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d406dad0 con 0x7fd3d4156880 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40a9e80 con 0x7fd3b8004d40 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d406ddb0 con 0x7fd3d41de7d0 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3d4156880 0x7fd3d40c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3d4156880 0x7fd3d40c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44150/0 (socket says 172.21.15.67:44150) 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.364+0000 7fd3d2000640 1 -- 172.21.15.67:0/605852064 learned_addr learned my addr 172.21.15.67:0/605852064 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:16.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d3400640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d41de7d0 0x7fd3d41dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2000640 1 -- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d41de7d0 msgr2=0x7fd3d41dc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2a00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d40 0x7fd3d41ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2000640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3d41de7d0 0x7fd3d41dc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2000640 1 -- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d40 msgr2=0x7fd3d41ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2000640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d40 0x7fd3d41ecd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2000640 1 -- 172.21.15.67:0/605852064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4157040 con 0x7fd3d4156880 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2a00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d40 0x7fd3d41ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:16.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d2000640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3d4156880 0x7fd3d40c5360 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e550 tx=0x7fd3bc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0052e0 con 0x7fd3d4156880 2024-02-26T10:43:16.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc018070 con 0x7fd3d4156880 2024-02-26T10:43:16.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc003810 con 0x7fd3d4156880 2024-02-26T10:43:16.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.365+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d4156d60 con 0x7fd3d4156880 2024-02-26T10:43:16.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.366+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d41ec570 con 0x7fd3d4156880 2024-02-26T10:43:16.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.366+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3d40c0f80 con 0x7fd3d4156880 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.368+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3bc0039b0 con 0x7fd3d4156880 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.368+0000 7fd3cb400640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3b006b500 0x7fd3b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.369+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd3bc089c30 con 0x7fd3d4156880 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.369+0000 7fd3cb400640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fd3b0071460 0x7fd3b0073880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.369+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 --> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3bc007630 con 0x7fd3b0071460 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.369+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3bc08e2a0 con 0x7fd3d4156880 2024-02-26T10:43:16.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.369+0000 7fd3d3400640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fd3b0071460 0x7fd3b0073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.369+0000 7fd3d2a00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3b006b500 0x7fd3b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.370+0000 7fd3d3400640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fd3b0071460 0x7fd3b0073880 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.370+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== osd.3 v2:172.21.15.67:6808/755501566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd3bc007630 con 0x7fd3b0071460 2024-02-26T10:43:16.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.371+0000 7fd3d2a00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3b006b500 0x7fd3b006d9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4007c40 tx=0x7fd3c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.385+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 --> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd3d4074a30 con 0x7fd3b0071460 2024-02-26T10:43:16.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.385+0000 7fd3cb400640 1 -- 172.21.15.67:0/605852064 <== osd.3 v2:172.21.15.67:6808/755501566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd3d4074a30 con 0x7fd3b0071460 2024-02-26T10:43:16.383 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:16.383 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:16.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fd3b0071460 msgr2=0x7fd3b0073880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:6808/755501566,v1:172.21.15.67:6809/755501566] conn(0x7fd3b0071460 0x7fd3b0073880 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3b006b500 msgr2=0x7fd3b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fd3b006b500 0x7fd3b006d9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4007c40 tx=0x7fd3c4009040 comp rx=0 tx=0).stop 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3d4156880 msgr2=0x7fd3d40c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 --2- 172.21.15.67:0/605852064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3d4156880 0x7fd3d40c5360 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e550 tx=0x7fd3bc00ea10 comp rx=0 tx=0).stop 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3400640 1 -- 172.21.15.67:0/605852064 reap_dead start 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 shutdown_connections 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 >> 172.21.15.67:0/605852064 conn(0x7fd3d40c6380 msgr2=0x7fd3d40c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 shutdown_connections 2024-02-26T10:43:16.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.386+0000 7fd3d3e00640 1 -- 172.21.15.67:0/605852064 wait complete. 2024-02-26T10:43:16.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-02-26T10:43:16.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.630+0000 7fdd7ea00640 1 Processor -- start 2024-02-26T10:43:16.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.630+0000 7fdd7ea00640 1 -- start start 2024-02-26T10:43:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd78153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd7806b000 0x7fdd781524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7ea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd78155f10 con 0x7fdd7806ac20 2024-02-26T10:43:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7ea00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7806b470 con 0x7fdd7806b000 2024-02-26T10:43:16.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780c28d0 con 0x7fdd780c2310 2024-02-26T10:43:16.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7d600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdd7806ac20 0x7fdd78152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43560/0 (socket says 172.21.15.67:43560) 2024-02-26T10:43:16.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7d600640 1 -- 172.21.15.67:0/3700794294 learned_addr learned my addr 172.21.15.67:0/3700794294 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:16.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7e000640 1 --2- 172.21.15.67:0/3700794294 >> v2:172.21.15.67:3300/0 conn(0x7fdd7806b000 0x7fdd781524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.632+0000 7fdd7cc00640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd78153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd77e00640 1 -- 172.21.15.67:0/3700794294 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473045815 0 0) 0x7fdd78155f10 con 0x7fdd7806ac20 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd77e00640 1 -- 172.21.15.67:0/3700794294 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd60005730 con 0x7fdd7806ac20 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd7e000640 1 -- 172.21.15.67:0/3700794294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 msgr2=0x7fdd78153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd7e000640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd78153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd7e000640 1 -- 172.21.15.67:0/3700794294 >> v1:172.21.15.67:6789/0 conn(0x7fdd7806ac20 legacy=0x7fdd78152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd7e000640 1 -- 172.21.15.67:0/3700794294 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78155150 con 0x7fdd7806b000 2024-02-26T10:43:16.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.633+0000 7fdd7cc00640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd78153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-26T10:43:16.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.634+0000 7fdd7e000640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806b000 0x7fdd781524b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009d90 tx=0x7fdd7002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf4a0da630a12d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.634+0000 7fdd77e00640 1 -- 172.21.15.67:0/3700794294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70039070 con 0x7fdd7806b000 2024-02-26T10:43:16.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.634+0000 7fdd77e00640 1 -- 172.21.15.67:0/3700794294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7002cb30 con 0x7fdd7806b000 2024-02-26T10:43:16.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.634+0000 7fdd77e00640 1 -- 172.21.15.67:0/3700794294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70034440 con 0x7fdd7806b000 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 -- 172.21.15.67:0/3700794294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806b000 msgr2=0x7fdd781524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806b000 0x7fdd781524b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009d90 tx=0x7fdd7002bd80 comp rx=0 tx=0).stop 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 -- 172.21.15.67:0/3700794294 shutdown_connections 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806b000 0x7fdd781524b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/3700794294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd78153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 -- 172.21.15.67:0/3700794294 >> 172.21.15.67:0/3700794294 conn(0x7fdd780c63f0 msgr2=0x7fdd780c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 -- 172.21.15.67:0/3700794294 shutdown_connections 2024-02-26T10:43:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.635+0000 7fdd7ea00640 1 -- 172.21.15.67:0/3700794294 wait complete. 2024-02-26T10:43:16.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.637+0000 7fdd7ea00640 1 Processor -- start 2024-02-26T10:43:16.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.637+0000 7fdd7ea00640 1 -- start start 2024-02-26T10:43:16.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 0x7fdd78150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd7806b000 0x7fdd7813e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd7813eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7ea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781559f0 con 0x7fdd7806ac20 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78155710 con 0x7fdd7806b000 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78148850 con 0x7fdd780c2310 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 0x7fdd78150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 0x7fdd78150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44166/0 (socket says 172.21.15.67:44166) 2024-02-26T10:43:16.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.638+0000 7fdd7d600640 1 -- 172.21.15.67:0/680614911 learned_addr learned my addr 172.21.15.67:0/680614911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7e000640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd7813eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7cc00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd7806b000 0x7fdd7813e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7d600640 1 -- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 msgr2=0x7fdd7813eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7d600640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd7813eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7d600640 1 -- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd7806b000 msgr2=0x7fdd7813e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7d600640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd7806b000 0x7fdd7813e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7d600640 1 -- 172.21.15.67:0/680614911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd781db500 con 0x7fdd7806ac20 2024-02-26T10:43:16.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.639+0000 7fdd7d600640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 0x7fdd78150b60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c390 tx=0x7fdd6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.640+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c017070 con 0x7fdd7806ac20 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.640+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c005ce0 con 0x7fdd7806ac20 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.640+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0072f0 con 0x7fdd7806ac20 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.640+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd78071640 con 0x7fdd7806ac20 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.640+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78071c00 con 0x7fdd7806ac20 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.641+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd78074a30 con 0x7fdd7806ac20 2024-02-26T10:43:16.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.642+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd6c01a070 con 0x7fdd7806ac20 2024-02-26T10:43:16.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.643+0000 7fdd75600640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdd5806b500 0x7fdd5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:16.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.643+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fdd6c0892a0 con 0x7fdd7806ac20 2024-02-26T10:43:16.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.643+0000 7fdd7cc00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdd5806b500 0x7fdd5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:16.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.644+0000 7fdd7cc00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdd5806b500 0x7fdd5806d9c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdd781db7e0 tx=0x7fdd64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.647+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd6c058430 con 0x7fdd7806ac20 2024-02-26T10:43:16.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.849+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd780c20f0 con 0x7fdd5806b500 2024-02-26T10:43:16.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.850+0000 7fdd75600640 1 -- 172.21.15.67:0/680614911 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdd780c20f0 con 0x7fdd5806b500 2024-02-26T10:43:16.847 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 3 to deep-scrub 2024-02-26T10:43:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.852+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdd5806b500 msgr2=0x7fdd5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.852+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdd5806b500 0x7fdd5806d9c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdd781db7e0 tx=0x7fdd64008040 comp rx=0 tx=0).stop 2024-02-26T10:43:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.852+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 msgr2=0x7fdd78150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.852+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 0x7fdd78150b60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c390 tx=0x7fdd6c00c850 comp rx=0 tx=0).stop 2024-02-26T10:43:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 shutdown_connections 2024-02-26T10:43:16.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdd780c2310 0x7fdd7813eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fdd5806b500 0x7fdd5806d9c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd7806b000 0x7fdd7813e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 --2- 172.21.15.67:0/680614911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd7806ac20 0x7fdd78150b60 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 >> 172.21.15.67:0/680614911 conn(0x7fdd780c63f0 msgr2=0x7fdd7806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 shutdown_connections 2024-02-26T10:43:16.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:16.853+0000 7fdd7ea00640 1 -- 172.21.15.67:0/680614911 wait complete. 2024-02-26T10:43:16.868 INFO:tasks.ceph:Scrubbing osd.4 2024-02-26T10:43:16.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.094+0000 7fcd3fe00640 1 Processor -- start 2024-02-26T10:43:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.094+0000 7fcd3fe00640 1 -- start start 2024-02-26T10:43:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd3fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd400c1eb0 0x7fcd4014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd3fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd4014f260 0x7fcd40151980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd3fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400c1be0 con 0x7fcd400c25a0 2024-02-26T10:43:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd3fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400c1750 con 0x7fcd400c1eb0 2024-02-26T10:43:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd3fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4014f640 con 0x7fcd4014f260 2024-02-26T10:43:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd37e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd400c1eb0 0x7fcd4014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd37e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd400c1eb0 0x7fcd4014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44170/0 (socket says 172.21.15.67:44170) 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd3ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcd400c25a0 0x7fcd400c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43570/0 (socket says 172.21.15.67:43570) 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.096+0000 7fcd37e00640 1 -- 172.21.15.67:0/3917648651 learned_addr learned my addr 172.21.15.67:0/3917648651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.097+0000 7fcd3f400640 1 --2- 172.21.15.67:0/3917648651 >> v2:172.21.15.82:3300/0 conn(0x7fcd4014f260 0x7fcd40151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.097+0000 7fcd37e00640 1 -- 172.21.15.67:0/3917648651 >> v2:172.21.15.82:3300/0 conn(0x7fcd4014f260 msgr2=0x7fcd40151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.097+0000 7fcd37e00640 1 --2- 172.21.15.67:0/3917648651 >> v2:172.21.15.82:3300/0 conn(0x7fcd4014f260 0x7fcd40151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.097+0000 7fcd37e00640 1 -- 172.21.15.67:0/3917648651 >> v1:172.21.15.67:6789/0 conn(0x7fcd400c25a0 legacy=0x7fcd400c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.097+0000 7fcd37e00640 1 -- 172.21.15.67:0/3917648651 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400c3660 con 0x7fcd400c1eb0 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.098+0000 7fcd37e00640 1 --2- 172.21.15.67:0/3917648651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c1eb0 0x7fcd4014ed20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcd400c3b50 tx=0x7fcd2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf475cd46f24f808 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.098+0000 7fcd3e000640 1 -- 172.21.15.67:0/3917648651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28035020 con 0x7fcd400c1eb0 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.098+0000 7fcd3e000640 1 -- 172.21.15.67:0/3917648651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28031870 con 0x7fcd400c1eb0 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.098+0000 7fcd3e000640 1 -- 172.21.15.67:0/3917648651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28031c40 con 0x7fcd400c1eb0 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 -- 172.21.15.67:0/3917648651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c1eb0 msgr2=0x7fcd4014ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 --2- 172.21.15.67:0/3917648651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c1eb0 0x7fcd4014ed20 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcd400c3b50 tx=0x7fcd2802b910 comp rx=0 tx=0).stop 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 -- 172.21.15.67:0/3917648651 shutdown_connections 2024-02-26T10:43:17.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 --2- 172.21.15.67:0/3917648651 >> v2:172.21.15.82:3300/0 conn(0x7fcd4014f260 0x7fcd40151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 --2- 172.21.15.67:0/3917648651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c1eb0 0x7fcd4014ed20 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 -- 172.21.15.67:0/3917648651 >> 172.21.15.67:0/3917648651 conn(0x7fcd400c6380 msgr2=0x7fcd400c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 -- 172.21.15.67:0/3917648651 shutdown_connections 2024-02-26T10:43:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.099+0000 7fcd3fe00640 1 -- 172.21.15.67:0/3917648651 wait complete. 2024-02-26T10:43:17.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.101+0000 7fcd3fe00640 1 Processor -- start 2024-02-26T10:43:17.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.101+0000 7fcd3fe00640 1 -- start start 2024-02-26T10:43:17.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd3fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd400c1eb0 0x7fcd40144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c25a0 0x7fcd40153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd3fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd4014f260 0x7fcd40143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd3fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c13e0 con 0x7fcd400c25a0 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd3fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c1100 con 0x7fcd400c1eb0 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd3fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c2980 con 0x7fcd4014f260 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c25a0 0x7fcd40153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c25a0 0x7fcd40153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44186/0 (socket says 172.21.15.67:44186) 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.102+0000 7fcd37e00640 1 -- 172.21.15.67:0/302687193 learned_addr learned my addr 172.21.15.67:0/302687193 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd3f400640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd4014f260 0x7fcd40143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd400c1eb0 0x7fcd40144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd37e00640 1 -- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd4014f260 msgr2=0x7fcd40143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd37e00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd4014f260 0x7fcd40143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd37e00640 1 -- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd400c1eb0 msgr2=0x7fcd40144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd37e00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd400c1eb0 0x7fcd40144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd37e00640 1 -- 172.21.15.67:0/302687193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd401eb3f0 con 0x7fcd400c25a0 2024-02-26T10:43:17.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd37e00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c25a0 0x7fcd40153360 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcd28000c00 tx=0x7fcd280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28035040 con 0x7fcd400c25a0 2024-02-26T10:43:17.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28030d00 con 0x7fcd400c25a0 2024-02-26T10:43:17.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280456a0 con 0x7fcd400c25a0 2024-02-26T10:43:17.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.103+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd40150a90 con 0x7fcd400c25a0 2024-02-26T10:43:17.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.104+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd40151050 con 0x7fcd400c25a0 2024-02-26T10:43:17.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.105+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd40074a30 con 0x7fcd400c25a0 2024-02-26T10:43:17.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.106+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd28038020 con 0x7fcd400c25a0 2024-02-26T10:43:17.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.107+0000 7fcd36a00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd1c06b4d0 0x7fcd1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.107+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcd280ac2f0 con 0x7fcd400c25a0 2024-02-26T10:43:17.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.107+0000 7fcd36a00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7fcd1c071430 0x7fcd1c073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.107+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 --> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd2803be50 con 0x7fcd1c071430 2024-02-26T10:43:17.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.107+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd280b3050 con 0x7fcd400c25a0 2024-02-26T10:43:17.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.107+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd1c06b4d0 0x7fcd1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.108+0000 7fcd3f400640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7fcd1c071430 0x7fcd1c073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.109+0000 7fcd3f400640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7fcd1c071430 0x7fcd1c073850 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.109+0000 7fcd3ea00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd1c06b4d0 0x7fcd1c06d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcd30000f60 tx=0x7fcd30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.111+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== osd.4 v2:172.21.15.82:6816/900472103 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcd2803be50 con 0x7fcd1c071430 2024-02-26T10:43:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.125+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 --> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcd400a9930 con 0x7fcd1c071430 2024-02-26T10:43:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.126+0000 7fcd36a00640 1 -- 172.21.15.67:0/302687193 <== osd.4 v2:172.21.15.82:6816/900472103 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcd400a9930 con 0x7fcd1c071430 2024-02-26T10:43:17.124 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:17.124 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7fcd1c071430 msgr2=0x7fcd1c073850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6816/900472103,v1:172.21.15.82:6817/900472103] conn(0x7fcd1c071430 0x7fcd1c073850 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd1c06b4d0 msgr2=0x7fcd1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fcd1c06b4d0 0x7fcd1c06d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcd30000f60 tx=0x7fcd30009040 comp rx=0 tx=0).stop 2024-02-26T10:43:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c25a0 msgr2=0x7fcd40153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 --2- 172.21.15.67:0/302687193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd400c25a0 0x7fcd40153360 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcd28000c00 tx=0x7fcd280046c0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3f400640 1 -- 172.21.15.67:0/302687193 reap_dead start 2024-02-26T10:43:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 shutdown_connections 2024-02-26T10:43:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 >> 172.21.15.67:0/302687193 conn(0x7fcd400c6380 msgr2=0x7fcd4014e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.127+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 shutdown_connections 2024-02-26T10:43:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.128+0000 7fcd3fe00640 1 -- 172.21.15.67:0/302687193 wait complete. 2024-02-26T10:43:17.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-02-26T10:43:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.371+0000 7f6dcb200640 1 Processor -- start 2024-02-26T10:43:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.371+0000 7f6dcb200640 1 -- start start 2024-02-26T10:43:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dcb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6dcc1526d0 0x7f6dcc0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dcb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dcc0c4c60 0x7f6dcc0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dcb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc155140 con 0x7f6dcc0c2d90 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dcb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc071a10 con 0x7f6dcc1526d0 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dcb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc155d50 con 0x7f6dcc0c4c60 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dc9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6dcc1526d0 0x7f6dcc0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dc9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6dcc1526d0 0x7f6dcc0714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44198/0 (socket says 172.21.15.67:44198) 2024-02-26T10:43:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.373+0000 7f6dc9e00640 1 -- 172.21.15.67:0/2354605865 learned_addr learned my addr 172.21.15.67:0/2354605865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.374+0000 7f6dca800640 1 --2- 172.21.15.67:0/2354605865 >> v2:172.21.15.82:3300/0 conn(0x7f6dcc0c4c60 0x7f6dcc0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.374+0000 7f6dc9e00640 1 -- 172.21.15.67:0/2354605865 >> v2:172.21.15.82:3300/0 conn(0x7f6dcc0c4c60 msgr2=0x7f6dcc0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.374+0000 7f6dc9e00640 1 --2- 172.21.15.67:0/2354605865 >> v2:172.21.15.82:3300/0 conn(0x7f6dcc0c4c60 0x7f6dcc0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.374+0000 7f6dc9e00640 1 -- 172.21.15.67:0/2354605865 >> v1:172.21.15.82:6789/0 conn(0x7f6dcc0c2d90 legacy=0x7f6dcc06bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.374+0000 7f6dc9e00640 1 -- 172.21.15.67:0/2354605865 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc155420 con 0x7f6dcc1526d0 2024-02-26T10:43:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.376+0000 7f6dc9e00640 1 --2- 172.21.15.67:0/2354605865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc0714d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc072e00 tx=0x7f6dbc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4adf05a7c6a490 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.376+0000 7f6dc8a00640 1 -- 172.21.15.67:0/2354605865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc02c9f0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.376+0000 7f6dc8a00640 1 -- 172.21.15.67:0/2354605865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc02cb90 con 0x7f6dcc1526d0 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.376+0000 7f6dc8a00640 1 -- 172.21.15.67:0/2354605865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0343f0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 -- 172.21.15.67:0/2354605865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 msgr2=0x7f6dcc0714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2354605865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc0714d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc072e00 tx=0x7f6dbc02be50 comp rx=0 tx=0).stop 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 -- 172.21.15.67:0/2354605865 shutdown_connections 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2354605865 >> v2:172.21.15.82:3300/0 conn(0x7f6dcc0c4c60 0x7f6dcc0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2354605865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc0714d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 -- 172.21.15.67:0/2354605865 >> 172.21.15.67:0/2354605865 conn(0x7f6dcc0c36b0 msgr2=0x7f6dcc0c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 -- 172.21.15.67:0/2354605865 shutdown_connections 2024-02-26T10:43:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.377+0000 7f6dcb200640 1 -- 172.21.15.67:0/2354605865 wait complete. 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.378+0000 7f6dcb200640 1 Processor -- start 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 -- start start 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6dcc0c2d90 0x7f6dcc153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc0c4c60 0x7f6dcc0a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc152ab0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1559e0 con 0x7f6dcc0c4c60 2024-02-26T10:43:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.379+0000 7f6dcb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc06c340 con 0x7f6dcc0c2d90 2024-02-26T10:43:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc1541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44212/0 (socket says 172.21.15.67:44212) 2024-02-26T10:43:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 -- 172.21.15.67:0/2048659386 learned_addr learned my addr 172.21.15.67:0/2048659386 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dc9e00640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6dcc0c2d90 0x7f6dcc153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dc9400640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc0c4c60 0x7f6dcc0a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 -- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6dcc0c2d90 msgr2=0x7f6dcc153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6dcc0c2d90 0x7f6dcc153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 -- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc0c4c60 msgr2=0x7f6dcc0a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc0c4c60 0x7f6dcc0a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dca800640 1 -- 172.21.15.67:0/2048659386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc06d5c0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dc9e00640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6dcc0c2d90 0x7f6dcc153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.380+0000 7f6dc9400640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc0c4c60 0x7f6dcc0a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.381+0000 7f6dca800640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc1541a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400d600 tx=0x7f6dc400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.381+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40125f0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.381+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc4019070 con 0x7f6dcc1526d0 2024-02-26T10:43:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.381+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc06df60 con 0x7f6dcc1526d0 2024-02-26T10:43:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.381+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40129f0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.381+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc147fb0 con 0x7f6dcc1526d0 2024-02-26T10:43:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.382+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc074a30 con 0x7f6dcc1526d0 2024-02-26T10:43:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.383+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dc4012b90 con 0x7f6dcc1526d0 2024-02-26T10:43:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.385+0000 7f6dc2000640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6dac06b4d0 0x7f6dac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.385+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6dc408a390 con 0x7f6dcc1526d0 2024-02-26T10:43:17.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.385+0000 7f6dc9e00640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6dac06b4d0 0x7f6dac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.386+0000 7f6dc9e00640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6dac06b4d0 0x7f6dac06d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc035000 tx=0x7f6dbc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.389+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6dc4059420 con 0x7f6dcc1526d0 2024-02-26T10:43:17.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.587+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dcc152f70 con 0x7f6dac06b4d0 2024-02-26T10:43:17.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.588+0000 7f6dc2000640 1 -- 172.21.15.67:0/2048659386 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6dbc02c8f0 con 0x7f6dac06b4d0 2024-02-26T10:43:17.586 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 4 to deep-scrub 2024-02-26T10:43:17.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6dac06b4d0 msgr2=0x7f6dac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6dac06b4d0 0x7f6dac06d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc035000 tx=0x7f6dbc030000 comp rx=0 tx=0).stop 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 msgr2=0x7f6dcc1541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc1541a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400d600 tx=0x7f6dc400dac0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 shutdown_connections 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc1526d0 0x7f6dcc1541a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f6dac06b4d0 0x7f6dac06d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc0c4c60 0x7f6dcc0a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 --2- 172.21.15.67:0/2048659386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6dcc0c2d90 0x7f6dcc153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 >> 172.21.15.67:0/2048659386 conn(0x7f6dcc0c36b0 msgr2=0x7f6dcc0c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 shutdown_connections 2024-02-26T10:43:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.592+0000 7f6dcb200640 1 -- 172.21.15.67:0/2048659386 wait complete. 2024-02-26T10:43:17.610 INFO:tasks.ceph:Scrubbing osd.5 2024-02-26T10:43:17.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.842+0000 7f364e200640 1 Processor -- start 2024-02-26T10:43:17.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.842+0000 7f364e200640 1 -- start start 2024-02-26T10:43:17.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 0x7f36480ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3648150780 con 0x7f364806e2d0 2024-02-26T10:43:17.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364e200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36480a98c0 con 0x7f36480a9430 2024-02-26T10:43:17.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364e200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480a9ba0 con 0x7f36481503a0 2024-02-26T10:43:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f364806e2d0 0x7f36480c2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43580/0 (socket says 172.21.15.67:43580) 2024-02-26T10:43:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364d800640 1 -- 172.21.15.67:0/3707738920 learned_addr learned my addr 172.21.15.67:0/3707738920 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:17.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.844+0000 7f364ce00640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 0x7f36480ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745473687 0 0) 0x7f3648150780 con 0x7f364806e2d0 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3630003520 con 0x7f364806e2d0 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765337326 0 0) 0x7f36480a98c0 con 0x7f36480a9430 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3648150780 con 0x7f36480a9430 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 851908744 0 0) 0x7f3630003520 con 0x7f364806e2d0 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 msgr2=0x7f36480ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 0x7f36480ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 >> v1:172.21.15.82:6789/0 conn(0x7f36480a9430 legacy=0x7f36481535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480a9e80 con 0x7f364806e2d0 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f36400019f0 con 0x7f364806e2d0 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 >> v1:172.21.15.67:6789/0 conn(0x7f364806e2d0 legacy=0x7f36480c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3630003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3630004800 0x7f3630004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 0x7f36300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480a9ba0 con 0x7f3630003b70 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.845+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363000ba70 con 0x7f3630004800 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3630010080 con 0x7f36481503a0 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f364ce00640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3630003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f3647e00640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3630004800 0x7f3630004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f364ce00640 1 -- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 msgr2=0x7f36300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f364ce00640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f36481503a0 0x7f36300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f364ce00640 1 -- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3630004800 msgr2=0x7f3630004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f364ce00640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3630004800 0x7f3630004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.846+0000 7f364ce00640 1 -- 172.21.15.67:0/3707738920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3630003520 con 0x7f3630003b70 2024-02-26T10:43:17.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.847+0000 7f364ce00640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3630003f70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f363c009ac0 tx=0x7f3630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d7f343f1ead4f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c039070 con 0x7f3630003b70 2024-02-26T10:43:17.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c02c710 con 0x7f3630003b70 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f3647400640 1 -- 172.21.15.67:0/3707738920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c009690 con 0x7f3630003b70 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364e200640 1 -- 172.21.15.67:0/3707738920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 msgr2=0x7f3630003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364e200640 1 --2- 172.21.15.67:0/3707738920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3630003f70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f363c009ac0 tx=0x7f3630005830 comp rx=0 tx=0).stop 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364d800640 1 -- 172.21.15.67:0/3707738920 reap_dead start 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364e200640 1 -- 172.21.15.67:0/3707738920 shutdown_connections 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364e200640 1 -- 172.21.15.67:0/3707738920 >> 172.21.15.67:0/3707738920 conn(0x7f36480c6380 msgr2=0x7f36480c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364e200640 1 -- 172.21.15.67:0/3707738920 shutdown_connections 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.848+0000 7f364e200640 1 -- 172.21.15.67:0/3707738920 wait complete. 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.850+0000 7f364e200640 1 Processor -- start 2024-02-26T10:43:17.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.850+0000 7f364e200640 1 -- start start 2024-02-26T10:43:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3648146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3648146cb0 0x7f364814af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364e200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36481e9cb0 0x7f36481e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c0f80 con 0x7f3630003b70 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364e200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364806ddb0 con 0x7f36481e9cb0 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364e200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36480c3520 con 0x7f3648146cb0 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3648146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3648146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44244/0 (socket says 172.21.15.67:44244) 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.851+0000 7f364ce00640 1 -- 172.21.15.67:0/4235755318 learned_addr learned my addr 172.21.15.67:0/4235755318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364ce00640 1 -- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3648146cb0 msgr2=0x7f364814af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f3647e00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3648146cb0 0x7f364814af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364d800640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36481e9cb0 0x7f36481e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364ce00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3648146cb0 0x7f364814af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364ce00640 1 -- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36481e9cb0 msgr2=0x7f36481e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364ce00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36481e9cb0 0x7f36481e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364ce00640 1 -- 172.21.15.67:0/4235755318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364814d490 con 0x7f3630003b70 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.852+0000 7f364ce00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3648146770 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f363c009be0 tx=0x7f363c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.853+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c039050 con 0x7f3630003b70 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.853+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c045070 con 0x7f3630003b70 2024-02-26T10:43:17.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.853+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c003a50 con 0x7f3630003b70 2024-02-26T10:43:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.853+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480c5800 con 0x7f3630003b70 2024-02-26T10:43:17.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.853+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36480c5dc0 con 0x7f3630003b70 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.854+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36480c5f90 con 0x7f3630003b70 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.855+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f363c003bf0 con 0x7f3630003b70 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.856+0000 7f3644c00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f362806b4d0 0x7f362806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.856+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f363c0abbc0 con 0x7f3630003b70 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.856+0000 7f3644c00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7f3628071430 0x7f3628073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.856+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 --> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f363c008eb0 con 0x7f3628071430 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.856+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363c0b02a0 con 0x7f3630003b70 2024-02-26T10:43:17.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.856+0000 7f3647e00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f362806b4d0 0x7f362806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.857+0000 7f364d800640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7f3628071430 0x7f3628073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.858+0000 7f3647e00640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f362806b4d0 0x7f362806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f36480ac650 tx=0x7f3634009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.858+0000 7f364d800640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7f3628071430 0x7f3628073850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:17.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.860+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== osd.5 v2:172.21.15.82:6800/431290960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f363c008eb0 con 0x7f3628071430 2024-02-26T10:43:17.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.875+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 --> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3648074a30 con 0x7f3628071430 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.876+0000 7f3644c00640 1 -- 172.21.15.67:0/4235755318 <== osd.5 v2:172.21.15.82:6800/431290960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3648074a30 con 0x7f3628071430 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:17.873 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:17.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.876+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7f3628071430 msgr2=0x7f3628073850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.876+0000 7f364e200640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6800/431290960,v1:172.21.15.82:6801/431290960] conn(0x7f3628071430 0x7f3628073850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:17.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f362806b4d0 msgr2=0x7f362806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f362806b4d0 0x7f362806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f36480ac650 tx=0x7f3634009040 comp rx=0 tx=0).stop 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 msgr2=0x7f3648146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 --2- 172.21.15.67:0/4235755318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3630003b70 0x7f3648146770 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f363c009be0 tx=0x7f363c004730 comp rx=0 tx=0).stop 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364d800640 1 -- 172.21.15.67:0/4235755318 reap_dead start 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 shutdown_connections 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 >> 172.21.15.67:0/4235755318 conn(0x7f36480c6380 msgr2=0x7f3648147880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 shutdown_connections 2024-02-26T10:43:17.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:17.877+0000 7f364e200640 1 -- 172.21.15.67:0/4235755318 wait complete. 2024-02-26T10:43:17.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-02-26T10:43:18.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.111+0000 7f2063e00640 1 Processor -- start 2024-02-26T10:43:18.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.111+0000 7f2063e00640 1 -- start start 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.112+0000 7f2063e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20640c2710 0x7f206406db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.112+0000 7f2063e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2064151d00 0x7f20640abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.112+0000 7f2063e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206406e0e0 con 0x7f20640ac230 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.112+0000 7f2063e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2064070c50 con 0x7f2064151d00 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.112+0000 7f2063e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206406e3c0 con 0x7f20640c2710 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2064151d00 0x7f20640abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2064151d00 0x7f20640abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44250/0 (socket says 172.21.15.67:44250) 2024-02-26T10:43:18.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 -- 172.21.15.67:0/4089878424 learned_addr learned my addr 172.21.15.67:0/4089878424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:18.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062a00640 1 --2- 172.21.15.67:0/4089878424 >> v2:172.21.15.82:3300/0 conn(0x7f20640c2710 0x7f206406db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 -- 172.21.15.67:0/4089878424 >> v2:172.21.15.82:3300/0 conn(0x7f20640c2710 msgr2=0x7f206406db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:18.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 --2- 172.21.15.67:0/4089878424 >> v2:172.21.15.82:3300/0 conn(0x7f20640c2710 0x7f206406db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 -- 172.21.15.67:0/4089878424 >> v1:172.21.15.82:6789/0 conn(0x7f20640ac230 legacy=0x7f20640716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:18.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062000640 1 -- 172.21.15.67:0/4089878424 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064070ea0 con 0x7f2064151d00 2024-02-26T10:43:18.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.113+0000 7f2062a00640 1 --2- 172.21.15.67:0/4089878424 >> v2:172.21.15.82:3300/0 conn(0x7f20640c2710 0x7f206406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:18.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.115+0000 7f2062000640 1 --2- 172.21.15.67:0/4089878424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064151d00 0x7f20640abcf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f204c00bb10 tx=0x7f204c02dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f784fc4b998f605a server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.115+0000 7f2061600640 1 -- 172.21.15.67:0/4089878424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c03b070 con 0x7f2064151d00 2024-02-26T10:43:18.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.115+0000 7f2061600640 1 -- 172.21.15.67:0/4089878424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c02ed60 con 0x7f2064151d00 2024-02-26T10:43:18.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.115+0000 7f2061600640 1 -- 172.21.15.67:0/4089878424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c036660 con 0x7f2064151d00 2024-02-26T10:43:18.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 -- 172.21.15.67:0/4089878424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064151d00 msgr2=0x7f20640abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 --2- 172.21.15.67:0/4089878424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064151d00 0x7f20640abcf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f204c00bb10 tx=0x7f204c02dfb0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 -- 172.21.15.67:0/4089878424 shutdown_connections 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 --2- 172.21.15.67:0/4089878424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064151d00 0x7f20640abcf0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 --2- 172.21.15.67:0/4089878424 >> v2:172.21.15.82:3300/0 conn(0x7f20640c2710 0x7f206406db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 -- 172.21.15.67:0/4089878424 >> 172.21.15.67:0/4089878424 conn(0x7f20640c63f0 msgr2=0x7f20640c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 -- 172.21.15.67:0/4089878424 shutdown_connections 2024-02-26T10:43:18.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.116+0000 7f2063e00640 1 -- 172.21.15.67:0/4089878424 wait complete. 2024-02-26T10:43:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.118+0000 7f2063e00640 1 Processor -- start 2024-02-26T10:43:18.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.118+0000 7f2063e00640 1 -- start start 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 0x7f2064154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640c2710 0x7f2064155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2064151d00 0x7f20641ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2064152160 con 0x7f20640ac230 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20640c2af0 con 0x7f20640c2710 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2064071db0 con 0x7f2064151d00 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2062a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 0x7f2064154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2062a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 0x7f2064154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44264/0 (socket says 172.21.15.67:44264) 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2062a00640 1 -- 172.21.15.67:0/1656574875 learned_addr learned my addr 172.21.15.67:0/1656574875 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2063400640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2064151d00 0x7f20641ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.119+0000 7f2062000640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640c2710 0x7f2064155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062a00640 1 -- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2064151d00 msgr2=0x7f20641ddf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062a00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2064151d00 0x7f20641ddf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062a00640 1 -- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640c2710 msgr2=0x7f2064155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062a00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640c2710 0x7f2064155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062a00640 1 -- 172.21.15.67:0/1656574875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064150110 con 0x7f20640ac230 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062000640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640c2710 0x7f2064155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:18.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2062a00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 0x7f2064154b50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f205400cf60 tx=0x7f205400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205401a070 con 0x7f20640ac230 2024-02-26T10:43:18.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205400d840 con 0x7f20640ac230 2024-02-26T10:43:18.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400dc10 con 0x7f20640ac230 2024-02-26T10:43:18.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20641503f0 con 0x7f20640ac230 2024-02-26T10:43:18.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.120+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641509b0 con 0x7f20640ac230 2024-02-26T10:43:18.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.121+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206406dbc0 con 0x7f20640ac230 2024-02-26T10:43:18.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.122+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f205401d070 con 0x7f20640ac230 2024-02-26T10:43:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.123+0000 7f205aa00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f204006b500 0x7f204006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.123+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f205408a7b0 con 0x7f20640ac230 2024-02-26T10:43:18.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.124+0000 7f2062000640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f204006b500 0x7f204006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.125+0000 7f2062000640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f204006b500 0x7f204006d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f204c038000 tx=0x7f204c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.128+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f206406dbc0 con 0x7f20640ac230 2024-02-26T10:43:18.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.329+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f2064074a30 con 0x7f204006b500 2024-02-26T10:43:18.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.331+0000 7f205aa00640 1 -- 172.21.15.67:0/1656574875 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2064074a30 con 0x7f204006b500 2024-02-26T10:43:18.328 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 5 to deep-scrub 2024-02-26T10:43:18.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f204006b500 msgr2=0x7f204006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f204006b500 0x7f204006d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f204c038000 tx=0x7f204c008040 comp rx=0 tx=0).stop 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 msgr2=0x7f2064154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 0x7f2064154b50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f205400cf60 tx=0x7f205400cf90 comp rx=0 tx=0).stop 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 shutdown_connections 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2064151d00 0x7f20641ddf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f204006b500 0x7f204006d9c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640c2710 0x7f2064155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 --2- 172.21.15.67:0/1656574875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20640ac230 0x7f2064154b50 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 >> 172.21.15.67:0/1656574875 conn(0x7f20640c63f0 msgr2=0x7f2064072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 shutdown_connections 2024-02-26T10:43:18.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.333+0000 7f2063e00640 1 -- 172.21.15.67:0/1656574875 wait complete. 2024-02-26T10:43:18.351 INFO:tasks.ceph:Scrubbing osd.6 2024-02-26T10:43:18.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.581+0000 7f23db000640 1 Processor -- start 2024-02-26T10:43:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.581+0000 7f23db000640 1 -- start start 2024-02-26T10:43:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.583+0000 7f23db000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f23d41523b0 0x7f23d40c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.583+0000 7f23db000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40c3840 con 0x7f23d4071fd0 2024-02-26T10:43:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.583+0000 7f23db000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d406dc80 con 0x7f23d4154cf0 2024-02-26T10:43:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.583+0000 7f23db000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d4152790 con 0x7f23d41523b0 2024-02-26T10:43:18.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.583+0000 7f23da600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f23d4071fd0 0x7f23d40723e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43596/0 (socket says 172.21.15.67:43596) 2024-02-26T10:43:18.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.583+0000 7f23da600640 1 -- 172.21.15.67:0/1861125374 learned_addr learned my addr 172.21.15.67:0/1861125374 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d9c00640 1 --2- 172.21.15.67:0/1861125374 >> v2:172.21.15.82:3300/0 conn(0x7f23d41523b0 0x7f23d40c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996476389 0 0) 0x7f23d40c3840 con 0x7f23d4071fd0 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23bc003520 con 0x7f23d4071fd0 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4276919945 0 0) 0x7f23bc003520 con 0x7f23d4071fd0 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 >> v2:172.21.15.82:3300/0 conn(0x7f23d41523b0 msgr2=0x7f23d40c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 --2- 172.21.15.67:0/1861125374 >> v2:172.21.15.82:3300/0 conn(0x7f23d41523b0 0x7f23d40c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 >> v1:172.21.15.82:6789/0 conn(0x7f23d4154cf0 legacy=0x7f23d40c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d406df60 con 0x7f23d4071fd0 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7f23cc0019f0 con 0x7f23d4071fd0 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 >> v1:172.21.15.67:6789/0 conn(0x7f23d4071fd0 legacy=0x7f23d40723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0039b0 0x7f23bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0052e0 0x7f23bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4152790 con 0x7f23bc0052e0 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc00c1b0 con 0x7f23bc004800 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.584+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc00c3b0 con 0x7f23bc0039b0 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23da600640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0052e0 0x7f23bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23d9c00640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0039b0 0x7f23bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23da600640 1 -- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0039b0 msgr2=0x7f23bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23d9200640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23da600640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0039b0 0x7f23bc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23da600640 1 -- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23bc004800 msgr2=0x7f23bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23da600640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23da600640 1 -- 172.21.15.67:0/1861125374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc003520 con 0x7f23bc0052e0 2024-02-26T10:43:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.585+0000 7f23d9200640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23bc004800 0x7f23bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:18.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.586+0000 7f23da600640 1 -- 172.21.15.67:0/1861125374 reap_dead start 2024-02-26T10:43:18.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.586+0000 7f23da600640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0052e0 0x7f23bc00b9c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00a830 tx=0x7f23bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e30d216d5ba1f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.586+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00a480 con 0x7f23bc0052e0 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc034630 con 0x7f23bc0052e0 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23d3e00640 1 -- 172.21.15.67:0/1861125374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc034a00 con 0x7f23bc0052e0 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 -- 172.21.15.67:0/1861125374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0052e0 msgr2=0x7f23bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0052e0 0x7f23bc00b9c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00a830 tx=0x7f23bc005830 comp rx=0 tx=0).stop 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 -- 172.21.15.67:0/1861125374 shutdown_connections 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 --2- 172.21.15.67:0/1861125374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0052e0 0x7f23bc00b9c0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 -- 172.21.15.67:0/1861125374 >> 172.21.15.67:0/1861125374 conn(0x7f23d40c6380 msgr2=0x7f23d40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 -- 172.21.15.67:0/1861125374 shutdown_connections 2024-02-26T10:43:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.587+0000 7f23db000640 1 -- 172.21.15.67:0/1861125374 wait complete. 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.589+0000 7f23db000640 1 Processor -- start 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.589+0000 7f23db000640 1 -- start start 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23db000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0052e0 0x7f23d4158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23db000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d41568a0 0x7f23d40c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23db000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d41eaf10 0x7f23d41eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23db000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d406e520 con 0x7f23d41568a0 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23db000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d406da30 con 0x7f23d41eaf10 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23db000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d406e800 con 0x7f23bc0052e0 2024-02-26T10:43:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d41568a0 0x7f23d40c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23d9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d41568a0 0x7f23d40c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44282/0 (socket says 172.21.15.67:44282) 2024-02-26T10:43:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.590+0000 7f23d9200640 1 -- 172.21.15.67:0/1191774341 learned_addr learned my addr 172.21.15.67:0/1191774341 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:18.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9c00640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0052e0 0x7f23d4158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23da600640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d41eaf10 0x7f23d41eb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9200640 1 -- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0052e0 msgr2=0x7f23d4158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9200640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23bc0052e0 0x7f23d4158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9200640 1 -- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d41eaf10 msgr2=0x7f23d41eb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9200640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d41eaf10 0x7f23d41eb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9200640 1 -- 172.21.15.67:0/1191774341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4156430 con 0x7f23d41568a0 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d9200640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d41568a0 0x7f23d40c5360 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f23c000d7a0 tx=0x7f23c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0005270 con 0x7f23d41568a0 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0017070 con 0x7f23d41568a0 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c000b6b0 con 0x7f23d41568a0 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.591+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d414d4e0 con 0x7f23d41568a0 2024-02-26T10:43:18.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.592+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d40c58a0 con 0x7f23d41568a0 2024-02-26T10:43:18.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.592+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23d4074a30 con 0x7f23d41568a0 2024-02-26T10:43:18.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.594+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23c000b850 con 0x7f23d41568a0 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.594+0000 7f23d1600640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23b406b4d0 0x7f23b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.595+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f23c0089e90 con 0x7f23d41568a0 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.595+0000 7f23d1600640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f23b4071430 0x7f23b4073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.595+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 --> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23b4073f00 con 0x7f23b4071430 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.595+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c008c4b0 con 0x7f23d41568a0 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.595+0000 7f23d9c00640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23b406b4d0 0x7f23b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.595+0000 7f23da600640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f23b4071430 0x7f23b4073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.597+0000 7f23da600640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f23b4071430 0x7f23b4073850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.597+0000 7f23d9c00640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23b406b4d0 0x7f23b406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f23d40c2440 tx=0x7f23c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.599+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== osd.6 v2:172.21.15.82:6808/1261740917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f23b4073f00 con 0x7f23b4071430 2024-02-26T10:43:18.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.613+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 --> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f23d40bf840 con 0x7f23b4071430 2024-02-26T10:43:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.614+0000 7f23d1600640 1 -- 172.21.15.67:0/1191774341 <== osd.6 v2:172.21.15.82:6808/1261740917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f23d40bf840 con 0x7f23b4071430 2024-02-26T10:43:18.611 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:18.612 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f23b4071430 msgr2=0x7f23b4073850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6808/1261740917,v1:172.21.15.82:6809/1261740917] conn(0x7f23b4071430 0x7f23b4073850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23b406b4d0 msgr2=0x7f23b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f23b406b4d0 0x7f23b406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f23d40c2440 tx=0x7f23c8008040 comp rx=0 tx=0).stop 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d41568a0 msgr2=0x7f23d40c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 --2- 172.21.15.67:0/1191774341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d41568a0 0x7f23d40c5360 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f23c000d7a0 tx=0x7f23c000dc60 comp rx=0 tx=0).stop 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23da600640 1 -- 172.21.15.67:0/1191774341 reap_dead start 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 shutdown_connections 2024-02-26T10:43:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 >> 172.21.15.67:0/1191774341 conn(0x7f23d40c6380 msgr2=0x7f23d4153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 shutdown_connections 2024-02-26T10:43:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.615+0000 7f23db000640 1 -- 172.21.15.67:0/1191774341 wait complete. 2024-02-26T10:43:18.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-02-26T10:43:18.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.848+0000 7f87e2600640 1 Processor -- start 2024-02-26T10:43:18.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.848+0000 7f87e2600640 1 -- start start 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.849+0000 7f87e2600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.849+0000 7f87e2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87dc06b000 0x7f87dc1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.849+0000 7f87e2600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc155f10 con 0x7f87dc06ac20 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.849+0000 7f87e2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc06b470 con 0x7f87dc06b000 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.849+0000 7f87e2600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c28d0 con 0x7f87dc0c2310 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f87dc06ac20 0x7f87dc152f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43600/0 (socket says 172.21.15.67:43600) 2024-02-26T10:43:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1200640 1 -- 172.21.15.67:0/614137565 learned_addr learned my addr 172.21.15.67:0/614137565 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:18.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1c00640 1 --2- 172.21.15.67:0/614137565 >> v2:172.21.15.67:3300/0 conn(0x7f87dc06b000 0x7f87dc1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87dbe00640 1 --2- 172.21.15.67:0/614137565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1c00640 1 -- 172.21.15.67:0/614137565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 msgr2=0x7f87dc153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1c00640 1 --2- 172.21.15.67:0/614137565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87db400640 1 -- 172.21.15.67:0/614137565 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661507495 0 0) 0x7f87dc155f10 con 0x7f87dc06ac20 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1c00640 1 -- 172.21.15.67:0/614137565 >> v1:172.21.15.67:6789/0 conn(0x7f87dc06ac20 legacy=0x7f87dc152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.850+0000 7f87e1c00640 1 -- 172.21.15.67:0/614137565 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc155150 con 0x7f87dc06b000 2024-02-26T10:43:18.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.852+0000 7f87e1c00640 1 --2- 172.21.15.67:0/614137565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1524b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f87d40097a0 tx=0x7f87d402d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5399b882d3dbbb38 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.852+0000 7f87db400640 1 -- 172.21.15.67:0/614137565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4036070 con 0x7f87dc06b000 2024-02-26T10:43:18.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.852+0000 7f87db400640 1 -- 172.21.15.67:0/614137565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d4032870 con 0x7f87dc06b000 2024-02-26T10:43:18.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.852+0000 7f87db400640 1 -- 172.21.15.67:0/614137565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4032c40 con 0x7f87dc06b000 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 -- 172.21.15.67:0/614137565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 msgr2=0x7f87dc1524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 --2- 172.21.15.67:0/614137565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1524b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f87d40097a0 tx=0x7f87d402d9b0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 -- 172.21.15.67:0/614137565 shutdown_connections 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 --2- 172.21.15.67:0/614137565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1524b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 --2- 172.21.15.67:0/614137565 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 -- 172.21.15.67:0/614137565 >> 172.21.15.67:0/614137565 conn(0x7f87dc0c63f0 msgr2=0x7f87dc0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:18.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 -- 172.21.15.67:0/614137565 shutdown_connections 2024-02-26T10:43:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.853+0000 7f87e2600640 1 -- 172.21.15.67:0/614137565 wait complete. 2024-02-26T10:43:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.855+0000 7f87e2600640 1 Processor -- start 2024-02-26T10:43:18.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.855+0000 7f87e2600640 1 -- start start 2024-02-26T10:43:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e2600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87dc06ac20 0x7f87dc154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e2600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc1ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1559f0 con 0x7f87dc06b000 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e2600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc155710 con 0x7f87dc06ac20 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e2600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc148850 con 0x7f87dc0c2310 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1ee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44308/0 (socket says 172.21.15.67:44308) 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 -- 172.21.15.67:0/2352404391 learned_addr learned my addr 172.21.15.67:0/2352404391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e1200640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87dc06ac20 0x7f87dc154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87e1c00640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc1ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 -- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 msgr2=0x7f87dc1ee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc1ee8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 -- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87dc06ac20 msgr2=0x7f87dc154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.856+0000 7f87dbe00640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87dc06ac20 0x7f87dc154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87dbe00640 1 -- 172.21.15.67:0/2352404391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc154df0 con 0x7f87dc06b000 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87e1200640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87dc06ac20 0x7f87dc154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87dbe00640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1ee380 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f87c400d800 tx=0x7f87c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c400b6c0 con 0x7f87dc06b000 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c4019070 con 0x7f87dc06b000 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c400ba80 con 0x7f87dc06b000 2024-02-26T10:43:18.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc151e10 con 0x7f87dc06b000 2024-02-26T10:43:18.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.857+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0c0670 con 0x7f87dc06b000 2024-02-26T10:43:18.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.858+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc074a30 con 0x7f87dc06b000 2024-02-26T10:43:18.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.859+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87c400bc20 con 0x7f87dc06b000 2024-02-26T10:43:18.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.860+0000 7f87d8c00640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87b806b4d0 0x7f87b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:18.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.860+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f87c408a450 con 0x7f87dc06b000 2024-02-26T10:43:18.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.861+0000 7f87e1200640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87b806b4d0 0x7f87b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:18.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.862+0000 7f87e1200640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87b806b4d0 0x7f87b806d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f87cc007fd0 tx=0x7f87cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:18.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:18.865+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87dc154df0 con 0x7f87dc06b000 2024-02-26T10:43:19.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.065+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f87dc0c20f0 con 0x7f87b806b4d0 2024-02-26T10:43:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.066+0000 7f87d8c00640 1 -- 172.21.15.67:0/2352404391 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f87dc0c20f0 con 0x7f87b806b4d0 2024-02-26T10:43:19.063 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 6 to deep-scrub 2024-02-26T10:43:19.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.068+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87b806b4d0 msgr2=0x7f87b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.068+0000 7f87e2600640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87b806b4d0 0x7f87b806d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f87cc007fd0 tx=0x7f87cc009040 comp rx=0 tx=0).stop 2024-02-26T10:43:19.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.068+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 msgr2=0x7f87dc1ee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.068+0000 7f87e2600640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1ee380 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f87c400d800 tx=0x7f87c400dcc0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 shutdown_connections 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87dc0c2310 0x7f87dc1ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87dc06b000 0x7f87dc1ee380 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f87b806b4d0 0x7f87b806d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 --2- 172.21.15.67:0/2352404391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87dc06ac20 0x7f87dc154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 >> 172.21.15.67:0/2352404391 conn(0x7f87dc0c63f0 msgr2=0x7f87dc146af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 shutdown_connections 2024-02-26T10:43:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.069+0000 7f87e2600640 1 -- 172.21.15.67:0/2352404391 wait complete. 2024-02-26T10:43:19.083 INFO:tasks.ceph:Scrubbing osd.7 2024-02-26T10:43:19.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-02-26T10:43:19.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.305+0000 7f32eae00640 1 Processor -- start 2024-02-26T10:43:19.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.305+0000 7f32eae00640 1 -- start start 2024-02-26T10:43:19.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.306+0000 7f32eae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f32e41503a0 0x7f32e40ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.306+0000 7f32eae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e4150780 con 0x7f32e40a9430 2024-02-26T10:43:19.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.306+0000 7f32eae00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40a98c0 con 0x7f32e406e2d0 2024-02-26T10:43:19.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.306+0000 7f32eae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e40a9ba0 con 0x7f32e41503a0 2024-02-26T10:43:19.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.307+0000 7f32e9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f32e41503a0 0x7f32e40ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.307+0000 7f32e9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f32e41503a0 0x7f32e40ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44312/0 (socket says 172.21.15.67:44312) 2024-02-26T10:43:19.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.307+0000 7f32e9a00640 1 -- 172.21.15.67:0/2784954969 learned_addr learned my addr 172.21.15.67:0/2784954969 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:19.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.307+0000 7f32e9a00640 1 -- 172.21.15.67:0/2784954969 >> v1:172.21.15.82:6789/0 conn(0x7f32e406e2d0 legacy=0x7f32e40c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:19.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.307+0000 7f32e9a00640 1 -- 172.21.15.67:0/2784954969 >> v1:172.21.15.67:6789/0 conn(0x7f32e40a9430 legacy=0x7f32e41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.307+0000 7f32e9a00640 1 -- 172.21.15.67:0/2784954969 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40a9e80 con 0x7f32e41503a0 2024-02-26T10:43:19.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.308+0000 7f32e9a00640 1 --2- 172.21.15.67:0/2784954969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41503a0 0x7f32e40ab480 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009870 tx=0x7f32d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ff8a91bfca825 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.308+0000 7f32e3e00640 1 -- 172.21.15.67:0/2784954969 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8034070 con 0x7f32e41503a0 2024-02-26T10:43:19.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.308+0000 7f32e3e00640 1 -- 172.21.15.67:0/2784954969 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d8030860 con 0x7f32e41503a0 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.308+0000 7f32e3e00640 1 -- 172.21.15.67:0/2784954969 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8030c30 con 0x7f32e41503a0 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 -- 172.21.15.67:0/2784954969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41503a0 msgr2=0x7f32e40ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 --2- 172.21.15.67:0/2784954969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41503a0 0x7f32e40ab480 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009870 tx=0x7f32d802b9a0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 -- 172.21.15.67:0/2784954969 shutdown_connections 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 --2- 172.21.15.67:0/2784954969 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41503a0 0x7f32e40ab480 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 -- 172.21.15.67:0/2784954969 >> 172.21.15.67:0/2784954969 conn(0x7f32e40c6380 msgr2=0x7f32e40c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 -- 172.21.15.67:0/2784954969 shutdown_connections 2024-02-26T10:43:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.309+0000 7f32eae00640 1 -- 172.21.15.67:0/2784954969 wait complete. 2024-02-26T10:43:19.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.311+0000 7f32eae00640 1 Processor -- start 2024-02-26T10:43:19.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.311+0000 7f32eae00640 1 -- start start 2024-02-26T10:43:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32eae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32e406e2d0 0x7f32e4149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32eae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32e41503a0 0x7f32e41ee160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32eae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41eb160 0x7f32e41ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32eae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40c0f80 con 0x7f32e41eb160 2024-02-26T10:43:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32eae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e406ddb0 con 0x7f32e41503a0 2024-02-26T10:43:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32eae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40c3520 con 0x7f32e406e2d0 2024-02-26T10:43:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41eb160 0x7f32e41ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41eb160 0x7f32e41ee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44318/0 (socket says 172.21.15.67:44318) 2024-02-26T10:43:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 -- 172.21.15.67:0/1768991807 learned_addr learned my addr 172.21.15.67:0/1768991807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32e9000640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32e41503a0 0x7f32e41ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32e9a00640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32e406e2d0 0x7f32e4149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 -- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32e406e2d0 msgr2=0x7f32e4149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32e406e2d0 0x7f32e4149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 -- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32e41503a0 msgr2=0x7f32e41ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32e41503a0 0x7f32e41ee160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.312+0000 7f32ea400640 1 -- 172.21.15.67:0/1768991807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e4072c30 con 0x7f32e41eb160 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.313+0000 7f32ea400640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41eb160 0x7f32e41ee6a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00c780 tx=0x7f32dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.313+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x7f32e41eb160 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.313+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc003da0 con 0x7f32e41eb160 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.313+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e41eebe0 con 0x7f32e41eb160 2024-02-26T10:43:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.313+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0052b0 con 0x7f32e41eb160 2024-02-26T10:43:19.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.313+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e40bfa20 con 0x7f32e41eb160 2024-02-26T10:43:19.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.314+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32e4074a30 con 0x7f32e41eb160 2024-02-26T10:43:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.315+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32dc01a070 con 0x7f32e41eb160 2024-02-26T10:43:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.316+0000 7f32e1600640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f32c406b4d0 0x7f32c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.316+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f32dc0895f0 con 0x7f32e41eb160 2024-02-26T10:43:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.316+0000 7f32e1600640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7f32c4071430 0x7f32c4073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.316+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 --> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32dc015e70 con 0x7f32c4071430 2024-02-26T10:43:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.317+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32dc090050 con 0x7f32e41eb160 2024-02-26T10:43:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.317+0000 7f32e9a00640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f32c406b4d0 0x7f32c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.317+0000 7f32e9000640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7f32c4071430 0x7f32c4073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.318+0000 7f32e9000640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7f32c4071430 0x7f32c4073850 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.318+0000 7f32e9a00640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f32c406b4d0 0x7f32c406d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f32d8036000 tx=0x7f32d803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.320+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== osd.7 v2:172.21.15.82:6824/3253207267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f32dc015e70 con 0x7f32c4071430 2024-02-26T10:43:19.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.337+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 --> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f32e40c1650 con 0x7f32c4071430 2024-02-26T10:43:19.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32e1600640 1 -- 172.21.15.67:0/1768991807 <== osd.7 v2:172.21.15.82:6824/3253207267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f32e40c1650 con 0x7f32c4071430 2024-02-26T10:43:19.336 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-26T10:43:19.336 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-02-26T10:43:19.336 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-26T10:43:19.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7f32c4071430 msgr2=0x7f32c4073850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32eae00640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6824/3253207267,v1:172.21.15.82:6825/3253207267] conn(0x7f32c4071430 0x7f32c4073850 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f32c406b4d0 msgr2=0x7f32c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32eae00640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f32c406b4d0 0x7f32c406d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f32d8036000 tx=0x7f32d803b040 comp rx=0 tx=0).stop 2024-02-26T10:43:19.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41eb160 msgr2=0x7f32e41ee6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32eae00640 1 --2- 172.21.15.67:0/1768991807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32e41eb160 0x7f32e41ee6a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00c780 tx=0x7f32dc00cc40 comp rx=0 tx=0).stop 2024-02-26T10:43:19.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.338+0000 7f32ea400640 1 -- 172.21.15.67:0/1768991807 reap_dead start 2024-02-26T10:43:19.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.339+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 shutdown_connections 2024-02-26T10:43:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.339+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 >> 172.21.15.67:0/1768991807 conn(0x7f32e40c6380 msgr2=0x7f32e414c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.339+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 shutdown_connections 2024-02-26T10:43:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.339+0000 7f32eae00640 1 -- 172.21.15.67:0/1768991807 wait complete. 2024-02-26T10:43:19.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-02-26T10:43:19.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.577+0000 7f3bba000640 1 Processor -- start 2024-02-26T10:43:19.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.577+0000 7f3bba000640 1 -- start start 2024-02-26T10:43:19.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.578+0000 7f3bba000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bb40a94a0 0x7f3bb406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.578+0000 7f3bba000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb40abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.578+0000 7f3bba000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb406df90 con 0x7f3bb406ac20 2024-02-26T10:43:19.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.578+0000 7f3bba000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb40a9a80 con 0x7f3bb40a94a0 2024-02-26T10:43:19.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.578+0000 7f3bba000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40a9d60 con 0x7f3bb406db00 2024-02-26T10:43:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.579+0000 7f3bb9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb40abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.579+0000 7f3bb8c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3bb406ac20 0x7f3bb4153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:60764/0 (socket says 172.21.15.67:60764) 2024-02-26T10:43:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.579+0000 7f3bb8c00640 1 -- 172.21.15.67:0/4020437425 learned_addr learned my addr 172.21.15.67:0/4020437425 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.579+0000 7f3bb3e00640 1 --2- 172.21.15.67:0/4020437425 >> v2:172.21.15.82:3300/0 conn(0x7f3bb40a94a0 0x7f3bb406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.580+0000 7f3bb9600640 1 -- 172.21.15.67:0/4020437425 >> v2:172.21.15.82:3300/0 conn(0x7f3bb40a94a0 msgr2=0x7f3bb406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.580+0000 7f3bb9600640 1 --2- 172.21.15.67:0/4020437425 >> v2:172.21.15.82:3300/0 conn(0x7f3bb40a94a0 0x7f3bb406b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.580+0000 7f3bb9600640 1 -- 172.21.15.67:0/4020437425 >> v1:172.21.15.82:6789/0 conn(0x7f3bb406ac20 legacy=0x7f3bb4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.580+0000 7f3bb9600640 1 -- 172.21.15.67:0/4020437425 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40ab750 con 0x7f3bb406db00 2024-02-26T10:43:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.581+0000 7f3bb9600640 1 --2- 172.21.15.67:0/4020437425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb40abf50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0097a0 tx=0x7f3bac02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344a6b01b245ad84 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.582+0000 7f3bb3400640 1 -- 172.21.15.67:0/4020437425 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac035020 con 0x7f3bb406db00 2024-02-26T10:43:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.582+0000 7f3bb3400640 1 -- 172.21.15.67:0/4020437425 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac031810 con 0x7f3bb406db00 2024-02-26T10:43:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.582+0000 7f3bb3400640 1 -- 172.21.15.67:0/4020437425 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac031be0 con 0x7f3bb406db00 2024-02-26T10:43:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 -- 172.21.15.67:0/4020437425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 msgr2=0x7f3bb40abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 --2- 172.21.15.67:0/4020437425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb40abf50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0097a0 tx=0x7f3bac02b8b0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 -- 172.21.15.67:0/4020437425 shutdown_connections 2024-02-26T10:43:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 --2- 172.21.15.67:0/4020437425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb40abf50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 --2- 172.21.15.67:0/4020437425 >> v2:172.21.15.82:3300/0 conn(0x7f3bb40a94a0 0x7f3bb406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 -- 172.21.15.67:0/4020437425 >> 172.21.15.67:0/4020437425 conn(0x7f3bb40c63f0 msgr2=0x7f3bb40c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 -- 172.21.15.67:0/4020437425 shutdown_connections 2024-02-26T10:43:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.583+0000 7f3bba000640 1 -- 172.21.15.67:0/4020437425 wait complete. 2024-02-26T10:43:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.585+0000 7f3bba000640 1 Processor -- start 2024-02-26T10:43:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.585+0000 7f3bba000640 1 -- start start 2024-02-26T10:43:19.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bba000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 0x7f3bb4072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bba000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb4147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bba000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb40a94a0 0x7f3bb4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bba000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb4150410 con 0x7f3bb406ac20 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bba000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40ac490 con 0x7f3bb40a94a0 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bba000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb4150590 con 0x7f3bb406db00 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bb8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 0x7f3bb4072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bb8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 0x7f3bb4072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34348/0 (socket says 172.21.15.67:34348) 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bb8c00640 1 -- 172.21.15.67:0/1846134683 learned_addr learned my addr 172.21.15.67:0/1846134683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bb3e00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb4147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.586+0000 7f3bb9600640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb40a94a0 0x7f3bb4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb8c00640 1 -- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 msgr2=0x7f3bb4147f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb8c00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb4147f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb8c00640 1 -- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb40a94a0 msgr2=0x7f3bb4156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb8c00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb40a94a0 0x7f3bb4156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb8c00640 1 -- 172.21.15.67:0/1846134683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40c5370 con 0x7f3bb406ac20 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb3e00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb4147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb9600640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb40a94a0 0x7f3bb4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb8c00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 0x7f3bb4072e80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400cbd0 tx=0x7f3ba400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4017070 con 0x7f3bb406ac20 2024-02-26T10:43:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba400ad40 con 0x7f3bb406ac20 2024-02-26T10:43:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4012600 con 0x7f3bb406ac20 2024-02-26T10:43:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.587+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb4147cd0 con 0x7f3bb406ac20 2024-02-26T10:43:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.588+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb41485d0 con 0x7f3bb406ac20 2024-02-26T10:43:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.589+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb4074a30 con 0x7f3bb406ac20 2024-02-26T10:43:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.590+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ba401a070 con 0x7f3bb406ac20 2024-02-26T10:43:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.590+0000 7f3bb0c00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3b9006b4d0 0x7f3b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.591+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3ba4089de0 con 0x7f3bb406ac20 2024-02-26T10:43:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.591+0000 7f3bb3e00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3b9006b4d0 0x7f3b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:19.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.592+0000 7f3bb3e00640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3b9006b4d0 0x7f3b9006d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4156000 tx=0x7f3b9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:19.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.595+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bb40c5370 con 0x7f3bb406ac20 2024-02-26T10:43:19.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.797+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bb406b9b0 con 0x7f3b9006b4d0 2024-02-26T10:43:19.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.798+0000 7f3bb0c00640 1 -- 172.21.15.67:0/1846134683 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3bb406b9b0 con 0x7f3b9006b4d0 2024-02-26T10:43:19.795 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 7 to deep-scrub 2024-02-26T10:43:19.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3b9006b4d0 msgr2=0x7f3b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3b9006b4d0 0x7f3b9006d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4156000 tx=0x7f3b9c008040 comp rx=0 tx=0).stop 2024-02-26T10:43:19.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 msgr2=0x7f3bb4072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 0x7f3bb4072e80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400cbd0 tx=0x7f3ba400a430 comp rx=0 tx=0).stop 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 shutdown_connections 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f3b9006b4d0 0x7f3b9006d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb40a94a0 0x7f3bb4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3bb406db00 0x7f3bb4147f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 --2- 172.21.15.67:0/1846134683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bb406ac20 0x7f3bb4072e80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 >> 172.21.15.67:0/1846134683 conn(0x7f3bb40c63f0 msgr2=0x7f3bb414f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 shutdown_connections 2024-02-26T10:43:19.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:19.801+0000 7f3bba000640 1 -- 172.21.15.67:0/1846134683 wait complete. 2024-02-26T10:43:19.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:20.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.038+0000 7fc98a200640 1 Processor -- start 2024-02-26T10:43:20.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.038+0000 7fc98a200640 1 -- start start 2024-02-26T10:43:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc98a200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc9840c4a50 0x7fc9840c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc98a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9840c2770 0x7fc984155340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc98a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc984159130 con 0x7fc9841581f0 2024-02-26T10:43:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc98a200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc984154eb0 con 0x7fc9840c4a50 2024-02-26T10:43:20.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc98a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc984070010 con 0x7fc9840c2770 2024-02-26T10:43:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc989800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc9841581f0 0x7fc9840aa020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33390/0 (socket says 172.21.15.67:33390) 2024-02-26T10:43:20.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.040+0000 7fc989800640 1 -- 172.21.15.67:0/200806278 learned_addr learned my addr 172.21.15.67:0/200806278 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc988e00640 1 --2- 172.21.15.67:0/200806278 >> v2:172.21.15.82:3300/0 conn(0x7fc9840c4a50 0x7fc9840c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983e00640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9840c2770 0x7fc984155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138466548 0 0) 0x7fc984159130 con 0x7fc9841581f0 2024-02-26T10:43:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc96c003520 con 0x7fc9841581f0 2024-02-26T10:43:20.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1061364488 0 0) 0x7fc96c003520 con 0x7fc9841581f0 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9840c2770 msgr2=0x7fc984155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9840c2770 0x7fc984155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 >> v2:172.21.15.82:3300/0 conn(0x7fc9840c4a50 msgr2=0x7fc9840c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 --2- 172.21.15.67:0/200806278 >> v2:172.21.15.82:3300/0 conn(0x7fc9840c4a50 0x7fc9840c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840c6bf0 con 0x7fc9841581f0 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fc97c0019f0 con 0x7fc9841581f0 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 >> v1:172.21.15.67:6789/0 conn(0x7fc9841581f0 legacy=0x7fc9840aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.041+0000 7fc983400640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c0039b0 0x7fc96c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983400640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc96c004310 0x7fc96c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983400640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc96c004e50 0x7fc96c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc984154eb0 con 0x7fc96c004e50 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc984070010 con 0x7fc96c004310 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c00bde0 con 0x7fc96c0039b0 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc989800640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc96c004e50 0x7fc96c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983e00640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc96c004310 0x7fc96c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc988e00640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c0039b0 0x7fc96c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc989800640 1 -- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c0039b0 msgr2=0x7fc96c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:20.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc989800640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c0039b0 0x7fc96c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc989800640 1 -- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc96c004310 msgr2=0x7fc96c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc989800640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc96c004310 0x7fc96c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc989800640 1 -- 172.21.15.67:0/200806278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c003520 con 0x7fc96c004e50 2024-02-26T10:43:20.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.042+0000 7fc983e00640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc96c004310 0x7fc96c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:20.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.043+0000 7fc989800640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc96c004e50 0x7fc96c0052d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0013d0 tx=0x7fc96c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b80384ceff750c server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:20.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c00a4e0 con 0x7fc96c004e50 2024-02-26T10:43:20.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c034650 con 0x7fc96c004e50 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc983400640 1 -- 172.21.15.67:0/200806278 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c034a40 con 0x7fc96c004e50 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc98a200640 1 -- 172.21.15.67:0/200806278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc96c004e50 msgr2=0x7fc96c0052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc98a200640 1 --2- 172.21.15.67:0/200806278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc96c004e50 0x7fc96c0052d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0013d0 tx=0x7fc96c005830 comp rx=0 tx=0).stop 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc989800640 1 -- 172.21.15.67:0/200806278 reap_dead start 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc98a200640 1 -- 172.21.15.67:0/200806278 shutdown_connections 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc98a200640 1 -- 172.21.15.67:0/200806278 >> 172.21.15.67:0/200806278 conn(0x7fc98406f8e0 msgr2=0x7fc98406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc98a200640 1 -- 172.21.15.67:0/200806278 shutdown_connections 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.044+0000 7fc98a200640 1 -- 172.21.15.67:0/200806278 wait complete. 2024-02-26T10:43:20.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.046+0000 7fc98a200640 1 Processor -- start 2024-02-26T10:43:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.046+0000 7fc98a200640 1 -- start start 2024-02-26T10:43:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc98a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c004e50 0x7fc9840adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc98a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9841e9c00 0x7fc984153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc98a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 0x7fc98414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc98a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc984152660 con 0x7fc9841541b0 2024-02-26T10:43:20.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc98a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840c71b0 con 0x7fc9841e9c00 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc98a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840c1710 con 0x7fc96c004e50 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.047+0000 7fc989800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 0x7fc98414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 0x7fc98414d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34370/0 (socket says 172.21.15.67:34370) 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 -- 172.21.15.67:0/3883726114 learned_addr learned my addr 172.21.15.67:0/3883726114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc988e00640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c004e50 0x7fc9840adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc983e00640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9841e9c00 0x7fc984153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 -- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c004e50 msgr2=0x7fc9840adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c004e50 0x7fc9840adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 -- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9841e9c00 msgr2=0x7fc984153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9841e9c00 0x7fc984153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 -- 172.21.15.67:0/3883726114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840ae090 con 0x7fc9841541b0 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.048+0000 7fc989800640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 0x7fc98414d170 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc97c000f80 tx=0x7fc97c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.049+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c043070 con 0x7fc9841541b0 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.049+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c0019b0 con 0x7fc9841541b0 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.049+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98414dcc0 con 0x7fc9841541b0 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.049+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c039040 con 0x7fc9841541b0 2024-02-26T10:43:20.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.049+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9840a8ce0 con 0x7fc9841541b0 2024-02-26T10:43:20.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.050+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984074a30 con 0x7fc9841541b0 2024-02-26T10:43:20.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.051+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc97c046070 con 0x7fc9841541b0 2024-02-26T10:43:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.052+0000 7fc980c00640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc96406b500 0x7fc96406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.052+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc97c0ab360 con 0x7fc9841541b0 2024-02-26T10:43:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.053+0000 7fc988e00640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc96406b500 0x7fc96406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:20.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.054+0000 7fc988e00640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc96406b500 0x7fc96406d9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc984155b40 tx=0x7fc978008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:20.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.056+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc97c07a3f0 con 0x7fc9841541b0 2024-02-26T10:43:20.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.253+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9840706c0 con 0x7fc96406b500 2024-02-26T10:43:20.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.256+0000 7fc980c00640 1 -- 172.21.15.67:0/3883726114 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13375 (secure 0 0 0) 0x7fc9840706c0 con 0x7fc96406b500 2024-02-26T10:43:20.253 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:20.254 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:43:20.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc96406b500 msgr2=0x7fc96406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc96406b500 0x7fc96406d9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc984155b40 tx=0x7fc978008040 comp rx=0 tx=0).stop 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 msgr2=0x7fc98414d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 0x7fc98414d170 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc97c000f80 tx=0x7fc97c001360 comp rx=0 tx=0).stop 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 shutdown_connections 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9841541b0 0x7fc98414d170 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fc96406b500 0x7fc96406d9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9841e9c00 0x7fc984153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 --2- 172.21.15.67:0/3883726114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc96c004e50 0x7fc9840adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 >> 172.21.15.67:0/3883726114 conn(0x7fc98406f8e0 msgr2=0x7fc9840c1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 shutdown_connections 2024-02-26T10:43:20.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:20.259+0000 7fc98a200640 1 -- 172.21.15.67:0/3883726114 wait complete. 2024-02-26T10:43:20.277 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-02-26T10:43:18.435863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":258980,"kb_used_data":9512,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715740,"statfs":{"total":773094113280,"available":772828917760,"internally_reserved":0,"allocated":9740288,"data_stored":5026480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-138,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003289"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":295,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T10:43:09.569456+0000","last_change":"2024-02-26T10:38:30.100877+0000","last_active":"2024-02-26T10:43:09.569456+0000","last_peered":"2024-02-26T10:43:09.569456+0000","last_clean":"2024-02-26T10:43:09.569456+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:43:09.569456+0000","last_undegraded":"2024-02-26T10:43:09.569456+0000","last_fullsized":"2024-02-26T10:43:09.569456+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-26T10:38:28.592309+0000","last_clean_scrub_stamp":"2024-02-26T10:38:28.592309+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T14:42:23.274695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":1071086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1662976,"data_stored":1071086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:20.279 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-02-26T10:38:28.592309+0000 time.struct_time(tm_year=2024, tm_mon=2, tm_mday=26, tm_hour=10, tm_min=38, tm_sec=28, tm_wday=0, tm_yday=57, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=2, tm_mday=26, tm_hour=10, tm_min=43, tm_sec=12, tm_wday=0, tm_yday=57, tm_isdst=0) 2024-02-26T10:43:20.279 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-02-26T10:43:31.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:31.194 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:40.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-26T10:43:40.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.513+0000 7f581b200640 1 Processor -- start 2024-02-26T10:43:40.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.514+0000 7f581b200640 1 -- start start 2024-02-26T10:43:40.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f581b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f581c06ac20 0x7f581c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f581b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f581c06db00 0x7f581c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f581b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c06b460 con 0x7f581c06b000 2024-02-26T10:43:40.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f581b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0a9a80 con 0x7f581c06ac20 2024-02-26T10:43:40.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f581b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0a9d60 con 0x7f581c06db00 2024-02-26T10:43:40.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f5819e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f581c06ac20 0x7f581c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f5819e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f581c06ac20 0x7f581c0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42862/0 (socket says 172.21.15.67:42862) 2024-02-26T10:43:40.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.515+0000 7f5819e00640 1 -- 172.21.15.67:0/2094453313 learned_addr learned my addr 172.21.15.67:0/2094453313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:40.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.516+0000 7f5819e00640 1 -- 172.21.15.67:0/2094453313 >> v2:172.21.15.82:3300/0 conn(0x7f581c06db00 msgr2=0x7f581c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:40.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.516+0000 7f5819e00640 1 --2- 172.21.15.67:0/2094453313 >> v2:172.21.15.82:3300/0 conn(0x7f581c06db00 0x7f581c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.516+0000 7f5819e00640 1 -- 172.21.15.67:0/2094453313 >> v1:172.21.15.82:6789/0 conn(0x7f581c06b000 legacy=0x7f581c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:40.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.516+0000 7f5819e00640 1 -- 172.21.15.67:0/2094453313 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c06df90 con 0x7f581c06ac20 2024-02-26T10:43:40.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.517+0000 7f5819e00640 1 --2- 172.21.15.67:0/2094453313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06ac20 0x7f581c0a94a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f581c157220 tx=0x7f580c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24d46fc75ac2c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.517+0000 7f5819400640 1 -- 172.21.15.67:0/2094453313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c035020 con 0x7f581c06ac20 2024-02-26T10:43:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.517+0000 7f5819400640 1 -- 172.21.15.67:0/2094453313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c031870 con 0x7f581c06ac20 2024-02-26T10:43:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.517+0000 7f5819400640 1 -- 172.21.15.67:0/2094453313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c031c40 con 0x7f581c06ac20 2024-02-26T10:43:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 -- 172.21.15.67:0/2094453313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06ac20 msgr2=0x7f581c0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 --2- 172.21.15.67:0/2094453313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06ac20 0x7f581c0a94a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f581c157220 tx=0x7f580c02b910 comp rx=0 tx=0).stop 2024-02-26T10:43:40.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 -- 172.21.15.67:0/2094453313 shutdown_connections 2024-02-26T10:43:40.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 --2- 172.21.15.67:0/2094453313 >> v2:172.21.15.82:3300/0 conn(0x7f581c06db00 0x7f581c0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 --2- 172.21.15.67:0/2094453313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06ac20 0x7f581c0a94a0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 -- 172.21.15.67:0/2094453313 >> 172.21.15.67:0/2094453313 conn(0x7f581c0c63f0 msgr2=0x7f581c0c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:40.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 -- 172.21.15.67:0/2094453313 shutdown_connections 2024-02-26T10:43:40.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.518+0000 7f581b200640 1 -- 172.21.15.67:0/2094453313 wait complete. 2024-02-26T10:43:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.520+0000 7f581b200640 1 Processor -- start 2024-02-26T10:43:40.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.520+0000 7f581b200640 1 -- start start 2024-02-26T10:43:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f581b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f581c06ac20 0x7f581c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f581b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 0x7f581c1eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f581b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f581c06db00 0x7f581c0bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f581b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0ac490 con 0x7f581c06b000 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f581b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0aba30 con 0x7f581c06db00 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f581b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c150410 con 0x7f581c06ac20 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f5813e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 0x7f581c1eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f5813e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 0x7f581c1eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42878/0 (socket says 172.21.15.67:42878) 2024-02-26T10:43:40.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.521+0000 7f5813e00640 1 -- 172.21.15.67:0/2367777937 learned_addr learned my addr 172.21.15.67:0/2367777937 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5819e00640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f581c06ac20 0x7f581c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f581a800640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f581c06db00 0x7f581c0bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5813e00640 1 -- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f581c06ac20 msgr2=0x7f581c0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5813e00640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f581c06ac20 0x7f581c0715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5813e00640 1 -- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f581c06db00 msgr2=0x7f581c0bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5813e00640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f581c06db00 0x7f581c0bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5813e00640 1 -- 172.21.15.67:0/2367777937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c1de9f0 con 0x7f581c06b000 2024-02-26T10:43:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f581a800640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f581c06db00 0x7f581c0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5813e00640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 0x7f581c1eb4c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f580400c400 tx=0x7f580400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804017070 con 0x7f581c06b000 2024-02-26T10:43:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58040035f0 con 0x7f581c06b000 2024-02-26T10:43:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.522+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58040039e0 con 0x7f581c06b000 2024-02-26T10:43:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.523+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c1ddc30 con 0x7f581c06b000 2024-02-26T10:43:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.523+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c1de1f0 con 0x7f581c06b000 2024-02-26T10:43:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.524+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c074a30 con 0x7f581c06b000 2024-02-26T10:43:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.525+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5804003b80 con 0x7f581c06b000 2024-02-26T10:43:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.526+0000 7f5812000640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57f806b4d0 0x7f57f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.526+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5804089270 con 0x7f581c06b000 2024-02-26T10:43:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.526+0000 7f5819e00640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57f806b4d0 0x7f57f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.527+0000 7f5819e00640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57f806b4d0 0x7f57f806d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f580c036000 tx=0x7f580c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:40.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.530+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58040010c0 con 0x7f581c06b000 2024-02-26T10:43:40.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.726+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 --> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f581c06b9b0 con 0x7f57f806b4d0 2024-02-26T10:43:40.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.728+0000 7f5812000640 1 -- 172.21.15.67:0/2367777937 <== mgr.4106 v2:172.21.15.82:6832/723923815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13374 (secure 0 0 0) 0x7f581c06b9b0 con 0x7f57f806b4d0 2024-02-26T10:43:40.725 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-26T10:43:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57f806b4d0 msgr2=0x7f57f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57f806b4d0 0x7f57f806d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f580c036000 tx=0x7f580c03c040 comp rx=0 tx=0).stop 2024-02-26T10:43:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 msgr2=0x7f581c1eb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 0x7f581c1eb4c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f580400c400 tx=0x7f580400c8c0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 shutdown_connections 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f57f806b4d0 0x7f57f806d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f581c06db00 0x7f581c0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f581c06b000 0x7f581c1eb4c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 --2- 172.21.15.67:0/2367777937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f581c06ac20 0x7f581c0715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 >> 172.21.15.67:0/2367777937 conn(0x7f581c0c63f0 msgr2=0x7f581c14b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 shutdown_connections 2024-02-26T10:43:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.731+0000 7f581b200640 1 -- 172.21.15.67:0/2367777937 wait complete. 2024-02-26T10:43:40.748 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-02-26T10:43:40.442042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":258964,"kb_used_data":9512,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715756,"statfs":{"total":773094113280,"available":772828934144,"internally_reserved":0,"allocated":9740288,"data_stored":5026480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003381"},"pg_stats":[{"pgid":"1.0","version":"86'119","reported_seq":306,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-26T10:43:20.286309+0000","last_change":"2024-02-26T10:43:20.286490+0000","last_active":"2024-02-26T10:43:20.286309+0000","last_peered":"2024-02-26T10:43:20.286309+0000","last_clean":"2024-02-26T10:43:20.286309+0000","last_became_active":"2024-02-26T10:38:30.097724+0000","last_became_peered":"2024-02-26T10:38:30.097724+0000","last_unstale":"2024-02-26T10:43:20.286309+0000","last_undegraded":"2024-02-26T10:43:20.286309+0000","last_fullsized":"2024-02-26T10:43:20.286309+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'119","last_scrub_stamp":"2024-02-26T10:43:20.286059+0000","last_deep_scrub":"86'119","last_deep_scrub_stamp":"2024-02-26T10:43:20.286059+0000","last_clean_scrub_stamp":"2024-02-26T10:43:20.286059+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-27T12:11:51.117525+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542209,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1662976,"data_stored":1071086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542210,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542210,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542210,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1069056,"data_stored":480718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1662976,"data_stored":1071086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-26T10:43:40.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-02-26T10:43:40.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.964+0000 7f3085800640 1 Processor -- start 2024-02-26T10:43:40.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.964+0000 7f3085800640 1 -- start start 2024-02-26T10:43:40.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f3085800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30800c2950 0x7f30801533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f3085800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3080153920 0x7f308015d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f3085800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30800a6eb0 con 0x7f30801503d0 2024-02-26T10:43:40.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f3085800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30800a7190 con 0x7f3080153920 2024-02-26T10:43:40.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f3085800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30800a7470 con 0x7f30800c2950 2024-02-26T10:43:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f3076000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30800c2950 0x7f30801533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f307ea00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f30801503d0 0x7f308006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.67:53580/0 (socket says 172.21.15.67:53580) 2024-02-26T10:43:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.966+0000 7f307ea00640 1 -- 172.21.15.67:0/2549771095 learned_addr learned my addr 172.21.15.67:0/2549771095 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.967+0000 7f307f400640 1 --2- 172.21.15.67:0/2549771095 >> v2:172.21.15.82:3300/0 conn(0x7f3080153920 0x7f308015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.967+0000 7f3076000640 1 -- 172.21.15.67:0/2549771095 >> v2:172.21.15.82:3300/0 conn(0x7f3080153920 msgr2=0x7f308015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:40.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.967+0000 7f3076000640 1 --2- 172.21.15.67:0/2549771095 >> v2:172.21.15.82:3300/0 conn(0x7f3080153920 0x7f308015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.967+0000 7f3076000640 1 -- 172.21.15.67:0/2549771095 >> v1:172.21.15.82:6789/0 conn(0x7f30801503d0 legacy=0x7f308006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.967+0000 7f3076000640 1 -- 172.21.15.67:0/2549771095 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800a7750 con 0x7f30800c2950 2024-02-26T10:43:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.968+0000 7f3076000640 1 --2- 172.21.15.67:0/2549771095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30800c2950 0x7f30801533e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30680097a0 tx=0x7f306802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8f0896c8be70b server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f307e000640 1 -- 172.21.15.67:0/2549771095 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068039070 con 0x7f30800c2950 2024-02-26T10:43:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f307e000640 1 -- 172.21.15.67:0/2549771095 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30680316f0 con 0x7f30800c2950 2024-02-26T10:43:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f307e000640 1 -- 172.21.15.67:0/2549771095 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3068031ac0 con 0x7f30800c2950 2024-02-26T10:43:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 -- 172.21.15.67:0/2549771095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30800c2950 msgr2=0x7f30801533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 --2- 172.21.15.67:0/2549771095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30800c2950 0x7f30801533e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30680097a0 tx=0x7f306802b790 comp rx=0 tx=0).stop 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 -- 172.21.15.67:0/2549771095 shutdown_connections 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 --2- 172.21.15.67:0/2549771095 >> v2:172.21.15.82:3300/0 conn(0x7f3080153920 0x7f308015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 --2- 172.21.15.67:0/2549771095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30800c2950 0x7f30801533e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 -- 172.21.15.67:0/2549771095 >> 172.21.15.67:0/2549771095 conn(0x7f30800c6da0 msgr2=0x7f30800c2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 -- 172.21.15.67:0/2549771095 shutdown_connections 2024-02-26T10:43:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.969+0000 7f3085800640 1 -- 172.21.15.67:0/2549771095 wait complete. 2024-02-26T10:43:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.971+0000 7f3085800640 1 Processor -- start 2024-02-26T10:43:40.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.971+0000 7f3085800640 1 -- start start 2024-02-26T10:43:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f3085800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 0x7f30800ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f3085800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30801503d0 0x7f308014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f3085800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3080153920 0x7f308014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f3085800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30801507b0 con 0x7f30800c2950 2024-02-26T10:43:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f3085800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308006ef00 con 0x7f30801503d0 2024-02-26T10:43:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f3085800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308006e560 con 0x7f3080153920 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f307ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 0x7f30800ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f307ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 0x7f30800ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42886/0 (socket says 172.21.15.67:42886) 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.972+0000 7f307ea00640 1 -- 172.21.15.67:0/3160710286 learned_addr learned my addr 172.21.15.67:0/3160710286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f3076000640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30801503d0 0x7f308014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307f400640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3080153920 0x7f308014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307ea00640 1 -- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3080153920 msgr2=0x7f308014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307ea00640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3080153920 0x7f308014b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307ea00640 1 -- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30801503d0 msgr2=0x7f308014cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307ea00640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30801503d0 0x7f308014cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307ea00640 1 -- 172.21.15.67:0/3160710286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308014a710 con 0x7f30800c2950 2024-02-26T10:43:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307f400640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3080153920 0x7f308014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.973+0000 7f307ea00640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 0x7f30800ac020 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f307000c3c0 tx=0x7f307000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.974+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070017070 con 0x7f30800c2950 2024-02-26T10:43:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.974+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070005ce0 con 0x7f30800c2950 2024-02-26T10:43:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.974+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700072f0 con 0x7f30800c2950 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.974+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30801eb780 con 0x7f30800c2950 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.974+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308014ac90 con 0x7f30800c2950 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.975+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3080074a30 con 0x7f30800c2950 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.976+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f307001a070 con 0x7f30800c2950 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.977+0000 7f3077400640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f305c06b4d0 0x7f305c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.977+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3070089390 con 0x7f30800c2950 2024-02-26T10:43:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.977+0000 7f3076000640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f305c06b4d0 0x7f305c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.978+0000 7f3076000640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f305c06b4d0 0x7f305c06d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3068036000 tx=0x7f3068008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:40.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:40.981+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3070058420 con 0x7f30800c2950 2024-02-26T10:43:41.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.178+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f308006ec50 con 0x7f30800c2950 2024-02-26T10:43:41.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.184+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3070057d20 con 0x7f30800c2950 2024-02-26T10:43:41.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.186+0000 7f3077400640 1 -- 172.21.15.67:0/3160710286 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f307005be70 con 0x7f30800c2950 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.188+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f305c06b4d0 msgr2=0x7f305c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.188+0000 7f3085800640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f305c06b4d0 0x7f305c06d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3068036000 tx=0x7f3068008030 comp rx=0 tx=0).stop 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.188+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 msgr2=0x7f30800ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.188+0000 7f3085800640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 0x7f30800ac020 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f307000c3c0 tx=0x7f307000c880 comp rx=0 tx=0).stop 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 shutdown_connections 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3080153920 0x7f308014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7f305c06b4d0 0x7f305c06d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30801503d0 0x7f308014cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 --2- 172.21.15.67:0/3160710286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30800c2950 0x7f30800ac020 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 >> 172.21.15.67:0/3160710286 conn(0x7f30800c6da0 msgr2=0x7f308014ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:41.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 shutdown_connections 2024-02-26T10:43:41.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.189+0000 7f3085800640 1 -- 172.21.15.67:0/3160710286 wait complete. 2024-02-26T10:43:41.204 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-26T10:43:41.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-26T10:43:41.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.427+0000 7fe4b8200640 1 Processor -- start 2024-02-26T10:43:41.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.427+0000 7fe4b8200640 1 -- start start 2024-02-26T10:43:41.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.428+0000 7fe4b8200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 0x7fe4b0071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.428+0000 7fe4b8200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b0155c00 con 0x7fe4b00c2870 2024-02-26T10:43:41.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.428+0000 7fe4b8200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b0155ee0 con 0x7fe4b014f3d0 2024-02-26T10:43:41.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.428+0000 7fe4b8200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b006dd80 con 0x7fe4b0071700 2024-02-26T10:43:41.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.429+0000 7fe4b4e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe4b00c2870 0x7fe4b0155160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58474/0 (socket says 172.21.15.67:58474) 2024-02-26T10:43:41.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.429+0000 7fe4b4e00640 1 -- 172.21.15.67:0/2988273374 learned_addr learned my addr 172.21.15.67:0/2988273374 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-02-26T10:43:41.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.429+0000 7fe4b6200640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 0x7fe4b0071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.429+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954493465 0 0) 0x7fe4b0155c00 con 0x7fe4b00c2870 2024-02-26T10:43:41.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.429+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe498003520 con 0x7fe4b00c2870 2024-02-26T10:43:41.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3267640412 0 0) 0x7fe498003520 con 0x7fe4b00c2870 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 msgr2=0x7fe4b0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 0x7fe4b0071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 >> v1:172.21.15.82:6789/0 conn(0x7fe4b014f3d0 legacy=0x7fe4b006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b006dfd0 con 0x7fe4b00c2870 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262437680 0 0) 0x7fe4a00019f0 con 0x7fe4b00c2870 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 >> v1:172.21.15.67:6789/0 conn(0x7fe4b00c2870 legacy=0x7fe4b0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4980039e0 0x7fe498003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 0x7fe498004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe498004d40 0x7fe4980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b006dd80 con 0x7fe4980039e0 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49800dd00 con 0x7fe498004d40 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49800df00 con 0x7fe4b0071700 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4980039e0 0x7fe498003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b4e00640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 0x7fe498004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b5800640 1 -- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 msgr2=0x7fe498004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b0071700 0x7fe498004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b5800640 1 -- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe498004d40 msgr2=0x7fe4980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-26T10:43:41.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe498004d40 0x7fe4980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.430+0000 7fe4b5800640 1 -- 172.21.15.67:0/2988273374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498003520 con 0x7fe4980039e0 2024-02-26T10:43:41.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.432+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4980039e0 0x7fe498003e00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4001750 tx=0x7fe498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c642fc6999eedf server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:41.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.432+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4001370 con 0x7fe4980039e0 2024-02-26T10:43:41.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.432+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe4a40052d0 con 0x7fe4980039e0 2024-02-26T10:43:41.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.432+0000 7fe4afe00640 1 -- 172.21.15.67:0/2988273374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4003850 con 0x7fe4980039e0 2024-02-26T10:43:41.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.432+0000 7fe4b8200640 1 -- 172.21.15.67:0/2988273374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4980039e0 msgr2=0x7fe498003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.432+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2988273374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4980039e0 0x7fe498003e00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4001750 tx=0x7fe498005830 comp rx=0 tx=0).stop 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.433+0000 7fe4b6200640 1 -- 172.21.15.67:0/2988273374 reap_dead start 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.433+0000 7fe4b8200640 1 -- 172.21.15.67:0/2988273374 shutdown_connections 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.433+0000 7fe4b8200640 1 -- 172.21.15.67:0/2988273374 >> 172.21.15.67:0/2988273374 conn(0x7fe4b00a84a0 msgr2=0x7fe4b00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.433+0000 7fe4b8200640 1 -- 172.21.15.67:0/2988273374 shutdown_connections 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.433+0000 7fe4b8200640 1 -- 172.21.15.67:0/2988273374 wait complete. 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.434+0000 7fe4b8200640 1 Processor -- start 2024-02-26T10:43:41.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.435+0000 7fe4b8200640 1 -- start start 2024-02-26T10:43:41.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b8200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4980039e0 0x7fe4b0152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b8200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b01eea10 0x7fe4b00c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b8200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 0x7fe4b00c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b8200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b0155870 con 0x7fe4b014f9f0 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b8200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00c1810 con 0x7fe4980039e0 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b8200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00c2cb0 con 0x7fe4b01eea10 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 0x7fe4b00c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 0x7fe4b00c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42904/0 (socket says 172.21.15.67:42904) 2024-02-26T10:43:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 -- 172.21.15.67:0/2269672001 learned_addr learned my addr 172.21.15.67:0/2269672001 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b4e00640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b01eea10 0x7fe4b00c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4980039e0 0x7fe4b0152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 -- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b01eea10 msgr2=0x7fe4b00c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b01eea10 0x7fe4b00c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 -- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4980039e0 msgr2=0x7fe4b0152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4980039e0 0x7fe4b0152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b6200640 1 -- 172.21.15.67:0/2269672001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b014fef0 con 0x7fe4b014f9f0 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.436+0000 7fe4b4e00640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b01eea10 0x7fe4b00c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.437+0000 7fe4b6200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 0x7fe4b00c6eb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00c7690 tx=0x7fe4a800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.437+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8017070 con 0x7fe4b014f9f0 2024-02-26T10:43:41.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.437+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe4a800c680 con 0x7fe4b014f9f0 2024-02-26T10:43:41.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.437+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b01edf30 con 0x7fe4b014f9f0 2024-02-26T10:43:41.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.437+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800ca90 con 0x7fe4b014f9f0 2024-02-26T10:43:41.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.437+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b01526a0 con 0x7fe4b014f9f0 2024-02-26T10:43:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.438+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b0074a30 con 0x7fe4b014f9f0 2024-02-26T10:43:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.439+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4a801a070 con 0x7fe4b014f9f0 2024-02-26T10:43:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.440+0000 7fe4ad600640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe49406b500 0x7fe49406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-26T10:43:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.440+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe4a8089ad0 con 0x7fe4b014f9f0 2024-02-26T10:43:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.441+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe49406b500 0x7fe49406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-26T10:43:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.442+0000 7fe4b5800640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe49406b500 0x7fe49406d9c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4033000 tx=0x7fe4a4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-26T10:43:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.444+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4a8058b50 con 0x7fe4b014f9f0 2024-02-26T10:43:41.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.678+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4b014c820 con 0x7fe4b014f9f0 2024-02-26T10:43:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.679+0000 7fe4ad600640 1 -- 172.21.15.67:0/2269672001 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fe4a8058450 con 0x7fe4b014f9f0 2024-02-26T10:43:41.676 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-26T10:43:41.676 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2978766376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2978766376},{"type":"v1","addr":"172.21.15.82:6835","nonce":2978766376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1959386296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1959386296},{"type":"v1","addr":"172.21.15.82:6837","nonce":1959386296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7110,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/3395706465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":3395706465},{"type":"v1","addr":"172.21.15.67:6835","nonce":3395706465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/3368310036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3368310036},{"type":"v1","addr":"172.21.15.67:6834","nonce":3368310036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[]} 2024-02-26T10:43:41.676 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 57 2024-02-26T10:43:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe49406b500 msgr2=0x7fe49406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe49406b500 0x7fe49406d9c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4033000 tx=0x7fe4a4009030 comp rx=0 tx=0).stop 2024-02-26T10:43:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 msgr2=0x7fe4b00c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-26T10:43:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 0x7fe4b00c6eb0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00c7690 tx=0x7fe4a800ec00 comp rx=0 tx=0).stop 2024-02-26T10:43:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 shutdown_connections 2024-02-26T10:43:41.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4b014f9f0 0x7fe4b00c6eb0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe4b01eea10 0x7fe4b00c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:6832/723923815,v1:172.21.15.82:6833/723923815] conn(0x7fe49406b500 0x7fe49406d9c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 --2- 172.21.15.67:0/2269672001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4980039e0 0x7fe4b0152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-26T10:43:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 >> 172.21.15.67:0/2269672001 conn(0x7fe4b00a84a0 msgr2=0x7fe4b00a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-26T10:43:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 shutdown_connections 2024-02-26T10:43:41.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-26T10:43:41.682+0000 7fe4b8200640 1 -- 172.21.15.67:0/2269672001 wait complete. 2024-02-26T10:43:41.699 DEBUG:tasks.cephfs.filesystem:already dead... 2024-02-26T10:43:41.699 INFO:teuthology.misc:Shutting down mds daemons... 2024-02-26T10:43:41.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-26T10:43:41.699 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:41.717 INFO:tasks.ceph.mds.a:Stopped 2024-02-26T10:43:41.717 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-26T10:43:41.717 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:41.786 INFO:tasks.ceph.mds.c:Stopped 2024-02-26T10:43:41.786 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-26T10:43:41.787 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:41.811 INFO:tasks.ceph.mds.b:Stopped 2024-02-26T10:43:41.811 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-26T10:43:41.811 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:41.885 INFO:tasks.ceph.mds.d:Stopped 2024-02-26T10:43:41.885 INFO:teuthology.misc:Shutting down osd daemons... 2024-02-26T10:43:41.886 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-02-26T10:43:41.886 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:41.963 INFO:tasks.ceph.osd.0:Stopped 2024-02-26T10:43:41.963 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-02-26T10:43:41.963 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.044 INFO:tasks.ceph.osd.1:Stopped 2024-02-26T10:43:42.044 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-02-26T10:43:42.044 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.087 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-02-26T10:43:42.087 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-02-26T10:43:42.110 INFO:tasks.ceph.osd.2:Stopped 2024-02-26T10:43:42.110 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-02-26T10:43:42.110 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.170 INFO:tasks.ceph.osd.3:Stopped 2024-02-26T10:43:42.170 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-02-26T10:43:42.170 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.241 INFO:tasks.ceph.osd.4:Stopped 2024-02-26T10:43:42.242 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-02-26T10:43:42.242 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.293 INFO:tasks.ceph.osd.5:Stopped 2024-02-26T10:43:42.293 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-02-26T10:43:42.293 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.350 INFO:tasks.ceph.osd.6:Stopped 2024-02-26T10:43:42.350 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-02-26T10:43:42.350 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.396 INFO:tasks.ceph.osd.7:Stopped 2024-02-26T10:43:42.397 INFO:teuthology.misc:Shutting down mgr daemons... 2024-02-26T10:43:42.397 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-02-26T10:43:42.397 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.466 INFO:tasks.ceph.mgr.y:Stopped 2024-02-26T10:43:42.466 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-02-26T10:43:42.466 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.549 INFO:tasks.ceph.mgr.x:Stopped 2024-02-26T10:43:42.549 INFO:teuthology.misc:Shutting down mon daemons... 2024-02-26T10:43:42.549 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-02-26T10:43:42.549 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.580 INFO:tasks.ceph.mon.a:Stopped 2024-02-26T10:43:42.581 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-02-26T10:43:42.581 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.612 INFO:tasks.ceph.mon.b:Stopped 2024-02-26T10:43:42.612 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-02-26T10:43:42.612 INFO:teuthology.orchestra.run:waiting for 300 2024-02-26T10:43:42.686 INFO:tasks.ceph.mon.c:Stopped 2024-02-26T10:43:42.686 INFO:tasks.ceph:Checking cluster log for badness... 2024-02-26T10:43:42.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-02-26T10:43:42.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:43:42.800 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-02-26T10:43:43.022 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:43:43.022 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-02-26T10:43:43.181 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:43:43.181 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-02-26T10:43:43.366 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:43:43.366 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-02-26T10:43:43.551 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:43:43.552 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-02-26T10:43:43.812 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:43:43.813 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-02-26T10:43:43.983 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:43:43.983 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-02-26T10:43:44.130 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:43:44.131 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-02-26T10:43:44.326 INFO:tasks.ceph:Archiving mon data... 2024-02-26T10:43:44.327 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/data/mon.a.tgz 2024-02-26T10:43:44.327 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-26T10:43:44.387 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.h3bPiusD3s 2024-02-26T10:43:44.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.h3bPiusD3s 2024-02-26T10:43:44.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.h3bPiusD3s 2024-02-26T10:43:44.775 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.h3bPiusD3s is 469KB 2024-02-26T10:43:44.887 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.h3bPiusD3s 2024-02-26T10:43:44.946 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/data/mon.b.tgz 2024-02-26T10:43:44.947 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-02-26T10:43:45.005 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.dkgb5qiIjW 2024-02-26T10:43:45.005 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.dkgb5qiIjW 2024-02-26T10:43:45.256 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.dkgb5qiIjW 2024-02-26T10:43:45.406 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.dkgb5qiIjW is 600KB 2024-02-26T10:43:45.512 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.dkgb5qiIjW 2024-02-26T10:43:45.569 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/data/mon.c.tgz 2024-02-26T10:43:45.569 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-02-26T10:43:45.620 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.HSfdnaJuMo 2024-02-26T10:43:45.620 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HSfdnaJuMo 2024-02-26T10:43:45.852 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.HSfdnaJuMo 2024-02-26T10:43:45.996 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.HSfdnaJuMo is 601KB 2024-02-26T10:43:46.103 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.HSfdnaJuMo 2024-02-26T10:43:46.158 INFO:tasks.ceph:Cleaning ceph cluster... 2024-02-26T10:43:46.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-26T10:43:46.166 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-02-26T10:43:46.320 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-02-26T10:43:46.373 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-02-26T10:43:46.373 INFO:tasks.ceph:Archiving crash dumps... 2024-02-26T10:43:46.375 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/remote/smithi067/crash 2024-02-26T10:43:46.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-26T10:43:46.476 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/remote/smithi082/crash 2024-02-26T10:43:46.477 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-02-26T10:43:46.569 INFO:tasks.ceph:Shutting down logrotate 2024-02-26T10:43:46.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:46.576 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:46.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:46.801 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-02-26T10:43:46.898 INFO:tasks.ceph:Compressing logs... 2024-02-26T10:43:46.898 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T10:43:46.906 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-26T10:43:46.997 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-02-26T10:43:46.998 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43911.log 2024-02-26T10:43:46.999 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43887.log 2024-02-26T10:43:47.000 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45029.log 2024-02-26T10:43:47.000 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54738.log 2024-02-26T10:43:47.002 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47583.log 2024-02-26T10:43:47.002 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43911.log.gz 2024-02-26T10:43:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.43887.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.1.43887.log.gz -5 2024-02-26T10:43:47.003 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.1.42547.log 2024-02-26T10:43:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.45029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45029.log.gz 2024-02-26T10:43:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37204.log 2024-02-26T10:43:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.42547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42547.log.gz 2024-02-26T10:43:47.005 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43086.log 2024-02-26T10:43:47.005 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37364.log 2024-02-26T10:43:47.005 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37204.log.gz 2024-02-26T10:43:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-02-26T10:43:47.006 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54738.log.gz 2024-02-26T10:43:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-02-26T10:43:47.006 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47583.log.gz 2024-02-26T10:43:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58994.log 2024-02-26T10:43:47.007 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-02-26T10:43:47.007 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43617.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-02-26T10:43:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56080.log 2024-02-26T10:43:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-02-26T10:43:47.010 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58994.log.gz 2024-02-26T10:43:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56080.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56080.log.gz 2024-02-26T10:43:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-02-26T10:43:47.011 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.43086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43086.log.gz 2024-02-26T10:43:47.011 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-02-26T10:43:47.011 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37364.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-02-26T10:43:47.012 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37364.log.gz 2024-02-26T10:43:47.012 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43424.log 2024-02-26T10:43:47.012 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-02-26T10:43:47.012 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.43424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43424.log.gz 2024-02-26T10:43:47.012 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-02-26T10:43:47.012 INFO:teuthology.orchestra.run.smithi082.stderr: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-02-26T10:43:47.014 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-02-26T10:43:47.014 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-02-26T10:43:47.015 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37244.log: /var/log/ceph/ceph-osd.7.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-02-26T10:43:47.015 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44704.log 2024-02-26T10:43:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54086.log 2024-02-26T10:43:47.016 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-02-26T10:43:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64064.log 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42953.log 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54086.log.gz 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64064.log.gz 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42953.log.gz 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59131.log 2024-02-26T10:43:47.017 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-02-26T10:43:47.019 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37324.log 2024-02-26T10:43:47.020 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.44704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-02-26T10:43:47.020 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44704.log.gz 2024-02-26T10:43:47.020 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.testuser.45516.log 2024-02-26T10:43:47.020 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37324.log.gz 2024-02-26T10:43:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52248.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2024-02-26T10:43:47.021 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63342.log 2024-02-26T10:43:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59637.log 2024-02-26T10:43:47.021 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59131.log.gz 2024-02-26T10:43:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51610.log 2024-02-26T10:43:47.022 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63342.log.gz 2024-02-26T10:43:47.022 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59637.log: gzip -5 89.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.59637.log.gz 2024-02-26T10:43:47.022 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.56327.log 2024-02-26T10:43:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51610.log.gz 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56327.log.gz 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60982.log 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-02-26T10:43:47.025 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.testuser.45516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.testuser.45516.log.gz 2024-02-26T10:43:47.026 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37164.log 2024-02-26T10:43:47.026 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-02-26T10:43:47.027 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37444.log 2024-02-26T10:43:47.027 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37164.log.gz 2024-02-26T10:43:47.027 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44223.log 2024-02-26T10:43:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-02-26T10:43:47.028 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-02-26T10:43:47.028 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-02-26T10:43:47.029 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60982.log.gz 2024-02-26T10:43:47.029 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59175.log 2024-02-26T10:43:47.029 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-02-26T10:43:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55870.log 2024-02-26T10:43:47.031 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59175.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59175.log.gz 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62527.log 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55870.log.gz 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37444.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37444.log.gz 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.44223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44223.log.gz 2024-02-26T10:43:47.032 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-02-26T10:43:47.033 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-02-26T10:43:47.033 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62527.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62527.log.gz 2024-02-26T10:43:47.034 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41979.log 2024-02-26T10:43:47.034 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59763.log 2024-02-26T10:43:47.034 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-02-26T10:43:47.035 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-02-26T10:43:47.035 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37284.log 2024-02-26T10:43:47.035 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62611.log 2024-02-26T10:43:47.036 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41979.log.gz 2024-02-26T10:43:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59763.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59763.log.gz 2024-02-26T10:43:47.036 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63212.log 2024-02-26T10:43:47.037 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60940.log 2024-02-26T10:43:47.037 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62611.log.gz 2024-02-26T10:43:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-02-26T10:43:47.038 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37284.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-02-26T10:43:47.038 INFO:teuthology.orchestra.run.smithi082.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37284.log.gz 2024-02-26T10:43:47.038 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44121.log 2024-02-26T10:43:47.039 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63212.log.gz 2024-02-26T10:43:47.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63392.log 2024-02-26T10:43:47.039 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60940.log.gz 2024-02-26T10:43:47.040 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48588.log 2024-02-26T10:43:47.041 INFO:teuthology.orchestra.run.smithi067.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44121.log.gz 2024-02-26T10:43:47.042 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2024-02-26T10:43:47.042 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63392.log.gz 2024-02-26T10:43:47.043 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63050.log 2024-02-26T10:43:47.043 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48588.log.gz 2024-02-26T10:43:47.044 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64207.log 2024-02-26T10:43:47.044 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2024-02-26T10:43:47.045 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57324.log 2024-02-26T10:43:47.045 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63050.log.gz 2024-02-26T10:43:47.046 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56621.log 2024-02-26T10:43:47.047 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64207.log.gz 2024-02-26T10:43:47.047 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57324.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57324.log.gz -5 2024-02-26T10:43:47.048 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55000.log 2024-02-26T10:43:47.048 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60256.log 2024-02-26T10:43:47.049 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56621.log.gz 2024-02-26T10:43:47.050 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54338.log 2024-02-26T10:43:47.050 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55000.log.gz 2024-02-26T10:43:47.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-mds.d.log: 94.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-02-26T10:43:47.051 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56411.log 2024-02-26T10:43:47.052 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60256.log.gz 2024-02-26T10:43:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54338.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54338.log.gz 2024-02-26T10:43:47.052 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63254.log 2024-02-26T10:43:47.053 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58391.log 2024-02-26T10:43:47.053 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56411.log.gz 2024-02-26T10:43:47.054 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-02-26T10:43:47.055 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63254.log.gz 2024-02-26T10:43:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56537.log 2024-02-26T10:43:47.056 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58391.log.gz 2024-02-26T10:43:47.057 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47243.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.46223.log 2024-02-26T10:43:47.057 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-02-26T10:43:47.058 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41630.log 2024-02-26T10:43:47.059 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56537.log.gz 2024-02-26T10:43:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57083.log 2024-02-26T10:43:47.059 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-02-26T10:43:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52935.log 2024-02-26T10:43:47.061 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41630.log.gz 2024-02-26T10:43:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57083.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.57083.log.gz --verbose 2024-02-26T10:43:47.062 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.57576.log 2024-02-26T10:43:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-02-26T10:43:47.063 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52935.log.gz 2024-02-26T10:43:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63518.log 2024-02-26T10:43:47.064 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57576.log.gz 2024-02-26T10:43:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62238.log 2024-02-26T10:43:47.066 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-02-26T10:43:47.067 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63518.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.56495.log 2024-02-26T10:43:47.067 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63518.log.gz 2024-02-26T10:43:47.068 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58866.log 2024-02-26T10:43:47.068 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62238.log.gz 2024-02-26T10:43:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59856.log 2024-02-26T10:43:47.073 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56495.log.gz 2024-02-26T10:43:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48544.log 2024-02-26T10:43:47.073 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58866.log.gz 2024-02-26T10:43:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59856.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.59856.log.gz 2024-02-26T10:43:47.073 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56999.log 2024-02-26T10:43:47.073 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42619.log 2024-02-26T10:43:47.074 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2024-02-26T10:43:47.074 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41371.log 2024-02-26T10:43:47.074 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56999.log.gz 2024-02-26T10:43:47.075 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45767.log 2024-02-26T10:43:47.075 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42619.log.gz 2024-02-26T10:43:47.076 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42357.log 2024-02-26T10:43:47.076 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41371.log.gz 2024-02-26T10:43:47.077 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60898.log 2024-02-26T10:43:47.078 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45767.log.gz 2024-02-26T10:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42357.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42357.log.gz 2024-02-26T10:43:47.079 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48895.log 2024-02-26T10:43:47.080 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2024-02-26T10:43:47.080 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60898.log.gz 2024-02-26T10:43:47.081 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57664.log 2024-02-26T10:43:47.081 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48895.log.gz 2024-02-26T10:43:47.083 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52007.log 2024-02-26T10:43:47.083 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2024-02-26T10:43:47.084 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2024-02-26T10:43:47.084 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57664.log.gz 2024-02-26T10:43:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55996.log 2024-02-26T10:43:47.085 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52007.log.gz 2024-02-26T10:43:47.086 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42441.log 2024-02-26T10:43:47.087 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2024-02-26T10:43:47.087 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-02-26T10:43:47.087 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-02-26T10:43:47.088 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45893.log 2024-02-26T10:43:47.088 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55996.log.gz 2024-02-26T10:43:47.089 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-02-26T10:43:47.089 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-02-26T10:43:47.090 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49124.log 2024-02-26T10:43:47.091 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45893.log.gz 2024-02-26T10:43:47.092 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41895.log 2024-02-26T10:43:47.092 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-02-26T10:43:47.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2024-02-26T10:43:47.093 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49124.log.gz 2024-02-26T10:43:47.094 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61986.log 2024-02-26T10:43:47.094 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41895.log.gz 2024-02-26T10:43:47.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48683.log 2024-02-26T10:43:47.095 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 2024-02-26T10:43:47.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62821.log 2024-02-26T10:43:47.097 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61986.log.gz 2024-02-26T10:43:47.097 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56873.log 2024-02-26T10:43:47.098 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48683.log.gz 2024-02-26T10:43:47.099 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.testuser.55303.log 2024-02-26T10:43:47.099 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62821.log.gz 2024-02-26T10:43:47.100 INFO:teuthology.orchestra.run.smithi082.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-02-26T10:43:47.100 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56873.log.gz 2024-02-26T10:43:47.101 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46013.log 2024-02-26T10:43:47.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.testuser.55303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58740.log 2024-02-26T10:43:47.101 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.testuser.55303.log.gz 2024-02-26T10:43:47.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63092.log 2024-02-26T10:43:47.102 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46013.log.gz 2024-02-26T10:43:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2024-02-26T10:43:47.104 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58740.log.gz 2024-02-26T10:43:47.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41853.log 2024-02-26T10:43:47.105 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63092.log.gz 2024-02-26T10:43:47.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-02-26T10:43:47.106 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2024-02-26T10:43:47.107 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52893.log 2024-02-26T10:43:47.108 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41853.log.gz 2024-02-26T10:43:47.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50656.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-02-26T10:43:47.109 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60856.log 2024-02-26T10:43:47.110 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-02-26T10:43:47.111 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52893.log.gz 2024-02-26T10:43:47.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-02-26T10:43:47.111 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60856.log.gz 2024-02-26T10:43:47.113 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-02-26T10:43:47.113 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-02-26T10:43:47.113 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-02-26T10:43:47.114 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41811.log 2024-02-26T10:43:47.114 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-02-26T10:43:47.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62779.log 2024-02-26T10:43:47.115 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-02-26T10:43:47.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54506.log 2024-02-26T10:43:47.116 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41811.log.gz 2024-02-26T10:43:47.117 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63812.log 2024-02-26T10:43:47.118 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62779.log.gz 2024-02-26T10:43:47.118 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50740.log 2024-02-26T10:43:47.119 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54506.log.gz 2024-02-26T10:43:47.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61445.log 2024-02-26T10:43:47.120 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63812.log.gz 2024-02-26T10:43:47.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51839.log 2024-02-26T10:43:47.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61445.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50740.log.gz 2024-02-26T10:43:47.122 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61445.log.gz 2024-02-26T10:43:47.122 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2024-02-26T10:43:47.123 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57534.log 2024-02-26T10:43:47.123 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51839.log.gz 2024-02-26T10:43:47.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54044.log 2024-02-26T10:43:47.124 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2024-02-26T10:43:47.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-02-26T10:43:47.126 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57534.log.gz 2024-02-26T10:43:47.127 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56747.log 2024-02-26T10:43:47.127 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54044.log.gz 2024-02-26T10:43:47.128 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53085.log 2024-02-26T10:43:47.128 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-02-26T10:43:47.129 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43283.log 2024-02-26T10:43:47.129 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56747.log.gz 2024-02-26T10:43:47.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53085.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.48954.log 2024-02-26T10:43:47.131 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53085.log.gz 2024-02-26T10:43:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-02-26T10:43:47.131 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43283.log.gz 2024-02-26T10:43:47.132 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57041.log 2024-02-26T10:43:47.133 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2024-02-26T10:43:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61487.log 2024-02-26T10:43:47.134 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-02-26T10:43:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-02-26T10:43:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61487.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57041.log.gz 2024-02-26T10:43:47.136 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61487.log.gz 2024-02-26T10:43:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2024-02-26T10:43:47.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2024-02-26T10:43:47.138 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-02-26T10:43:47.138 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45809.log 2024-02-26T10:43:47.139 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2024-02-26T10:43:47.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49776.log 2024-02-26T10:43:47.140 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2024-02-26T10:43:47.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45809.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-02-26T10:43:47.141 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44037.log 2024-02-26T10:43:47.142 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-02-26T10:43:47.142 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49776.log.gz 2024-02-26T10:43:47.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53560.log 2024-02-26T10:43:47.143 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44037.log.gz 2024-02-26T10:43:47.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52290.log 2024-02-26T10:43:47.145 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-02-26T10:43:47.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48460.log 2024-02-26T10:43:47.146 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53560.log.gz 2024-02-26T10:43:47.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52290.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52290.log.gz 2024-02-26T10:43:47.147 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58578.log 2024-02-26T10:43:47.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42621.log 2024-02-26T10:43:47.148 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48460.log.gz 2024-02-26T10:43:47.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.58578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58578.log.gz 2024-02-26T10:43:47.149 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-02-26T10:43:47.150 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59553.log 2024-02-26T10:43:47.150 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42621.log.gz 2024-02-26T10:43:47.151 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50614.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-02-26T10:43:47.151 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61860.log 2024-02-26T10:43:47.152 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54906.log 2024-02-26T10:43:47.152 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59553.log.gz 2024-02-26T10:43:47.153 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51923.log 2024-02-26T10:43:47.153 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61860.log.gz 2024-02-26T10:43:47.154 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44774.log 2024-02-26T10:43:47.155 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54906.log.gz 2024-02-26T10:43:47.156 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-02-26T10:43:47.156 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51923.log.gz 2024-02-26T10:43:47.157 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63770.log 2024-02-26T10:43:47.157 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44774.log.gz 2024-02-26T10:43:47.158 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41767.log 2024-02-26T10:43:47.158 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-02-26T10:43:47.159 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61571.log 2024-02-26T10:43:47.159 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63770.log.gz 2024-02-26T10:43:47.160 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45488.log 2024-02-26T10:43:47.161 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41767.log.gz 2024-02-26T10:43:47.161 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50488.log 2024-02-26T10:43:47.162 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61571.log.gz 2024-02-26T10:43:47.162 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49166.log 2024-02-26T10:43:47.162 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45488.log.gz 2024-02-26T10:43:47.163 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64124.log 2024-02-26T10:43:47.164 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50488.log.gz 2024-02-26T10:43:47.165 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58824.log 2024-02-26T10:43:47.165 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49166.log.gz 2024-02-26T10:43:47.166 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64124.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64124.log.gz 2024-02-26T10:43:47.166 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44223.log 2024-02-26T10:43:47.167 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56369.log 2024-02-26T10:43:47.167 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58824.log.gz 2024-02-26T10:43:47.168 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49334.log 2024-02-26T10:43:47.168 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44223.log.gz 2024-02-26T10:43:47.170 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56369.log: gzip -5 --verbose 89.0% -- -- replaced with /var/log/ceph/ceph-client.admin.56369.log.gz /var/log/ceph/ceph-client.admin.53918.log 2024-02-26T10:43:47.170 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.171 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47285.log 2024-02-26T10:43:47.171 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49334.log.gz 2024-02-26T10:43:47.172 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47919.log 2024-02-26T10:43:47.172 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53918.log.gz 2024-02-26T10:43:47.173 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59343.log 2024-02-26T10:43:47.174 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47285.log.gz 2024-02-26T10:43:47.175 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44942.log 2024-02-26T10:43:47.175 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47919.log.gz 2024-02-26T10:43:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2024-02-26T10:43:47.176 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59343.log.gz 2024-02-26T10:43:47.177 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-02-26T10:43:47.178 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44942.log.gz 2024-02-26T10:43:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2024-02-26T10:43:47.179 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55366.log 2024-02-26T10:43:47.180 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59721.log 2024-02-26T10:43:47.180 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz 2024-02-26T10:43:47.181 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44583.log 2024-02-26T10:43:47.181 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55366.log.gz 2024-02-26T10:43:47.182 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59721.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-02-26T10:43:47.182 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59721.log.gz 2024-02-26T10:43:47.183 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.44583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44583.log.gz 2024-02-26T10:43:47.184 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44858.log 2024-02-26T10:43:47.184 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48502.log 2024-02-26T10:43:47.185 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54548.log 2024-02-26T10:43:47.186 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44858.log.gz 2024-02-26T10:43:47.187 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48502.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-02-26T10:43:47.187 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53834.log 2024-02-26T10:43:47.188 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48502.log.gz 2024-02-26T10:43:47.188 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45236.log 2024-02-26T10:43:47.188 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54548.log.gz 2024-02-26T10:43:47.189 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62028.log 2024-02-26T10:43:47.189 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53834.log.gz 2024-02-26T10:43:47.190 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59259.log 2024-02-26T10:43:47.191 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45236.log.gz 2024-02-26T10:43:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42021.log 2024-02-26T10:43:47.192 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62028.log.gz 2024-02-26T10:43:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59259.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59259.log.gz 2024-02-26T10:43:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52458.log 2024-02-26T10:43:47.194 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37213.log 2024-02-26T10:43:47.195 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42021.log.gz 2024-02-26T10:43:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-02-26T10:43:47.195 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52458.log.gz 2024-02-26T10:43:47.196 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52977.log 2024-02-26T10:43:47.197 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37213.log.gz 2024-02-26T10:43:47.198 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58155.log 2024-02-26T10:43:47.198 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-02-26T10:43:47.199 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50446.log 2024-02-26T10:43:47.199 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52977.log.gz 2024-02-26T10:43:47.200 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52546.log 2024-02-26T10:43:47.201 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58155.log.gz 2024-02-26T10:43:47.201 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42623.log 2024-02-26T10:43:47.202 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50446.log.gz 2024-02-26T10:43:47.203 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43743.log 2024-02-26T10:43:47.203 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52546.log.gz 2024-02-26T10:43:47.204 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2024-02-26T10:43:47.204 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42623.log.gz 2024-02-26T10:43:47.205 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57366.log 2024-02-26T10:43:47.205 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43743.log.gz 2024-02-26T10:43:47.206 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2024-02-26T10:43:47.206 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2024-02-26T10:43:47.207 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48725.log 2024-02-26T10:43:47.208 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57366.log.gz 2024-02-26T10:43:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55954.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2024-02-26T10:43:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60088.log 2024-02-26T10:43:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60772.log 2024-02-26T10:43:47.211 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48725.log.gz 2024-02-26T10:43:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60130.log 2024-02-26T10:43:47.212 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60088.log.gz 2024-02-26T10:43:47.212 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43294.log 2024-02-26T10:43:47.213 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60772.log.gz 2024-02-26T10:43:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-02-26T10:43:47.214 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60130.log.gz 2024-02-26T10:43:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43294.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43294.log.gz 2024-02-26T10:43:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55534.log 2024-02-26T10:43:47.216 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53019.log 2024-02-26T10:43:47.216 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-02-26T10:43:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56663.log 2024-02-26T10:43:47.217 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55534.log.gz 2024-02-26T10:43:47.218 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-02-26T10:43:47.219 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56663.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53019.log.gz 2024-02-26T10:43:47.220 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2024-02-26T10:43:47.220 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56663.log.gz 2024-02-26T10:43:47.221 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-02-26T10:43:47.221 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-02-26T10:43:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43533.log 2024-02-26T10:43:47.223 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2024-02-26T10:43:47.223 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43785.log 2024-02-26T10:43:47.224 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-02-26T10:43:47.225 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51797.log 2024-02-26T10:43:47.225 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43533.log.gz 2024-02-26T10:43:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54128.log 2024-02-26T10:43:47.226 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43785.log.gz 2024-02-26T10:43:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49082.log 2024-02-26T10:43:47.227 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51797.log.gz 2024-02-26T10:43:47.228 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-02-26T10:43:47.229 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53350.log 2024-02-26T10:43:47.229 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54128.log.gz 2024-02-26T10:43:47.229 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57861.log 2024-02-26T10:43:47.230 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49082.log.gz 2024-02-26T10:43:47.230 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60214.log 2024-02-26T10:43:47.231 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53350.log.gz 2024-02-26T10:43:47.232 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2024-02-26T10:43:47.232 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57861.log.gz 2024-02-26T10:43:47.233 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60214.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60214.log.gz 2024-02-26T10:43:47.233 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-02-26T10:43:47.234 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52767.log 2024-02-26T10:43:47.234 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2024-02-26T10:43:47.235 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-02-26T10:43:47.235 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-02-26T10:43:47.237 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61529.log 2024-02-26T10:43:47.237 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52767.log.gz 2024-02-26T10:43:47.238 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46475.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-02-26T10:43:47.238 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54254.log 2024-02-26T10:43:47.239 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61024.log 2024-02-26T10:43:47.239 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61529.log.gz 2024-02-26T10:43:47.240 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2024-02-26T10:43:47.240 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54254.log.gz 2024-02-26T10:43:47.241 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-02-26T10:43:47.242 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61024.log.gz 2024-02-26T10:43:47.243 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57408.log 2024-02-26T10:43:47.243 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2024-02-26T10:43:47.244 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-02-26T10:43:47.244 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-02-26T10:43:47.245 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62569.log 2024-02-26T10:43:47.245 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57408.log.gz 2024-02-26T10:43:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2024-02-26T10:43:47.246 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-02-26T10:43:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63938.log 2024-02-26T10:43:47.248 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62569.log.gz 2024-02-26T10:43:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46391.log 2024-02-26T10:43:47.249 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2024-02-26T10:43:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-02-26T10:43:47.250 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63938.log.gz 2024-02-26T10:43:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60046.log 2024-02-26T10:43:47.251 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46391.log.gz 2024-02-26T10:43:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51088.log 2024-02-26T10:43:47.253 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-02-26T10:43:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54822.log 2024-02-26T10:43:47.254 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60046.log.gz 2024-02-26T10:43:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63008.log 2024-02-26T10:43:47.255 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51088.log.gz 2024-02-26T10:43:47.256 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54822.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-02-26T10:43:47.257 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54822.log.gz 2024-02-26T10:43:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61283.log 2024-02-26T10:43:47.258 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz 2024-02-26T10:43:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-02-26T10:43:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46867.log 2024-02-26T10:43:47.260 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61283.log.gz 2024-02-26T10:43:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48250.log 2024-02-26T10:43:47.261 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-02-26T10:43:47.262 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61697.log 2024-02-26T10:43:47.263 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46867.log.gz 2024-02-26T10:43:47.264 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57715.log 2024-02-26T10:43:47.264 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48250.log.gz 2024-02-26T10:43:47.265 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61697.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61697.log.gz 2024-02-26T10:43:47.265 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2024-02-26T10:43:47.267 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48767.log 2024-02-26T10:43:47.267 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57715.log.gz 2024-02-26T10:43:47.268 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50698.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2024-02-26T10:43:47.269 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55702.log 2024-02-26T10:43:47.270 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51340.log 2024-02-26T10:43:47.270 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48767.log.gz 2024-02-26T10:43:47.271 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55702.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.55702.log.gz 2024-02-26T10:43:47.271 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51550.log 2024-02-26T10:43:47.273 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47413.log 2024-02-26T10:43:47.273 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51340.log.gz 2024-02-26T10:43:47.274 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62112.log 2024-02-26T10:43:47.275 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51550.log.gz 2024-02-26T10:43:47.276 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53792.log 2024-02-26T10:43:47.276 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47413.log.gz 2024-02-26T10:43:47.277 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61367.log 2024-02-26T10:43:47.277 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62112.log.gz 2024-02-26T10:43:47.278 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63644.log 2024-02-26T10:43:47.278 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53792.log.gz 2024-02-26T10:43:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49376.log 2024-02-26T10:43:47.280 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61367.log.gz 2024-02-26T10:43:47.281 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2024-02-26T10:43:47.281 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63644.log.gz 2024-02-26T10:43:47.282 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57945.log 2024-02-26T10:43:47.282 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49376.log.gz 2024-02-26T10:43:47.283 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53272.log 2024-02-26T10:43:47.283 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2024-02-26T10:43:47.284 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45530.log 2024-02-26T10:43:47.284 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57945.log.gz 2024-02-26T10:43:47.285 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.53272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53272.log.gz 2024-02-26T10:43:47.286 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54654.log 2024-02-26T10:43:47.286 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-02-26T10:43:47.287 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61902.log 2024-02-26T10:43:47.287 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45530.log.gz 2024-02-26T10:43:47.288 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2024-02-26T10:43:47.288 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54654.log.gz 2024-02-26T10:43:47.289 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-02-26T10:43:47.289 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61902.log.gz 2024-02-26T10:43:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46769.log 2024-02-26T10:43:47.290 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2024-02-26T10:43:47.291 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61115.log 2024-02-26T10:43:47.291 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-02-26T10:43:47.292 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46769.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46769.log.gz -5 2024-02-26T10:43:47.292 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42105.log 2024-02-26T10:43:47.293 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50877.log 2024-02-26T10:43:47.294 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61115.log.gz 2024-02-26T10:43:47.295 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62737.log 2024-02-26T10:43:47.295 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz 2024-02-26T10:43:47.296 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-02-26T10:43:47.296 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50877.log.gz 2024-02-26T10:43:47.297 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52639.log 2024-02-26T10:43:47.297 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62737.log.gz 2024-02-26T10:43:47.298 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.36829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63602.log 2024-02-26T10:43:47.298 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-02-26T10:43:47.299 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52639.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.52639.log.gz -5 2024-02-26T10:43:47.299 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59595.log 2024-02-26T10:43:47.300 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57450.log 2024-02-26T10:43:47.300 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63602.log.gz 2024-02-26T10:43:47.301 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56789.log 2024-02-26T10:43:47.301 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59595.log.gz 2024-02-26T10:43:47.302 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62425.log 2024-02-26T10:43:47.303 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57450.log.gz 2024-02-26T10:43:47.303 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56789.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56789.log.gz 2024-02-26T10:43:47.304 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59469.log 2024-02-26T10:43:47.305 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63896.log 2024-02-26T10:43:47.305 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62425.log.gz 2024-02-26T10:43:47.306 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59469.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59469.log.gz 2024-02-26T10:43:47.306 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2024-02-26T10:43:47.307 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52374.log 2024-02-26T10:43:47.308 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45194.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63896.log.gz 2024-02-26T10:43:47.309 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2024-02-26T10:43:47.309 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41275.log 2024-02-26T10:43:47.310 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-02-26T10:43:47.311 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52374.log.gz 2024-02-26T10:43:47.312 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41275.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41275.log.gz 2024-02-26T10:43:47.312 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45068.log 2024-02-26T10:43:47.313 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43575.log 2024-02-26T10:43:47.314 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-02-26T10:43:47.315 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45068.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45068.log.gz 2024-02-26T10:43:47.315 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43995.log 2024-02-26T10:43:47.316 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52851.log 2024-02-26T10:43:47.316 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1%/var/log/ceph/ceph-client.admin.43995.log: -- replaced with /var/log/ceph/ceph-client.admin.43575.log.gz 2024-02-26T10:43:47.317 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43995.log.gz 2024-02-26T10:43:47.317 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63560.log 2024-02-26T10:43:47.318 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-02-26T10:43:47.319 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52851.log.gz 2024-02-26T10:43:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63560.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63560.log.gz 2024-02-26T10:43:47.320 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62340.log 2024-02-26T10:43:47.321 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56122.log 2024-02-26T10:43:47.322 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-02-26T10:43:47.323 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62966.log 2024-02-26T10:43:47.323 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62340.log.gz 2024-02-26T10:43:47.324 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54380.log 2024-02-26T10:43:47.324 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56122.log.gz 2024-02-26T10:43:47.326 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58283.log 2024-02-26T10:43:47.326 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62966.log.gz 2024-02-26T10:43:47.327 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-02-26T10:43:47.327 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54380.log.gz 2024-02-26T10:43:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58283.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-02-26T10:43:47.329 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58283.log.gz 2024-02-26T10:43:47.330 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63434.log 2024-02-26T10:43:47.330 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-02-26T10:43:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44079.log 2024-02-26T10:43:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63434.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63434.log.gz 2024-02-26T10:43:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52416.log 2024-02-26T10:43:47.334 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48632.log 2024-02-26T10:43:47.335 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44079.log.gz 2024-02-26T10:43:47.336 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37085.log 2024-02-26T10:43:47.336 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52416.log.gz 2024-02-26T10:43:47.337 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63854.log 2024-02-26T10:43:47.338 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48632.log.gz 2024-02-26T10:43:47.339 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62923.log 2024-02-26T10:43:47.339 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37085.log.gz 2024-02-26T10:43:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57801.log 2024-02-26T10:43:47.340 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63854.log.gz 2024-02-26T10:43:47.342 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51670.log 2024-02-26T10:43:47.342 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62923.log.gz 2024-02-26T10:43:47.343 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62653.log 2024-02-26T10:43:47.343 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57801.log.gz 2024-02-26T10:43:47.344 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-02-26T10:43:47.344 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51670.log.gz 2024-02-26T10:43:47.346 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48998.log 2024-02-26T10:43:47.346 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62653.log.gz 2024-02-26T10:43:47.347 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-02-26T10:43:47.348 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-02-26T10:43:47.349 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45935.log 2024-02-26T10:43:47.349 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48998.log.gz 2024-02-26T10:43:47.350 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-02-26T10:43:47.351 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-02-26T10:43:47.351 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2024-02-26T10:43:47.352 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45935.log.gz 2024-02-26T10:43:47.353 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56831.log 2024-02-26T10:43:47.353 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-02-26T10:43:47.355 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52502.log 2024-02-26T10:43:47.355 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2024-02-26T10:43:47.356 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43827.log 2024-02-26T10:43:47.357 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56831.log.gz 2024-02-26T10:43:47.357 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52502.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30841.log 2024-02-26T10:43:47.357 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52502.log.gz 2024-02-26T10:43:47.358 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42608.log 2024-02-26T10:43:47.359 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43827.log.gz 2024-02-26T10:43:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.tmp-client.admin.30841.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30841.log.gz 2024-02-26T10:43:47.360 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50236.log 2024-02-26T10:43:47.361 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61241.log 2024-02-26T10:43:47.361 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42608.log.gz 2024-02-26T10:43:47.362 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59679.log 2024-02-26T10:43:47.363 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50236.log.gz 2024-02-26T10:43:47.364 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44984.log 2024-02-26T10:43:47.364 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61241.log.gz 2024-02-26T10:43:47.365 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61157.log 2024-02-26T10:43:47.365 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59679.log.gz 2024-02-26T10:43:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59089.log 2024-02-26T10:43:47.367 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44984.log.gz 2024-02-26T10:43:47.368 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51130.log 2024-02-26T10:43:47.368 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61157.log.gz 2024-02-26T10:43:47.368 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42399.log 2024-02-26T10:43:47.369 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59089.log.gz 2024-02-26T10:43:47.370 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-02-26T10:43:47.370 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51130.log.gz 2024-02-26T10:43:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42399.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-02-26T10:43:47.371 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-02-26T10:43:47.372 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-02-26T10:43:47.373 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-02-26T10:43:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46643.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-02-26T10:43:47.374 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62383.log 2024-02-26T10:43:47.375 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61655.log 2024-02-26T10:43:47.375 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-02-26T10:43:47.376 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45110.log 2024-02-26T10:43:47.376 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62383.log.gz 2024-02-26T10:43:47.377 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58071.log 2024-02-26T10:43:47.378 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61655.log.gz 2024-02-26T10:43:47.378 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58908.log 2024-02-26T10:43:47.379 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45110.log.gz 2024-02-26T10:43:47.380 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55618.log 2024-02-26T10:43:47.380 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58071.log.gz 2024-02-26T10:43:47.381 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58908.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58908.log.gz 2024-02-26T10:43:47.381 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54864.log 2024-02-26T10:43:47.382 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58698.log 2024-02-26T10:43:47.383 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55618.log.gz 2024-02-26T10:43:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54864.log: gzip -5 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54864.log.gz /var/log/ceph/ceph-client.admin.42616.log 2024-02-26T10:43:47.384 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58698.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58698.log.gz 2024-02-26T10:43:47.385 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49524.log 2024-02-26T10:43:47.386 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-02-26T10:43:47.387 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-02-26T10:43:47.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57492.log 2024-02-26T10:43:47.388 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49524.log.gz 2024-02-26T10:43:47.389 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-02-26T10:43:47.389 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz 2024-02-26T10:43:47.390 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53434.log 2024-02-26T10:43:47.390 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57492.log.gz 2024-02-26T10:43:47.391 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58782.log 2024-02-26T10:43:47.392 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-02-26T10:43:47.393 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43265.log 2024-02-26T10:43:47.393 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53434.log.gz 2024-02-26T10:43:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51424.log 2024-02-26T10:43:47.394 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58782.log.gz 2024-02-26T10:43:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50320.log 2024-02-26T10:43:47.395 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43265.log.gz 2024-02-26T10:43:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-02-26T10:43:47.397 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51424.log.gz 2024-02-26T10:43:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-02-26T10:43:47.398 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50320.log.gz 2024-02-26T10:43:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61818.log 2024-02-26T10:43:47.399 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-02-26T10:43:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45572.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-02-26T10:43:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62695.log 2024-02-26T10:43:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52681.log 2024-02-26T10:43:47.402 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61818.log.gz 2024-02-26T10:43:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-02-26T10:43:47.403 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62695.log.gz 2024-02-26T10:43:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62070.log 2024-02-26T10:43:47.404 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52681.log.gz 2024-02-26T10:43:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.45656.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50962.log 2024-02-26T10:43:47.405 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-02-26T10:43:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43869.log 2024-02-26T10:43:47.406 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62070.log.gz 2024-02-26T10:43:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50962.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2024-02-26T10:43:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60814.log 2024-02-26T10:43:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62196.log 2024-02-26T10:43:47.409 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43869.log.gz 2024-02-26T10:43:47.410 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42956.log 2024-02-26T10:43:47.410 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60814.log.gz 2024-02-26T10:43:47.411 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55786.log 2024-02-26T10:43:47.411 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62196.log.gz 2024-02-26T10:43:47.412 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42956.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-02-26T10:43:47.413 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42956.log.gz 2024-02-26T10:43:47.414 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59427.log 2024-02-26T10:43:47.414 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55786.log.gz 2024-02-26T10:43:47.415 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63980.log 2024-02-26T10:43:47.416 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58029.log 2024-02-26T10:43:47.416 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59427.log.gz 2024-02-26T10:43:47.418 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64165.log 2024-02-26T10:43:47.418 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63980.log.gz 2024-02-26T10:43:47.419 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56164.log 2024-02-26T10:43:47.419 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58029.log.gz 2024-02-26T10:43:47.420 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-02-26T10:43:47.420 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64165.log.gz 2024-02-26T10:43:47.421 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56164.log: 93.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-02-26T10:43:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55408.log 2024-02-26T10:43:47.422 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56164.log.gz 2024-02-26T10:43:47.422 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56224.log 2024-02-26T10:43:47.423 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-02-26T10:43:47.424 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45152.log 2024-02-26T10:43:47.424 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55408.log.gz 2024-02-26T10:43:47.425 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56224.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56224.log.gz 2024-02-26T10:43:47.425 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63728.log 2024-02-26T10:43:47.426 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51508.log 2024-02-26T10:43:47.427 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45152.log.gz 2024-02-26T10:43:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63728.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63728.log.gz 2024-02-26T10:43:47.428 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-02-26T10:43:47.429 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49418.log 2024-02-26T10:43:47.429 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51508.log.gz 2024-02-26T10:43:47.430 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48334.log 2024-02-26T10:43:47.430 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-02-26T10:43:47.431 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43953.log 2024-02-26T10:43:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48334.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49418.log.gz 2024-02-26T10:43:47.432 INFO:teuthology.orchestra.run.smithi067.stderr:gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.48334.log.gz --verbose 2024-02-26T10:43:47.433 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.51214.log 2024-02-26T10:43:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2024-02-26T10:43:47.434 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43953.log.gz 2024-02-26T10:43:47.435 INFO:teuthology.orchestra.run.smithi082.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-02-26T10:43:47.435 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48172.log 2024-02-26T10:43:47.436 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51214.log.gz 2024-02-26T10:43:47.436 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48376.log 2024-02-26T10:43:47.437 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2024-02-26T10:43:47.437 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.48172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48172.log.gz 2024-02-26T10:43:47.438 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-02-26T10:43:47.438 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48376.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48376.log.gz 2024-02-26T10:43:47.439 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43659.log 2024-02-26T10:43:47.439 INFO:teuthology.orchestra.run.smithi082.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-02-26T10:43:47.440 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52725.log 2024-02-26T10:43:47.440 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz/var/log/ceph/ceph-client.admin.43659.log: 2024-02-26T10:43:47.441 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-02-26T10:43:47.441 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43659.log.gz 2024-02-26T10:43:47.442 INFO:teuthology.orchestra.run.smithi082.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-02-26T10:43:47.442 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52725.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52725.log.gz 2024-02-26T10:43:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53644.log 2024-02-26T10:43:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54170.log 2024-02-26T10:43:47.444 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz/var/log/ceph/ceph-client.admin.53644.log: 2024-02-26T10:43:47.445 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53644.log.gz 2024-02-26T10:43:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51256.log 2024-02-26T10:43:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.testuser.60653.log 2024-02-26T10:43:47.446 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54170.log.gz 2024-02-26T10:43:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59511.log 2024-02-26T10:43:47.447 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51256.log.gz 2024-02-26T10:43:47.448 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.testuser.60653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44816.log 2024-02-26T10:43:47.448 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.testuser.60653.log.gz 2024-02-26T10:43:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64022.log 2024-02-26T10:43:47.450 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59511.log.gz 2024-02-26T10:43:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44816.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44816.log.gz 2024-02-26T10:43:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55744.log 2024-02-26T10:43:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46947.log 2024-02-26T10:43:47.452 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64022.log.gz 2024-02-26T10:43:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54296.log 2024-02-26T10:43:47.453 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55744.log.gz 2024-02-26T10:43:47.454 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46947.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46947.log.gz 2024-02-26T10:43:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-02-26T10:43:47.455 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58241.log 2024-02-26T10:43:47.456 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54296.log.gz 2024-02-26T10:43:47.457 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58656.log 2024-02-26T10:43:47.457 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-02-26T10:43:47.458 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56038.log 88.8% 2024-02-26T10:43:47.458 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58241.log.gz 2024-02-26T10:43:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58656.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-02-26T10:43:47.459 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58656.log.gz 2024-02-26T10:43:47.460 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54696.log 2024-02-26T10:43:47.461 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56038.log.gz 2024-02-26T10:43:47.461 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44900.log 2024-02-26T10:43:47.463 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62280.log 2024-02-26T10:43:47.463 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54696.log.gz 2024-02-26T10:43:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59301.log 2024-02-26T10:43:47.464 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44900.log.gz 2024-02-26T10:43:47.466 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62280.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62280.log.gz 2024-02-26T10:43:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49870.log 2024-02-26T10:43:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50828.log 2024-02-26T10:43:47.470 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59301.log.gz 2024-02-26T10:43:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53750.log 2024-02-26T10:43:47.471 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49870.log.gz 2024-02-26T10:43:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57204.log 2024-02-26T10:43:47.472 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50828.log.gz 2024-02-26T10:43:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log 2024-02-26T10:43:47.473 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53750.log.gz 2024-02-26T10:43:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50919.log 2024-02-26T10:43:47.475 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57204.log.gz 2024-02-26T10:43:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58325.log 2024-02-26T10:43:47.476 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-02-26T10:43:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49608.log 2024-02-26T10:43:47.478 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50919.log.gz 2024-02-26T10:43:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61776.log 2024-02-26T10:43:47.479 INFO:teuthology.orchestra.run.smithi067.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58325.log.gz 2024-02-26T10:43:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57125.log 2024-02-26T10:43:47.480 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 88.5% 2024-02-26T10:43:47.480 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49608.log.gz 2024-02-26T10:43:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-02-26T10:43:47.481 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61776.log.gz 2024-02-26T10:43:47.482 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51172.log 2024-02-26T10:43:47.482 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57125.log.gz 2024-02-26T10:43:47.483 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59217.log 2024-02-26T10:43:47.483 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-02-26T10:43:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51172.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51172.log.gz 2024-02-26T10:43:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-02-26T10:43:47.485 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.59217.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59038.log 2024-02-26T10:43:47.485 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59217.log.gz 2024-02-26T10:43:47.486 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45614.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-02-26T10:43:47.486 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55576.log 2024-02-26T10:43:47.487 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45320.log 2024-02-26T10:43:47.488 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59038.log: /var/log/ceph/ceph-client.admin.55576.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59038.log.gz 2024-02-26T10:43:47.488 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55576.log.gz 2024-02-26T10:43:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42949.log 2024-02-26T10:43:47.489 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-02-26T10:43:47.490 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45320.log.gz 2024-02-26T10:43:47.490 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42949.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42949.log.gz 2024-02-26T10:43:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49734.log 2024-02-26T10:43:47.491 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53876.log 2024-02-26T10:43:47.492 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-02-26T10:43:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49734.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49734.log.gz 2024-02-26T10:43:47.493 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-02-26T10:43:47.494 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57987.log 2024-02-26T10:43:47.494 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53876.log.gz 2024-02-26T10:43:47.495 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53688.log 2024-02-26T10:43:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57987.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-02-26T10:43:47.496 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55492.log 2024-02-26T10:43:47.496 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57987.log.gz 2024-02-26T10:43:47.497 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45026.log 2024-02-26T10:43:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53688.log: /var/log/ceph/ceph-client.admin.55492.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53688.log.gz 2024-02-26T10:43:47.498 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.56453.log -- replaced with /var/log/ceph/ceph-client.admin.55492.log.gz 2024-02-26T10:43:47.498 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58113.log 2024-02-26T10:43:47.500 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45026.log.gz 2024-02-26T10:43:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56453.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56453.log.gz 2024-02-26T10:43:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50572.log 2024-02-26T10:43:47.501 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56957.log 2024-02-26T10:43:47.502 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58113.log.gz 2024-02-26T10:43:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50572.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50572.log.gz 2024-02-26T10:43:47.503 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47793.log 2024-02-26T10:43:47.504 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55828.log 2024-02-26T10:43:47.504 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56957.log.gz 2024-02-26T10:43:47.505 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47793.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47793.log.gz 2024-02-26T10:43:47.505 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48853.log 2024-02-26T10:43:47.506 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2024-02-26T10:43:47.506 INFO:teuthology.orchestra.run.smithi082.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-02-26T10:43:47.506 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0%/var/log/ceph/ceph-client.admin.48853.log: -- replaced with /var/log/ceph/ceph-client.admin.55828.log.gz 2024-02-26T10:43:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.48853.log.gz /var/log/ceph/ceph-osd.2.log 2024-02-26T10:43:47.507 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-02-26T10:43:47.508 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz 2024-02-26T10:43:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43285.log 2024-02-26T10:43:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47877.log 2024-02-26T10:43:47.512 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2024-02-26T10:43:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-02-26T10:43:47.512 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43285.log.gz 2024-02-26T10:43:47.513 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-02-26T10:43:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m0.554s 2024-02-26T10:43:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m2.222s 2024-02-26T10:43:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.313s 2024-02-26T10:43:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-02-26T10:43:47.514 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47877.log.gz 2024-02-26T10:43:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42941.log 2024-02-26T10:43:47.515 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-02-26T10:43:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61199.log 2024-02-26T10:43:47.516 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-02-26T10:43:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42941.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-02-26T10:43:47.517 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42941.log.gz 2024-02-26T10:43:47.517 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60004.log 2024-02-26T10:43:47.518 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61199.log.gz 2024-02-26T10:43:47.519 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55912.log 2024-02-26T10:43:47.520 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41715.log 2024-02-26T10:43:47.520 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60004.log.gz 2024-02-26T10:43:47.522 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59385.log 2024-02-26T10:43:47.522 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55912.log.gz 2024-02-26T10:43:47.523 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63476.log 2024-02-26T10:43:47.523 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41715.log.gz 2024-02-26T10:43:47.524 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51382.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59385.log.gz 2024-02-26T10:43:47.524 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.526 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63476.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53518.log 2024-02-26T10:43:47.526 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63476.log.gz 2024-02-26T10:43:47.527 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54422.log 2024-02-26T10:43:47.527 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51382.log.gz 2024-02-26T10:43:47.528 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51046.log 2024-02-26T10:43:47.529 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53518.log.gz 2024-02-26T10:43:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54422.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54422.log.gz 2024-02-26T10:43:47.530 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57620.log 2024-02-26T10:43:47.531 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51046.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51046.log.gz 2024-02-26T10:43:47.531 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.testuser.50173.log 2024-02-26T10:43:47.533 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56579.log 2024-02-26T10:43:47.533 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57620.log.gz 2024-02-26T10:43:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.testuser.50173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.testuser.50173.log.gz 2024-02-26T10:43:47.534 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58198.log 2024-02-26T10:43:47.535 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56579.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56579.log.gz 2024-02-26T10:43:47.536 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54212.log 2024-02-26T10:43:47.537 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41937.log 2024-02-26T10:43:47.538 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58198.log.gz 2024-02-26T10:43:47.538 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54212.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54212.log.gz 2024-02-26T10:43:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-02-26T10:43:47.539 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41937.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41937.log.gz 2024-02-26T10:43:47.540 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-02-26T10:43:47.541 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52809.log 2024-02-26T10:43:47.542 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz 2024-02-26T10:43:47.543 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42315.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-02-26T10:43:47.543 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55660.log 2024-02-26T10:43:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52809.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52809.log.gz 2024-02-26T10:43:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61944.log 2024-02-26T10:43:47.546 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61325.log 2024-02-26T10:43:47.546 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55660.log.gz 2024-02-26T10:43:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61944.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61944.log.gz 2024-02-26T10:43:47.548 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58950.log 2024-02-26T10:43:47.549 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61325.log.gz 2024-02-26T10:43:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62485.log 2024-02-26T10:43:47.551 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-02-26T10:43:47.551 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58950.log.gz 2024-02-26T10:43:47.553 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62485.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62485.log.gz 2024-02-26T10:43:47.553 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56705.log 2024-02-26T10:43:47.553 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-02-26T10:43:47.554 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57903.log 2024-02-26T10:43:47.555 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-02-26T10:43:47.556 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56705.log.gz 2024-02-26T10:43:47.558 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57903.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57903.log.gz 2024-02-26T10:43:47.558 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62863.log 2024-02-26T10:43:47.558 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2024-02-26T10:43:47.559 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63297.log 2024-02-26T10:43:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42958.log 2024-02-26T10:43:47.561 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62863.log.gz 2024-02-26T10:43:47.561 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63297.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63297.log.gz 2024-02-26T10:43:47.562 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55450.log 2024-02-26T10:43:47.563 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49292.log 2024-02-26T10:43:47.564 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42958.log.gz 2024-02-26T10:43:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55450.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55450.log.gz 2024-02-26T10:43:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63686.log 2024-02-26T10:43:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49292.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49292.log.gz 2024-02-26T10:43:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59898.log 2024-02-26T10:43:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48418.log 2024-02-26T10:43:47.568 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63686.log.gz 2024-02-26T10:43:47.569 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59898.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59898.log.gz 2024-02-26T10:43:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53392.log 2024-02-26T10:43:47.571 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51713.log 2024-02-26T10:43:47.571 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-02-26T10:43:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60350.log 2024-02-26T10:43:47.573 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53392.log.gz 2024-02-26T10:43:47.574 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51713.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51713.log.gz 2024-02-26T10:43:47.574 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-02-26T10:43:47.575 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60350.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60350.log.gz 2024-02-26T10:43:47.575 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-02-26T10:43:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48292.log 2024-02-26T10:43:47.577 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-02-26T10:43:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46517.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.54002.log -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-02-26T10:43:47.579 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.580 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-02-26T10:43:47.580 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48292.log.gz 2024-02-26T10:43:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51755.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54002.log.gz 2024-02-26T10:43:47.581 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47201.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-02-26T10:43:47.583 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43491.log 2024-02-26T10:43:47.584 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56283.log 2024-02-26T10:43:47.584 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51755.log.gz 2024-02-26T10:43:47.585 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43491.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.53960.log 2024-02-26T10:43:47.586 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43491.log.gz 2024-02-26T10:43:47.587 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56283.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-02-26T10:43:47.587 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56283.log.gz 2024-02-26T10:43:47.588 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60172.log 2024-02-26T10:43:47.588 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53960.log.gz 2024-02-26T10:43:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60730.log 2024-02-26T10:43:47.590 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60172.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60172.log.gz 2024-02-26T10:43:47.591 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-02-26T10:43:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60730.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60730.log.gz 2024-02-26T10:43:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61613.log 2024-02-26T10:43:47.594 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47369.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-02-26T10:43:47.595 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61613.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61613.log.gz 2024-02-26T10:43:47.596 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56915.log 2024-02-26T10:43:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47835.log 2024-02-26T10:43:47.599 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56915.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56915.log.gz 2024-02-26T10:43:47.600 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62154.log 2024-02-26T10:43:47.600 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47835.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47835.log.gz 2024-02-26T10:43:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51466.log 2024-02-26T10:43:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62154.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62154.log.gz 2024-02-26T10:43:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53476.log 2024-02-26T10:43:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51466.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51466.log.gz 2024-02-26T10:43:47.605 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-02-26T10:43:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53476.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53476.log.gz 2024-02-26T10:43:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49040.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-02-26T10:43:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57757.log 2024-02-26T10:43:47.609 INFO:teuthology.orchestra.run.smithi067.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-02-26T10:43:47.611 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57757.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57757.log.gz 2024-02-26T10:43:47.640 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-02-26T10:43:47.684 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-02-26T10:43:47.773 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-02-26T10:43:47.860 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-02-26T10:43:47.885 INFO:teuthology.orchestra.run.smithi067.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-02-26T10:43:47.889 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-26T10:43:47.890 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m0.935s 2024-02-26T10:43:47.890 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m2.318s 2024-02-26T10:43:47.890 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m1.497s 2024-02-26T10:43:47.891 INFO:tasks.ceph:Archiving logs... 2024-02-26T10:43:47.891 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/remote/smithi067/log 2024-02-26T10:43:47.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T10:43:48.615 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/remote/smithi082/log 2024-02-26T10:43:48.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-26T10:43:48.987 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-26T10:43:48.996 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-26T10:43:48.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T10:43:49.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-26T10:43:49.166 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-26T10:43:49.175 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-26T10:43:49.176 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:43:49.183 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-26T10:43:49.234 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:43:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-02-26T10:43:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:43:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-02-26T10:43:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 +1189us[+1189us] +/- 117ms 2024-02-26T10:43:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -2773us[-2760us] +/- 77ms 2024-02-26T10:43:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +2393us[+2393us] +/- 67ms 2024-02-26T10:43:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:43:49.246 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-26T10:43:49.246 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-02-26T10:43:49.246 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +1061us[+1240us] +/- 117ms 2024-02-26T10:43:49.246 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -2904us[-2724us] +/- 77ms 2024-02-26T10:43:49.246 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +2262us[+2441us] +/- 67ms 2024-02-26T10:43:49.247 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-26T10:43:49.248 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-26T10:43:49.257 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-26T10:43:49.258 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-26T10:43:49.267 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T10:43:49.518 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-26T10:43:49.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:43:49.845 INFO:teuthology.orchestra.run.smithi067.stdout:type=AVC msg=audit(1708943256.377:198): avc: denied { checkpoint_restore } for pid=1204 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-02-26T10:43:49.846 DEBUG:teuthology.task.selinux:ubuntu@smithi067.front.sepia.ceph.com has 1 denials 2024-02-26T10:43:49.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-26T10:43:49.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:43:49.948 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi067.front.sepia.ceph.com: ['type=AVC msg=audit(1708943256.377:198): avc: denied { checkpoint_restore } for pid=1204 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-02-26T10:43:49.957 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-26T10:43:49.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-26T10:43:49.978 INFO:teuthology.task.internal:Duration was 975.131108 seconds 2024-02-26T10:43:49.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-26T10:43:49.988 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-26T10:43:49.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T10:43:49.998 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-26T10:43:50.166 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:43:50.166 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-26T10:43:50.487 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-26T10:43:50.487 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-02-26T10:43:50.487 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T10:43:50.551 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-02-26T10:43:50.551 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-26T10:43:50.625 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-26T10:43:50.625 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T10:43:50.632 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-26T10:43:50.727 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-26T10:43:50.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T10:43:50.732 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-26T10:43:50.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-26T10:43:50.928 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-26T10:43:50.928 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T10:43:51.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-26T10:43:51.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-26T10:43:51.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:43:51.142 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:43:51.245 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-02-26T10:43:51.249 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-02-26T10:43:51.304 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:43:51.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:43:51.355 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-26T10:43:51.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-26T10:43:51.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-26T10:43:51.419 INFO:teuthology.task.internal:Transferring archived files... 2024-02-26T10:43:51.419 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/remote/smithi067 2024-02-26T10:43:51.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T10:43:51.531 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557/remote/smithi082 2024-02-26T10:43:51.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-26T10:43:51.635 INFO:teuthology.task.internal:Removing archive directory... 2024-02-26T10:43:51.635 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T10:43:51.643 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-26T10:43:51.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-26T10:43:51.709 INFO:teuthology.task.internal:Not uploading archives. 2024-02-26T10:43:51.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-26T10:43:51.719 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-26T10:43:51.719 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T10:43:51.727 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-26T10:43:51.775 INFO:teuthology.orchestra.run.smithi067.stdout: 395193 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 10:43 /home/ubuntu/cephtest 2024-02-26T10:43:51.782 INFO:teuthology.orchestra.run.smithi082.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 26 10:43 /home/ubuntu/cephtest 2024-02-26T10:43:51.785 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-26T10:43:51.795 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-26T10:43:51.806 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi067.front.sepia.ceph.com: ['type=AVC msg=audit(1708943256.377:198): avc: denied { checkpoint_restore } for pid=1204 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-02-26T10:43:51.806 INFO:teuthology.nuke:Checking targets against current locks 2024-02-26T10:43:51.833 DEBUG:teuthology.nuke:shortname: smithi067 2024-02-26T10:43:51.833 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:43:51.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:04.852051', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNZXEQayW+z0FSWCIGyNxq4hGDs3AtBWyqL+83QzXIFLyaMXmnNDH7mSH/q9RIINXLSmXCuPUhqNV+2Xkv6VY3s='} 2024-02-26T10:43:51.859 DEBUG:teuthology.nuke:shortname: smithi082 2024-02-26T10:43:51.859 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-26T10:43:51.881 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-26_10:07:12-fs-wip-vshankar-testing-20240226.064629-testing-default-smithi/7573557', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-26 10:09:04.850833', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzIN0+PSmnpktoeKuKPW5cX+J0qDvyfnUFf3H5ZNN5S4jHoYDQc8Ab/xnkEM2vKIs4KnCVib96B9xtHEbIUR54='} 2024-02-26T10:43:51.904 INFO:teuthology.orchestra.console.smithi067:Power off 2024-02-26T10:43:51.904 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T10:43:51.927 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2024-02-26T10:43:51.927 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:43:51.942 INFO:teuthology.orchestra.console.smithi082:Power off 2024-02-26T10:43:51.942 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-26T10:43:51.952 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-26T10:43:51.964 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-02-26T10:43:51.964 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:43:51.988 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-26T10:43:55.953 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:43:55.989 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:43:56.062 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-26T10:43:56.097 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-26T10:44:00.067 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:44:00.097 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-26T10:44:00.176 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2024-02-26T10:44:00.207 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-02-26T10:44:00.278 INFO:teuthology.orchestra.console.smithi067:Power off completed 2024-02-26T10:44:00.308 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-02-26T10:44:00.414 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 client.1-kernel-sha1: 7c0fbba6c508351b48a34af7de7b074eb29ad0f9 description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} duration: 975.1311078071594 failure_reason: 'Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps)' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f7ce41d0343b44149a953eb69c385139 status: fail success: false 2024-02-26T10:44:00.414 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-26T10:44:00.491 INFO:teuthology.run:FAIL